~tim/scheme-vm

scheme-vm/src d---------
Oops. Don't need that any more.
Fix error when it's on line 1
Show better message on syntax error
Switch back to rust-peg and speed it up!

I was doing some stupid work repeatedly every time atom() was called.
Computing that work ahead-of-time puts rust-peg back on par with pest,
and I like rust-peg much better!
Use pest instead of rust-peg

The code is ugly, needs refactoring, but it works! And it's faster!

BUT, while optimizing this code, I realized I was doing something really
slow with find the line and column indices for each atom. That was
probably a big part of the slowness with my rust-peg implementation. So
I'd like to go back to the rust-peg implementation and see if I can make
some optimizations to get the speed closer to my implementation using pest.
String slice is fine here
Fix parser bug

(foo)(bar) could not be parsed
Store line and column number for better errors
I finally figured it out... It was GC!!!

I mistakenly assumed that Rust was my enemy here... freeing memory that
it no longer "saw", e.g. values passed back to Ruby. In reality, it was
Ruby garbage collecting memory for objects that hadn't yet been stored
in a variable.

One thing that kept me from considering GC as a suspect was that I
assumed that, while in "C" code, Ruby's garbage collector would not
attempt to free memory. This was wrong!

Ruby's GC will run any time Ruby attempts to allocate new memory and
decides it has already allocated a bunch of objects -- it runs a GC to
possibly free up existing objects rather than just consume more memory.
I had assumed that since I was in Rust (er.. "C" extension code), the GC
would wait until control returned to Ruby. Well, duh! Any time my Rust
code called a `rb_*` function, *that's* Ruby code!

I read about lots of tricks for telling Ruby not to GC objects created
from within a C extension, including marking objects as global
variables, registering them with GC, etc. In the end, I decided it was
best to just disable GC before building the AST and re-enabling it just
before passing the AST back to Ruby-land.

Whew!

Funny thing is: this bug took me a full year to understand! I've been
toying with this bug off-and-on since Feb 13, 2017! Yay for persistence!

:-D
Still trying to figure out where my memory is getting free'd
Attempt to fix use-after-free
Deep dup AST until I can figure out the bug

There is a use-after-free error I believe.
Hook Rust parser into Ruby

I have a bunch of failing tests, so I've focused the parser test for now.
Make parser use Atom instead of plain strings

This breaks the rust tests because the constant VM is unavailable when
the parser is loaded directly from tests. Should be fixable.
Improve API for parser rust code; return better exception
Return ruby object (ast) from rust parser
Use ruby-sys to integrate better with Ruby
Playing around with ruby-sys
Use FFI instead of ruru
Next