f2bc6e3f490f6c2e22378d29386df510578e213b — Stephen Brennan 2 years ago 1e02ce1
Advanced topics docs. Closes #8
4 files changed, 75 insertions(+), 2 deletions(-)

A doc/advanced-gc.rst
A doc/advanced-iterator.rst
M doc/advanced-types.rst
M doc/advanced.rst
A doc/advanced-gc.rst => doc/advanced-gc.rst +29 -0
@@ 0,0 1,29 @@
Garbage Collection

Garbage collection is implemented in Lisp using a mark-and-sweep system. In this
system, values are first marked as to whether they are "reachable", and then all
objects which are unreachable can be freed.

To implement this, we need two key components. First, we need a way to mark
reachable objects. Second, we need a way to track all objects so that we can
find the unreachable ones to free. It turns out the second one is pretty easy:
just maintain a linked list of values as they are created. This logic is nicely
handled inside of ``lisp_new()``.

The second one is trickier. My strategy was for every type to implement its own
``expand()`` operation. This is a function which returns an iterator that yields
every reference the object contains. For example, strings and integers yield
empty iterators. However, lists yield their left and right objects (if they
exist). Scopes yield every symbol and then every value stored within them. They
also yield a reference to their parent scope, if it exists.

The mark operation then simply performs a breadth-first search. It starts with a
single value, marks it "black", then uses the expand operation. It goes through
each value, adding all "white" items to the queue, marking them "grey" as it
goes. Then it chooses the next item in the queue and does the same operation,
until the queue is empty.

To do the breadth-first search, we use a "ring buffer" implementation, which
implements a circular, dynamically expanding double-ended queue. It is quite
simple and useful. It can be found in ``src/ringbuf.c``.

A doc/advanced-iterator.rst => doc/advanced-iterator.rst +41 -0
@@ 0,0 1,41 @@

Iterators are one of several internal APIs used by funlisp but not exposed in
the public API. They are similar to the lazy evaluation constructs supported by
other languages (e.g. generators in Python), but for C. Here is what an iterator
looks like:

.. code:: C

  struct iterator {
      void *ds;        /* the container data structure */
      int index;       /* zero-based index for the iterator */
      int state_int;   /* some state variables that may help */
      void *state_ptr;

      /* do we have a next item? */
      bool (*has_next)(struct iterator *iter);

      /* return the next item (or null) */
      void *(*next)(struct iterator *iter);

      /* free resources held by the iterator */
      void (*close)(struct iterator *iter);

The iterator holds some state, and three operations:

- has next: tell us whether the iterator can be advanced
- next: advance the iterator and return the next value
- close: free any resources held by the iterator

Any data structure could implement these (e.g. an array, a linked list, a hash
table, or others), and could be iterated over by code that knows nothing about
the underlying implementation. Iterators can also be composed in interesting
ways, with some of the helpers given below:

- empty iterator: return an empty iterator
- single value iterator: return an iterator that contains one item
- concatenate: given an array of iterators, yield from each of them until they
  run out

M doc/advanced-types.rst => doc/advanced-types.rst +3 -2
@@ 64,7 64,8 @@ the following operations:
- print: writes a representation of the object to a file, without newline
- new: allocates and initializes a new instance of the object
- free: cleans up and frees an instance
- expand: creates an iterator of ALL references to objects this object owns (see
  the garbage collection documentation)
- expand: creates an :doc:`iterator<advanced-iterator>` of ALL references to
  objects this object owns (see the
  :doc:`garbage collection documentation<advanced-gc>`)
- eval: evaluate this in a scope
- call: call this item in a scope with arguments

M doc/advanced.rst => doc/advanced.rst +2 -0
@@ 8,3 8,5 @@ your own types, or contribute to funlisp.
   :maxdepth: 1