7ffceb3524947ac7bafb7394d584240d90f8623a — Zach Smith 4 months ago efb83c3
Add to manual: extending Bagatto
2 files changed, 146 insertions(+), 1 deletions(-)

M bagatto.janet
M MANUAL.md => MANUAL.md +145 -1
@@ 189,7 189,7 @@ process files in more interesting ways.

repl:28:> (bagatto/eval-loader (bagatto/slurp-* "demo/posts/*.md"))
@[("demo/posts/post.md" @"## A Post That You Might Be Interested In.\n\nI'm writing this in *Markdown*. Therefore, in my page template, I'll\ncall `bagatto/markdown->html` in order to render this markdown into\nHTML.\n") ("demo/posts/post2.md" @"title: A Really Good Title\nstatus: post\n---\n\n## The Other Post on My Blog.\n\nThis post is very similar to the other post. \n\nHowever, it contains footnotes[^1].\n\n[^1]: Bagatto has built-in support for Markdown via the\n    [moondown](https://github.com/joy-framework/moondown)\n    library. However, moondown doesn't have support for footnotes, so\n    we should use multimarkdown to render this instead. Bagatto has\n    the `mmarkdown->html` (notice the extra `m`!) function which\n    shells out to the multimarkdown application. So if you install\n    that, you can take advantage of more features.\n\n    Of course, if you want to use some other markup or text processing\n    application, you can write your own function, either in your index\n    or a helper module, and use that instead.\n")]
@[("demo/posts/post.md" @"## A Post That You Might Be Interested In...") ...]

The return value of the loader for each matching file is a two-tuple

@@ 608,3 608,147 @@ template. Evaluating the site we get the same thing:
repl:7:> (bagatto/eval-site site {"topic" "Web Design"})
@[(:write "out.txt" @"I am known for my Web Design skills.\n")]

## Filters

A site spec with an `:each` can include a `:filter` attribute,
too. This can be any predicate function which takes the site and an
individual item from the spec's site selector, and returns true or
false. If the return value is false, the site spec will skip that elements.

This can be very useful when handling an input of mixed files. For
instance, with a `static/` directory that contains both CSS and
supplementary HTML files, we might want to have different render steps
for each. We could then write two site specs, that both take that data
entry in their `:each`, but have different `:filter` attributes (we
could also have written two different wildcards in two different data
specs, but hopefully you get my point).

# Extending Bagatto

Bagatto bills itself as a "transparent" static site generator. By this
we mean: we should favor *first-class functions* over *configuration*,
and *native terms and data structures* over *indirect control flow*
whenever possible.

Here's a simple example: Bagatto creates files by combining a *file
path* with some *file contents*. The values that can go in the `:out`
section of a site specification can either be strings, or functions
which produce strings. 

We might be tempted as application authors to introduce a layer of
abstraction in front of the render process and ask the user to specify
the *name* of a render function built into Bagatto. This would provide
a simple, convenient DSL. Unfortunately, it has the very unfortunate
side effect of effectively walling off that function from a site
author. If---when---the author needs to understand what specifically
is being passed into the render function, or needs to tweak its output
slightly, they're out of luck. The logic that reads this name,
translates it into a render function, calls the function with some
inputs and uses the output is all stuck within the belly of Bagatto
and the author might need to recompile the whole application to get
into it. 

Similarly, if they want to introduce a new renderer---a new template
language for instance---they can only do so by introducing the
function directly into Bagatto, giving it a name, and then passing the
name in a site specification.

Therefore we keep the operation of the renderer within inspection of
the author. By specifying a literal function, we can easily wrap other
functions and debug their output or change it. Similarly, we do
attempt to offer an author the same level of convenience as the above
DSL; but instead of offering them the ability to name a function that
we control, we offer them the ability to call a function that outputs
the renderer function itself, so that they still have access to its
inputs and outputs.

Thus, we have a pretty straightforward way to write our own loaders,
attributes, path-generator and renderer functions.

Each of the below entries will have a typespec describing the
signature of the functions that can be implemented. This isn't
meaningful Janet, but hopefully gives a succinct picture of the types
that will be meaningful.

## Loaders

(let [element (or 'source-path '(source-path file-contents))]
 (defn loader [] 
  (or '{:each (element ...)} '{:some element)}))

The `:src` attribute in a data spec can take a 0-arity function which,
when called, returns one of two types of values:

### `{:each values}`

`values` is any indexable data structure, the elements of which are
either two-tuples or single values. Two-tuples will be treated by the
base attribute parser as `[source-path file-contents]`. Single values
will be treated as `source-path` only.

### `{:some value}`

`value` is a single instance of the above value type: either a
two-tuple or a single path value.

We could, for instance, write a custom loader function that accepted a
URL, made a web request, and returned a (file-url file-contents)

## Parsers

(defn parser [contents attrs] attrs)

`:attrs` can take any parser function. The purpose of a parser is to
transform the individual outputs of a data loader into an attributes
table. There are two attributes that are guaranteed to be present when
the parser is called, `:path` and `:contents`. A parser function
shouldn't remove either of these attributes, but can use them to
generate new ones. For instance, if `contents` is unparsed Markdown
with YAML frontmatter, then a parser function could extract metadata
from the frontmatter and return an updated attributes table with those
arbitrary metadata.

`contents` and the `:contents` attribute can be expected to be
identical, and the former is provided as a convenience.

An example of a custom parser would be one that shelled out to
[Asciidoctor](https://asciidoctor.org/) to extract attributes from an
asciidoc document.

## Path Generators

(defn each-parser [data item] path)
(defn some-parser [data] path)

In the site specification, `:dest` can take any function which returns
a file path string. If the spec has an `:each`, the generator function
should take the site data and the individual item as arguments, and
return the destination path for the individual item.

Otherwise, it should take the site data as a single argument and
return the destination path for its entry output.

## Renderers

(defn each-renderer [data item] file-contents)
(defn some-renderer [data] file-contents)

`:out` takes any renderer function---these work along exactly the
same lines as path generators. If the site spec has an `:each`, the
function should take two arguments, otherwise it should take one. The
return value of the function will be written directly to the file path
in its site spec.

Following from the parser example above, an example custom renderer
could take an asciidoc document and shell out to Asciidoctor to render
it into HTML.

M bagatto.janet => bagatto.janet +1 -0
@@ 48,6 48,7 @@
  (setdyn :bagatto-output-dir dir))