~poptart/stagnant: Started refactoring for public consumption. This will take some time and will absolutely need some squashing

22 files changed, 325 insertions(+), 0 deletions(-)

A .gitignore
A README.md
A doc/examples/complex/site/403.md
A doc/examples/complex/site/404.md
A doc/examples/complex/site/50x.md
A doc/examples/complex/site/index.md
A doc/examples/complex/stages/00-depends.sh
A doc/examples/complex/stages/01-slides.sh
A doc/examples/complex/stages/02-http.sh
A doc/examples/complex/stages/03-removemd.sh
A doc/examples/complex/stages/04-minify.sh
A doc/examples/complex/stages/scripts/clean.sh
A gen.sh
A site/403.md
A site/404.md
A site/50x.md
A site/index.md
A stages/00-depends.sh
A stages/01-http.sh
A stages/02-removemd.sh
A stages/03-minify.sh
A stages/scripts/clean.sh

A  => .gitignore +10 -0

@@ 1,10 @@
+temp/
+build/
+util/src/
+hmdtohtml
+backup/
+test/
+genz.sh
+buildz/
+.*.sw*
+expr/

A  => README.md +117 -0

@@ 1,117 @@
+
+
+A pipeline based static "site" generator focused on allowing flexible composability using built in UNIX tools. Fundamentally, this project is simple and only enforces a simple set of rules, filesystem structures, and a small subset of exposed environment variables.
+
+Reasoning
+---------
+
+I write my entire life in markdown, and things that aren't in markdown get generators to turn them into markdown for my own consumption. While some static site generators have a common markdown backend I found that often the generated code was hard to pipeline into other outputs, such as Gopher. 
+
+Additionally, my CI of choice is Laminar-CI which takes the opinionated stance of "just use job shell scripts". I found this rather freeing from implementation specific DSLs or janky non-reproducible web interface configuration management.
+
+I wanted something that allowed me to write a simple step-by-step sychronous pipeline using POSIX.1-2008 shell scripts and allow me to write the pipeline steps in any language I desired. Use the tools you already have.
+
+Example Structure
+-----------------
+
+The only directories that are fundamental to the creation of a "site" is the `site` directory which contains the initial format of the site and the functions (`stages`) directory that contains the staged pipeline pieces of code. It is also highly suggested to create a `util` directory that contains executables or scripts used during pipelining.
+
+My personal project root looks like this (directories marked with `<` are required for the generator to work by default):
+
+```
+.
+|-- README.md
+|-- build/
+|-- stages/	<
+|-- gen.sh*
+|-- prod/
+|-- site/	<
+|-- tmpl/
+`-- util/	<
+```
+
+```
+.
+|-- build/
+|   |-- 1581996075/
+|   |-- 1581996127/
+|   |-- 1581996133/
+|   `-- latest@ -> 1581996133
+|-- stages/
+|   |-- 00-depends.sh*
+|   |-- 01-slides.sh*
+|   |-- 02-http.sh*
+|   |-- 03-removemd.sh*
+|   |-- 04-minify.sh*
+|   `-- scripts/
+|       |-- clean.sh*
+|       `-- deploy.sh*
+|-- gen.sh*
+|-- prod/
+|   |-- 403.html
+|   |-- 404.html
+|   |-- 50x.html
+|   |-- a/
+|   |-- d/
+|   |-- e/
+|   |-- f/
+|   |-- favicon.ico
+|   |-- favicon.png
+|   |-- i/
+|   |-- id.html
+|   |-- index.html
+|   |-- keybase.txt
+|   |-- keys.txt
+|   |-- n/
+|   |-- p/
+|   |-- pub.key
+|   |-- resume.pdf
+|   |-- rss.xml
+|   |-- style.css
+|   |-- t/
+|   `-- talks.html
+|-- site/
+|   |-- 403.md
+|   |-- 404.md
+|   |-- 50x.md
+|   |-- f/
+|   |-- favicon.ico
+|   |-- favicon.png
+|   |-- i/
+|   |-- id.md
+|   |-- index.md
+|   |-- keys.txt
+|   |-- p/
+|   |-- pub.key
+|   |-- rss.xml
+|   |-- t/
+|   `-- talks.md
+|-- tmpl/
+|   |-- footer.html
+|   |-- header.html
+|   |-- meta.html
+|   |-- rss.xml
+|   |-- rss_item.xml
+|   |-- style.css
+|   `-- template.html
+`-- util/
+    |-- hdev*
+    |-- hmdtohtml*
+    |-- hosakahashi*
+    `-- minify*
+
+``` 
+
+Exposed Variables
+-----------------
+
+The following variables are exposed to all running stage executables/scripts and to any generator scripts:
+
+* `_FUNCDIR`
+* `_SCRIPTDIR`
+* `_BUILDROOT`
+* `_UTIL`
+* `_PROD`
+* `_ID`
+* `_BUILD`
+* `_DEPLOY`

A  => doc/examples/complex/site/403.md +1 -0

@@ 1,1 @@
+# ACCESS DENIED

A  => doc/examples/complex/site/404.md +1 -0

@@ 1,1 @@
+# ☠ Page does not exist...

A  => doc/examples/complex/site/50x.md +1 -0

@@ 1,1 @@
+# ☠ BAN HAMMER

A  => doc/examples/complex/site/index.md +20 -0

@@ 1,20 @@
+//META:title Hosaka Corp - Shell Distributors
+//META:description Index of Research and Blog Posts
+//META:style /style.css
+<div id=post>0x0B <a href=/p/ssh-pki.html>creating a PKI for OpenSSH</a></div>
+<div id=post>0x0A <a href=/p/slides.html>hosakahashi: takahashi in pure css</a></div>
+<div id=post>0x09 <a href=/p/voidmap-pentest.html>using voidmap for penetration test project management</a></div>
+<div id=post>0x08 <a href=/p/ldpreload-hashcat.html>LD_PRELOAD, hashcat, and bad ideas</a></div>
+<div id=post>0x07 <a href=/p/riscv-asm.html>RISC-V assembly and shellcode creation series</a>
+<ul>
+ <li>Part 1: <a href="/p/riscv-asm-1.html">the basics</a></li>
+ <li>Part 2: <a href="/p/riscv-asm-2.html">some complexity</a></li>
+ <li>...</li>
+</div>
+<div id=post>0x06 <a href=/p/systemd-user-msf.html>systemd user persistence in metasploit</a></div>
+<div id="post">0x05 <a href="/t/ipv6.html">talk: ipv6 for pentesters</a></div>
+<div id="post">0x04 <a href="/p/go-spark.html">Go+SPARK</a></div>
+<div id="post">0x03 <a href="/p/wireguard-basic.html">wireguard quickstart</a></div>
+<div id="post">0x02 <a href="/p/systemd-user.html">abusing systemd user services</a></div>
+<div id="post">0x01 <a href="/p/unicode-rot.html">unicode rotation cipher</a></div>
+<div id="post">0x00 <a href="/p/tinc-1.1-overview.html">tinc 1.1 overview</a></div>

A  => doc/examples/complex/stages/00-depends.sh +7 -0

@@ 1,7 @@
+#!/bin/sh -e
+[ ! -z "$_ROOT" ] || exit 1
+[ -f "$_ROOT/util/minify" ] || exit 2
+[ -f "$_ROOT/util/smu" ] || exit 2
+[ -f "$_ROOT/util/hmdtohtml" ] || exit 2
+[ -f "$_ROOT/util/hosakahashi" ] || exit 2
+[ -d "$_ROOT/site/" ] || exit 2

A  => doc/examples/complex/stages/01-slides.sh +6 -0

@@ 1,6 @@
+#!/bin/sh -e
+[ ! -z "$_BUILD" ] || exit 2
+find "${_BUILD}" -name '*.sent' | while IFS= read -r file
+do
+       "$_UTIL"/hosakahashi < "$file" > "$(echo "$file" | sed 's/\.sent/\.html/')"
+done

A  => doc/examples/complex/stages/02-http.sh +15 -0

@@ 1,15 @@
+#!/bin/sh
+META="$(cat "$_ROOT"/tmpl/meta.html)"
+HEADER="$(cat "$_ROOT"/tmpl/header.html)"
+FOOTER="$(cat "$_ROOT"/tmpl/footer.html)"
+export TMPL_TITLE="⊕ hosaka corp"
+export TMPL_DESCR="Digital voodoo, cyber-witchcraft, and shell conjuring."
+export TMPL_STYLE="/style.css"
+find "${_BUILD}" -name '*.md' | while IFS= read -r file; do
+        # Search for template meta tags
+        [ -n "$(grep '//META:style' "$file")" ] && TMPL_STYLE="$(grep '//META:style' "$file" | sed -e 's%//META:style %%')"
+        [ -n "$(grep '//META:title' "$file")" ] && TMPL_TITLE="$(grep '//META:title' "$file" | sed -e 's%//META:title %%')"
+        [ -n "$(grep '//META:description' "$file")" ] && TMPL_DESCR="$(grep '//META:description' "$file" | sed -e 's%//META:description %%')"
+        BODY="$(grep -v '//META:' "$file" | "$_UTIL"/hmdtohtml | "$_UTIL"/minify --type html)"
+        echo "$META$HEADER$BODY$FOOTER" | sed -e "s/{{title}}/$TMPL_TITLE/g" -e "s/{{description}}/$TMPL_DESCR/g" -e "s%{{style}}%$TMPL_STYLE%g" > "$(echo "$file" | sed 's/\.md/\.html/')"
+done

A  => doc/examples/complex/stages/03-removemd.sh +2 -0

@@ 1,2 @@
+#!/bin/sh
+find "$_BUILD" -iname '*.md' -exec rm {} \;

A  => doc/examples/complex/stages/04-minify.sh +3 -0

@@ 1,3 @@
+#!/bin/sh
+"$_UTIL"/minify --type css < "$_ROOT"/tmpl/style.css > "$_BUILD"/style.css
+find "$_BUILD" -type f -name '*.html' -exec sh -c 'cat "$1" | $_UTIL/minify --type html -o "$1"' -- {} \;

A  => doc/examples/complex/stages/scripts/clean.sh +6 -0

@@ 1,6 @@
+#!/bin/sh -e
+NUM="$(find "${_BUILDROOT}"/* -type d -print0 -maxdepth 1 -prune | xargs -0 -I {} basename {} | wc -l | tr -d ' ')"
+if [ $NUM -gt 5 ]; then
+	#Extra because of the symlink latest
+	find "${_BUILDROOT}"/* -type d -print0 -maxdepth 1 -prune | xargs -0 -I {} basename {} | sort -n -r | tail -n +6 | xargs -I {a} find "${_BUILDROOT}"/{a} -type d -maxdepth 1 -prune -exec rm -rf "{}" \;
+fi

A  => gen.sh +80 -0

@@ 1,80 @@
+#!/bin/sh -e
+export _ORIGIN
+export _ROOT
+
+_ORIGIN="$(pwd)"
+_ROOT=$(dirname "$(readlink -f "$0")")
+
+export _FUNCDIR="$_ROOT/stages"
+export _SCRIPTDIR="$_ROOT/stages/scripts"
+export _BUILDROOT="$_ROOT/build"
+export _UTIL="$_ROOT/util"
+export _PROD="$_ROOT/prod"
+export _ID=""
+export _BUILD=""
+export _DEPLOY=0
+_ID="$(date +%s)"
+_BUILD="$_BUILDROOT/$_ID"
+
+HELPDIAG="[-p]\\n   -p: tag and stage for deployment\\n   -s [script]: list scripts with no arguments, or run a script"
+
+runscript() {
+	if [ ! -n "$1" ]; then
+		find "$_SCRIPTDIR" -type f -name '*.sh' -print0 | xargs -0 -I {} basename {} | sed 's/\.sh//g'
+		exit 0
+	fi
+	COMM="$(printf "%s/%s.sh" "$_SCRIPTDIR" "$1")"
+	if [ ! -f "$COMM" ]; then
+		printf "Script command '%s' not found...\\n" "$1" >&2
+		exit 1
+	fi
+	find "$COMM" -type f -print0 | xargs -0 -I {} sh -c "{}"
+}
+
+#TODO build locking
+while :; do
+        case $1 in
+                -h|-\?|--help)
+                        printf "%s %b\\n" "$0" "$HELPDIAG"
+                        exit
+                        ;;
+		-p)
+			_DEPLOY=1
+			;;
+		-s)
+			shift
+			runscript "$1"
+			exit
+			;;
+                -?*)
+                        printf 'Unknown option: %s\n' "$1" >&2
+                        exit
+                        ;;
+                *)
+                        break
+        esac
+	shift
+done
+
+cd "$_ROOT" || exit 3
+mkdir -p "$_BUILD"
+printf "%s\\n" "$_ID"
+cp -r "${_ROOT}"/site/* "$_BUILD"
+[ -h "$_BUILDROOT/latest" ] && rm "$_BUILDROOT/latest"
+ln -s "$_BUILD" "$_BUILDROOT/latest"
+cd "$_BUILD" || exit 3
+find "$_FUNCDIR/" -type f -name '[0-9]*-*.sh' -print0 | sort -n -z | xargs -0 -r sh -c '"$@" || exit 255' 
+
+# This is the old order rendering function that was based around staged execution. In reality just making the "stages" numeric makes more sense
+#export _ORDER="pre:render:post"
+#for f in $(printf "%s\\n" "$_ORDER" | tr ':' '\n'); do 
+#	find "$_FUNCDIR/$f" -type f -print0 -name '[0-9]*-*.sh' | xargs -0 -I {} sh -c "{}"
+#done 
+
+if [ "$_DEPLOY" -ne 0 ]; then
+	[ -d "$_PROD" ] && rm -rf "$_PROD"
+	mkdir -p "$_PROD"
+	cp -r "$_BUILDROOT/latest/"* "$_PROD"
+fi
+
+cd "$_ORIGIN" || exit 3

A  => site/403.md +1 -0

@@ 1,1 @@
+# ACCESS DENIED

A  => site/404.md +1 -0

@@ 1,1 @@
+# ☠ Page does not exist...

A  => site/50x.md +1 -0

@@ 1,1 @@
+# ☠ BAN HAMMER

A  => site/index.md +20 -0

@@ 1,20 @@
+//META:title Hosaka Corp - Shell Distributors
+//META:description Index of Research and Blog Posts
+//META:style /style.css
+<div id=post>0x0B <a href=/p/ssh-pki.html>creating a PKI for OpenSSH</a></div>
+<div id=post>0x0A <a href=/p/slides.html>hosakahashi: takahashi in pure css</a></div>
+<div id=post>0x09 <a href=/p/voidmap-pentest.html>using voidmap for penetration test project management</a></div>
+<div id=post>0x08 <a href=/p/ldpreload-hashcat.html>LD_PRELOAD, hashcat, and bad ideas</a></div>
+<div id=post>0x07 <a href=/p/riscv-asm.html>RISC-V assembly and shellcode creation series</a>
+<ul>
+ <li>Part 1: <a href="/p/riscv-asm-1.html">the basics</a></li>
+ <li>Part 2: <a href="/p/riscv-asm-2.html">some complexity</a></li>
+ <li>...</li>
+</div>
+<div id=post>0x06 <a href=/p/systemd-user-msf.html>systemd user persistence in metasploit</a></div>
+<div id="post">0x05 <a href="/t/ipv6.html">talk: ipv6 for pentesters</a></div>
+<div id="post">0x04 <a href="/p/go-spark.html">Go+SPARK</a></div>
+<div id="post">0x03 <a href="/p/wireguard-basic.html">wireguard quickstart</a></div>
+<div id="post">0x02 <a href="/p/systemd-user.html">abusing systemd user services</a></div>
+<div id="post">0x01 <a href="/p/unicode-rot.html">unicode rotation cipher</a></div>
+<div id="post">0x00 <a href="/p/tinc-1.1-overview.html">tinc 1.1 overview</a></div>

A  => stages/00-depends.sh +7 -0

@@ 1,7 @@
+#!/bin/sh -e
+[ ! -z "$_ROOT" ] || exit 1
+[ -f "$_ROOT/util/minify" ] || exit 2
+[ -f "$_ROOT/util/smu" ] || exit 2
+[ -f "$_ROOT/util/hmdtohtml" ] || exit 2
+[ -f "$_ROOT/util/hosakahashi" ] || exit 2
+[ -d "$_ROOT/site/" ] || exit 2

A  => stages/01-http.sh +15 -0

@@ 1,15 @@
+#!/bin/sh
+META="$(cat "$_ROOT"/tmpl/meta.html)"
+HEADER="$(cat "$_ROOT"/tmpl/header.html)"
+FOOTER="$(cat "$_ROOT"/tmpl/footer.html)"
+export TMPL_TITLE="⊕ hosaka corp"
+export TMPL_DESCR="Digital voodoo, cyber-witchcraft, and shell conjuring."
+export TMPL_STYLE="/style.css"
+find "${_BUILD}" -name '*.md' | while IFS= read -r file; do
+        # Search for template meta tags
+        [ -n "$(grep '//META:style' "$file")" ] && TMPL_STYLE="$(grep '//META:style' "$file" | sed -e 's%//META:style %%')"
+        [ -n "$(grep '//META:title' "$file")" ] && TMPL_TITLE="$(grep '//META:title' "$file" | sed -e 's%//META:title %%')"
+        [ -n "$(grep '//META:description' "$file")" ] && TMPL_DESCR="$(grep '//META:description' "$file" | sed -e 's%//META:description %%')"
+        BODY="$(grep -v '//META:' "$file" | "$_UTIL"/hmdtohtml | "$_UTIL"/minify --type html)"
+        echo "$META$HEADER$BODY$FOOTER" | sed -e "s/{{title}}/$TMPL_TITLE/g" -e "s/{{description}}/$TMPL_DESCR/g" -e "s%{{style}}%$TMPL_STYLE%g" > "$(echo "$file" | sed 's/\.md/\.html/')"
+done

A  => stages/02-removemd.sh +2 -0

@@ 1,2 @@
+#!/bin/sh
+find "$_BUILD" -iname '*.md' -exec rm {} \;

A  => stages/03-minify.sh +3 -0

@@ 1,3 @@
+#!/bin/sh
+"$_UTIL"/minify --type css < "$_ROOT"/tmpl/style.css > "$_BUILD"/style.css
+find "$_BUILD" -type f -name '*.html' -exec sh -c 'cat "$1" | $_UTIL/minify --type html -o "$1"' -- {} \;

A  => stages/scripts/clean.sh +6 -0

@@ 1,6 @@
+#!/bin/sh -e
+NUM="$(find "${_BUILDROOT}"/* -type d -print0 -maxdepth 1 -prune | xargs -0 -I {} basename {} | wc -l | tr -d ' ')"
+if [ $NUM -gt 5 ]; then
+	#Extra because of the symlink latest
+	find "${_BUILDROOT}"/* -type d -print0 -maxdepth 1 -prune | xargs -0 -I {} basename {} | sort -n -r | tail -n +6 | xargs -I {a} find "${_BUILDROOT}"/{a} -type d -maxdepth 1 -prune -exec rm -rf "{}" \;
+fi