~hokiegeek/tiddlybucket

#tiddlybucket

This is my attempt at solving my specific use case where I want to run my TiddlyWiki5 instances with the tiddlers persisted in a Google Cloud or AWS bucket.

I like having each tiddler as a separate file, in the way that the TiddlyWiki5 server supports, so that I can create and edit from a terminal. Unfortunately, mounting a bucket to the filesystem incurred a large performance hit and scheduled synchronization tasks lead to race conditions and unintended overwrites.

This tool replicates the TiddlyWeb backend API and can read and write the tiddler files to a local directory like the canonical TiddlyWiki5 app. But, in addition, it can do the same with a given a Google Cloud Storage bucket or AWS S3 bucket.

#grab a copy

#already built binaries

how to install
`go install git.sr.ht/~hokiegeek/tiddlybucket/cmd/tiddlybucket@latest`
`docker pull registry.gitlab.com/hokiegeek/tiddlybucket`

^{note: a "latest" image is not published for the container images}

#build it yourself

git clone https://git.sr.ht/~hokiegeek/tiddlybucket
cd tiddlybucket
make index
go install ./...

make index will clone a copy of TiddlyWiki5 and builds an index.html with with a custom tiddlywiki.info.

#how to run

#environment variables

variable	store type	notes
`WIKI_LOCATION`	all
`PORT`	n/a
`DEBUG_LEVEL`	n/a
`GOOGLE_APPLICATION_CREDENTIALS`	gs	required if using a Google Cloud Storage bucket
`AWS_ACCESS_KEY_ID`	s3	required if using an AWS S3 bucket
`AWS_SECRET_ACCESS_KEY`	s3	required if using an AWS S3 bucket
`AWS_REGION`	s3	required if using an AWS S3 bucket

#command-line options

command-line options override their respective environment variables.

flag	option	default
`-port`	NUM	8080
`-debug_level`	trace\|debug\|info\|warn\|error\|fatal	info
`-credentials_file`	PATH	none
`-readers`	comma-separated list of usernames	none
`-writers`	comma-separated list of usernames	none

#positional arguments

Only one positional argument is expected. All others are ignored. The first positional argument is interpreted as being the location of the wiki files. A full URI is required (see below). The type of storage is determined from the URI scheme. The structure of the location is the same as with the TiddlyWiki5 server where it includes a directory named "tiddlers".

[gs|s3|file]://[WIKI_PATH]

This overrides the WIKI_LOCATION variable.

⚠️ This or WIKI_LOCATION must be provided or the server will panic upon startup.

#examples

#Google Cloud Storage bucket with access restrictions

env GOOGLE_APPLICATION_CREDENTIALS=/etc/secrets/ghammond.json \
    tiddlybucket \
    --credentials_file=personnel.csv --readers='(anon)' --writers='(authenticated)' \
    gs://sgc/sg-1/misson-logs

#AWS S3 bucket with minimal logging

env AWS_ACCESS_KEY_ID=xxxxx \
    AWS_SECRET_ACCESS_KEY=uuuuu \
    AWS_REGION=us-west-1 \
    tiddlybucket --port=9876 --debug_level=warn \
    s3://sgc/known-races/furlings

#local filesystem

tiddlybucket file:///Users/joneill/fishing

#deployment suggestions

#Google Cloud

Take a look at the terraform I am using for my main wiki. It runs tiddlybucket as a Google Cloud Run (kubernetes) service: https://git.sr.ht/~hokiegeek/wiki/tree/main/item/infrastructure/tf-cloudrun/main.tf

#AWS

TODO

#TiddlyWiki5 server feature parity

My goal with this project is to duplicate the functionality and features of the TiddlyWiki5 server that I use with my wiki. As such, not everything is supported.

feature	progress	notes
reading tiddler files from local disk	✅
writing tiddler files to local disk	✅
reading and writing meta tiddlers	✅	might need to revisit how text vs binary files are identified
TiddlyWeb API support	partial	the skinny list endpoint ignores the filter query
preloading tiddlers in store on HTML file	✅
serving $:/favicon.ico	✅
raw markup support	partial	wikified markup is not currently supported (see TODO #1)
basic auth	✅	the endpoint `/login-basic` behaves as the TW5 server
credentials file with readers and writers	✅
header credentials

#contributing

A patchset is always welcomed!

If you want to submit a bug report or enhancement, the issue tracker is: https://todo.sr.ht/~hokiegeek/tiddlybucket

#dependencies

There isn't much in the way of direct, third-party, dependencies. The tabel below lists the highlights.

package	use
github.com/spf13/viper	configuration management (env vars and command-line)
github.com/go-chi/chi/v5	a nicely idiomatic and minimal router
github.com/rs/zerolog	structured logging
cloud.google.com/go/storage	API to directly access a Google Cloud Storage bucket
github.com/aws/aws-sdk-go	API to directly access an AWS S3 bucket

#vulnerability scanning

dependencies are scanned for vulnerabilities using Nancy on every CI build.

#licensing

The index.html file that is generated upon build and distributed in the container releases is copyrighted to UnaMesa Association. The full license can be seen here: https://tiddlywiki.com/#License

All of the code within this project - aside from the generated index.html - is licensed under an MIT license. The full license can be seen here: https://git.sr.ht/~hokiegeek/tiddlybucket/tree/main/item/LICENSE

refs

clone