~hokiegeek/tiddlybucket

a go port of the TiddlyWeb backend that comunicates directly with Google Storage or AWS S3 buckets
corrects the default port
updates the project overview and reason to be
corrects huge bug with how tags are handled

clone

read-only
https://git.sr.ht/~hokiegeek/tiddlybucket
read/write
git@git.sr.ht:~hokiegeek/tiddlybucket

You can also use your local clone with git send-email.

#tiddlybucket

builds.sr.ht status docs tiddlywiki supported version

This is my attempt at solving my specific use case where I want to run my TiddlyWiki5 instances with the tiddlers persisted in a Google Cloud or AWS bucket.

I like having each tiddler as a separate file, in the way that the TiddlyWiki5 server supports, so that I can create and edit from a terminal. Unfortunately, mounting a bucket to the filesystem incurred a large performance hit and scheduled synchronization tasks lead to race conditions and unintended overwrites.

This tool replicates the TiddlyWeb backend API and can read and write the tiddler files to a local directory like the canonical TiddlyWiki5 app. But, in addition, it can do the same with a given a Google Cloud Storage bucket or AWS S3 bucket.

#grab a copy

#already built binaries

how to install
go install git.sr.ht/~hokiegeek/tiddlybucket/cmd/tiddlybucket@latest
docker pull registry.gitlab.com/hokiegeek/tiddlybucket

note: a "latest" image is not published for the container images

#build it yourself

git clone https://git.sr.ht/~hokiegeek/tiddlybucket
cd tiddlybucket
make index
go install ./...

make index will clone a copy of TiddlyWiki5 and builds an index.html with with a custom tiddlywiki.info.

#how to run

#environment variables

variable store type notes
WIKI_LOCATION all
PORT n/a
DEBUG_LEVEL n/a
GOOGLE_APPLICATION_CREDENTIALS gs required if using a Google Cloud Storage bucket
AWS_ACCESS_KEY_ID s3 required if using an AWS S3 bucket
AWS_SECRET_ACCESS_KEY s3 required if using an AWS S3 bucket
AWS_REGION s3 required if using an AWS S3 bucket

#command-line options

command-line options override their respective environment variables.

flag option default
-port NUM 8080
-debug_level trace|debug|info|warn|error|fatal info
-credentials_file PATH none
-readers comma-separated list of usernames none
-writers comma-separated list of usernames none
#positional arguments

Only one positional argument is expected. All others are ignored. The first positional argument is interpreted as being the location of the wiki files. A full URI is required (see below). The type of storage is determined from the URI scheme. The structure of the location is the same as with the TiddlyWiki5 server where it includes a directory named "tiddlers".

[gs|s3|file]://[WIKI_PATH]

This overrides the WIKI_LOCATION variable.

⚠️ This or WIKI_LOCATION must be provided or the server will panic upon startup.

#examples

#Google Cloud Storage bucket with access restrictions
env GOOGLE_APPLICATION_CREDENTIALS=/etc/secrets/ghammond.json \
    tiddlybucket \
    --credentials_file=personnel.csv --readers='(anon)' --writers='(authenticated)' \
    gs://sgc/sg-1/misson-logs
#AWS S3 bucket with minimal logging
env AWS_ACCESS_KEY_ID=xxxxx \
    AWS_SECRET_ACCESS_KEY=uuuuu \
    AWS_REGION=us-west-1 \
    tiddlybucket --port=9876 --debug_level=warn \
    s3://sgc/known-races/furlings
#local filesystem
tiddlybucket file:///Users/joneill/fishing

#deployment suggestions

#Google Cloud

Take a look at the terraform I am using for my main wiki. It runs tiddlybucket as a Google Cloud Run (kubernetes) service: https://git.sr.ht/~hokiegeek/wiki/tree/main/item/infrastructure/tf-cloudrun/main.tf

#AWS

TODO

#TiddlyWiki5 server feature parity

My goal with this project is to duplicate the functionality and features of the TiddlyWiki5 server that I use with my wiki. As such, not everything is supported.

feature progress notes
reading tiddler files from local disk
writing tiddler files to local disk
reading and writing meta tiddlers might need to revisit how text vs binary files are identified
TiddlyWeb API support partial the skinny list endpoint ignores the filter query
preloading tiddlers in store on HTML file
serving $:/favicon.ico
raw markup support partial wikified markup is not currently supported (see TODO #1)
basic auth the endpoint /login-basic behaves as the TW5 server
credentials file with readers and writers
header credentials

#contributing

A patchset is always welcomed!

If you want to submit a bug report or enhancement, the issue tracker is: https://todo.sr.ht/~hokiegeek/tiddlybucket

#dependencies

There isn't much in the way of direct, third-party, dependencies. The tabel below lists the highlights.

package use
github.com/spf13/viper configuration management (env vars and command-line)
github.com/go-chi/chi/v5 a nicely idiomatic and minimal router
github.com/rs/zerolog structured logging
cloud.google.com/go/storage API to directly access a Google Cloud Storage bucket
github.com/aws/aws-sdk-go API to directly access an AWS S3 bucket
#vulnerability scanning

dependencies are scanned for vulnerabilities using Nancy on every CI build.

#licensing

The index.html file that is generated upon build and distributed in the container releases is copyrighted to UnaMesa Association. The full license can be seen here: https://tiddlywiki.com/#License

All of the code within this project - aside from the generated index.html - is licensed under an MIT license. The full license can be seen here: https://git.sr.ht/~hokiegeek/tiddlybucket/tree/main/item/LICENSE