A froghat.ca/2023/11/sourcehut-cloudflare-pages/index.rst => froghat.ca/2023/11/sourcehut-cloudflare-pages/index.rst +274 -0
@@ 0,0 1,274 @@
+Continuous Deployment to Cloudflare Pages from a SourceHut repository
+=====================================================================
+
+:date: Nov 21 2023
+:summary: Modernizing froghat.ca with reverse cloud at the edge using
+ `socat tcp-listen:8080 exec:systemctl start important-things.service`
+
+
+
+I made some JavaScript heavy webshit for viewing crafting recipes for the
+computer game Barotrauma_. I wanted to host the static website somewhere that
+wasn't my own infrastructure; but I'm also very interested in not spending more
+money than I need to right now.
+
+I won't over-excite you with every detail of every decision I made. Here's a summary:
+
+- GitHub has gotten really annoying to use in the last year. Code search
+ requires login; code viewer bloated and crummy on mobile; asking for an OTP
+ every other time I click a button; popups to review/update/confirm personal
+ information every other time I visit the site.
+- Building my project requires scanning game assets. Those assets aren't public
+ [#]_ so I don't want to copy them all over the internet. I want to
+ build and publish the site on my own infrastructure.
+- SourceHut Pages is actually great.
+
+.. aside::
+.. [#]
+ Much of the the content I need is available in the game's dedicated server
+ files, which is distributed on Steam without requiring having bought the
+ full game. Unfortunately, those files were missing visual things I use,
+ like item icons and sprites.
+
+.. aside::
+
+ One day I might host my own projects but I looked into this for two seconds and
+ saw the drama between Gitea and Forgejo and completely lost interest.
+
+
+SourceHut Pages
+---------------
+
+In my opinion, publishing to `SourceHut Pages`_ is very easy.
+
+.. class:: full-width
+.. code:: bash
+
+ tar zc index.html \
+ | curl --oauth2-bearer $srhttoken \
+ -Fcontent=@/dev/stdin \
+ https://pages.sr.ht/publish/sqwishy.srht.site
+
+I don't have to install anything scrupulous with npm. It uses existing things I'm familiar with.
+
+One irritation about SourceHut in general is that they have two
+different things called Personal Access Tokens. One at https://meta.sr.ht/oauth;
+another at https://meta.sr.ht/oauth2. The former is *"legacy"*. But the
+non-legacy API *still* lacks features that exist in the legacy API [#]_.
+
+.. aside::
+.. [#]
+ Accessing or creating repository webhooks, for which there are no pages or
+ forms for on git.sr.ht, requires the legacy API and its tokens while
+ SourceHut Pages uses non-legacy tokens.
+
+They use different
+credentials but both are called Personal Access Tokens.
+They aren't interchangeable and if you use the wrong Personal Access Token, you
+don't always get a clear error back.
+
+----
+
+Another important consideration is that this little webshit I made is super useful and
+everyone is going to love it for sure and it's going to get loads of visits.
+Probably top ten websites on the internet.
+
+But the data and images are about 500 kB with compression. So, should I
+put it on SourceHut Pages if the traffic is absolutely going to cause service
+degradation for other users? I asked on IRC and the only reply was:
+
+ If you have to ask about usage limits, that's generally a good indicator that you might want something different.
+
+ -- someone on #sr.ht
+
+Though, that was exactly the feeling I had that compelled me into asking the question to
+begin with. SourceHut Pages is free -- and not free as in email.
+Isn't *asking* about my obligations part of being a responsible user of the service?
+Or is it evidence that I don't belong. :nowrap:`¯\\_(ツ)_/¯`
+
+I didn't say anything to them,
+but it was like I had just walked into a restaurant and been handed a menu
+with no prices. And, when asking about what the food costs, the server,
+almost looking off and discretely screening themselves from
+second-hand embarrassment, peers at me with judgement and explains that, if I
+have to ask, then I'm at the wrong restaurant. And after a moment of silence for
+my dignity, politely moves on as if, for my sake, I had never asked such a
+fucking stupid question.
+
+
+Cloudflare Pages
+----------------
+
+To upload static assets to Cloudflare Pages, they advise using a JavaScript program
+called `wrangler` (or manually upload zip files to a web form). It's one of those
+programs that vomits emojis in every line of output to keep you engaged.
+It seems very engineered and I couldn't find a simpler way using
+curl so I conceded I'd run wrangler in a container and avoid letting it near
+anything with an emoji allergy.
+
+
+SourceHut Webhooks
+------------------
+
+I'm hosting the source code for my webshit on git.sr.ht, which has
+webhooks. The documentation for this is a little bit wild [#]_; there are two
+main resources:
+
+- `Information about webhooks across SourceHut services generally.
+ <https://man.sr.ht/api-conventions.md#webhooks>`__
+
+- `Information about the webhooks for git.sr.ht specifically.
+ <https://man.sr.ht/git.sr.ht/api.md#repoupdate>`__
+
+.. [#] One of the things they trip you up on is when they mention a username in
+ the endpoints. It's *is in fact* your *canonical name* (your username with a
+ `~` in front). It is *not* the username you log in with *nor* the
+ value of the `name` field returned by \https://meta.sr.ht/api/user/profile.
+
+At each respective page linked above they describe these URL patterns:
+
+- `/api/.../webhooks`
+
+- `/api/:username/repos/:name/...`
+
+Put them together and prefix your username with `~`. For example, this creates a
+webhook for my repository named europan-materialist:
+
+.. class:: full-width
+.. code:: shell
+
+ jo url=https://butt.froghat.ca/europan-materialist \
+ events[]=repo:post-update \
+ | curl --oauth2-bearer $srhttoken \
+ --json @- \
+ https://git.sr.ht/api/~sqwishy/repos/europan-materialist/webhooks
+
+On success, the request's response contains an object representing the webhook you created.
+
+Or we can query the list of webhooks with:
+
+.. class:: full-width
+.. code:: shell
+
+ $ curl --oauth2-bearer $srhttoken \
+ https://git.sr.ht/api/~sqwishy/repos/europan-materialist/webhooks
+
+... which replies ...
+
+.. class:: full-width
+.. code:: json
+
+ {
+ "next": null,
+ "results": [
+ {
+ "id": 28961,
+ "created": "2023-11-17T13:50:06+00:00",
+ "events": [
+ "repo:post-update"
+ ],
+ "url": "https://butt.froghat.ca/europan-materialist"
+ }
+ ],
+ "total": 1,
+ "results_per_page": 50
+ }
+
+
+
+butt.froghat.ca
+---------------
+
+I'm using git.sr.ht to host my project but I want to mirror it to GitHub for
+the social credit. So the webhooks at SourceHut POST to a Python script in
+my *"infrastructure"* that does two things.
+
+- Mirror the repository to GitHub.
+- Build and upload the site to Cloudflare Pages.
+
+This is broken down into three oneshot :hl-purple:`services` in systemd that the script
+activates through a :hl-yellow:`target` in systemd.
+
+.. picture:: systemd-chart.svg
+
+ .. source::
+ :srcset: systemd-chart-dm.svg
+ :media: (prefers-color-scheme: dark)
+
+:hl-purple:`europan-materialist-fetch.service` updates the remote tracking
+branches in a bare/mirror git repository. It is `RequiredBy` and `Before` both the mirror and deploy steps.
+
+:hl-purple:`europan-materialist-mirror.service` pushes that bare/mirror repository to GitHub.
+
+:hl-purple:`europan-materialist-deploy.service` updates its own checkout of the local mirror and builds and runs a container in podman that deploys to Cloudflare Pages.
+
+The :hl-yellow:`europan-materialist-cd.target` is activated over dbus when the Python
+script at butt.froghat.ca gets POSTed a webhook. This target `Requires` both
+mirror and deploy steps so they are started when the target is activated.
+And since the mirror and deploy steps have `Requires` and `After` to the fetch
+step, systemd's ordering ensures the local mirror at `/mirror` is updated successfully
+before the steps that need it are run in parallel.
+
+These are files.
+
+.. class:: full-width
+.. code:: ini
+
+ # ~/.config/systemd/user/europan-materialist-cd.target
+ [Unit]
+ Requires=europan-materialist-deploy.service
+ Requires=europan-materialist-mirror.service
+
+ # ~/.config/systemd/user/europan-materialist-deploy.service
+ [Unit]
+ Requires=europan-materialist-fetch.service
+ After=europan-materialist-fetch.service
+
+ [Service]
+ Type=oneshot
+ WorkingDirectory=%h/europan-materialist
+ ExecStartPre=/usr/bin/git fetch --prune
+ ExecStartPre=/usr/bin/git checkout FETCH_HEAD
+ ExecStartPre=/usr/bin/podman build -f Containerfile \
+ -v %h/Barotrauma/Content:/Content:ro \
+ --tag europan-materialist
+ ExecStart=/usr/bin/podman run \
+ --rm \
+ -v %h/cfkeys:/run/secrets/wrangler:ro \
+ europan-materialist \
+ ash -c 'npm x -- vite build && npm install wrangler && env $$(cat /run/secrets/wrangler) npm x -- wrangler pages deploy --project-name materialist-next ./dist'
+
+ # ~/.config/systemd/user/europan-materialist-mirror.service
+ [Unit]
+ Requires=europan-materialist-fetch.service
+ After=europan-materialist-fetch.service
+
+ [Service]
+ Type=oneshot
+ ExecStart=/usr/bin/git -C /mnt/kaput/projects/europan-materialist.git push --mirror github
+
+ # ~/.config/systemd/user/europan-materialist-fetch.service
+ [Service]
+ Type=oneshot
+ ExecStart=/usr/bin/git -C /mnt/kaput/projects/europan-materialist.git fetch --prune origin
+
+That's it. It hasn't fallen over yet. Last time it ran it finished in about
+thirty seconds. Nearly a third of that is the emoji program. Here's even some logs
+from it as bonus.
+
+.. class:: full-width
+.. code::
+
+ Nov 20 13:04:36 banana podman[352347]: Uploading... (15/15)
+ Nov 20 13:04:36 banana podman[352347]: ✨ Success! Uploaded 3 files (12 already uploaded) (0.95 sec)
+ Nov 20 13:04:42 banana podman[352347]: ✨ Deployment complete! Take a peek over at https://3fccd3c7.materialist-next.pages.dev
+
+
+The website is served at materialist.pages.dev_. The code is `on GitHub`_ and SourceHut_.
+
+.. _Barotrauma: https://barotraumagame.com/
+.. _`SourceHut Pages`: https://srht.site/
+.. _`on GitHub`: https://github.com/sqwishy/europan-materialist/
+.. _SourceHut: https://git.sr.ht/~sqwishy/europan-materialist
+.. _materialist.pages.dev: https://materialist.pages.dev/
+
A froghat.ca/2023/11/sourcehut-cloudflare-pages/systemd-chart-dm.svg => froghat.ca/2023/11/sourcehut-cloudflare-pages/systemd-chart-dm.svg +1 -0
@@ 0,0 1,1 @@
+<svg baseProfile="full" viewBox="0 0 512 192" xmlns="http://www.w3.org/2000/svg"><defs><style>text{fill:#f9f5d7;dominant-baseline:middle;font-family:sans-serif}path{fill:transparent;stroke:#f9f5d7}text.mono{font-family:monospace;font-size:90%}path.service{stroke-width:0;fill:#b16286;opacity:.2}.arrow,path{stroke-width:1px}</style></defs><path class="target" d="m150.953 8.411 4.842 15.328 200.46 2.617-.141-17.303z" style="stroke-width:0;fill:#d79921;opacity:.2"/><text text-anchor="middle" x="256" y="16.941">europan-materialist-cd.target</text><path class="service" d="m25.83 51.451 1.325 16.812 200.685 2.259-1.323-14z"/><text text-anchor="middle" x="128" y="62.118">europan-materialist-mirror.service</text><path class="service" d="m282.87 51.79-1.344 18.6 201.743 6.246-.219-22.63z"/><text text-anchor="middle" x="384" y="62.118">europan-materialist-deploy.service</text><text class="mono" text-anchor="middle" x="128" y="73.412">cd /mirror</text><text class="mono" text-anchor="middle" x="384" y="73.412">cd /deploy</text><text class="mono" text-anchor="middle" x="128" y="84.706">git push --mirror github</text><text class="mono" text-anchor="middle" x="384" y="84.706">git fetch --prune /mirror</text><text class="mono" text-anchor="middle" x="384" y="96">git checkout FETCH_HEAD</text><text class="mono" text-anchor="middle" x="384" y="107.294">podman build</text><path class="service" d="m154.919 145.217 3.434 15.303 202.027-1.22-2.93-14.29z"/><text text-anchor="middle" x="256" y="152.471">europan-materialist-fetch.service</text><text class="mono" text-anchor="middle" x="256" y="163.765">cd /mirror</text><text class="mono" text-anchor="middle" x="256" y="175.059">git fetch --prune srht</text><path class="arrow" d="M231.958 29.095c-18.93 3.82-35.89 10.292-54.802 16.117m-8.998 3.416c3.18-3.671 4.775-6.958 7.69-10.013m.798 2.44c.722 2.955 2.618 6.961 1.978 8.615m.665 1.026c-3.36-1.076-8.513-.912-10.63-2.348"/><path class="arrow" d="M230.118 29.42c-16.201 3.54-35.049 10.077-52.651 15.666m-9.985 2.985c4.723-4.926 7.541-4.737 9.264-6.257m-.088-.906c1.064 3.524 2.951 5.142 2.963 8.128m-.992.077c.464-.008-6.025.413-10.437-.901M281.885 29.48c16.289 3.764 30.18 10.086 46.305 14.962m9.609 4.672c-3.233-.293-6.174.271-11.01.07m.016.09c.368-3.387 1.36-7.743 2.165-10.407m3.652.984c.328 2.331 3.201 5.754 4.673 8.056"/><path class="arrow" d="M281.397 27.76c15.706 5.329 30.673 11.852 46.43 17.725m8.554 2.89c-1.872-.132-3.565-.473-10.82 1.896m.011-.1c1.612-2.153 5.317-6.794 5.019-9.561m-.033-1.785c-.567 4.884 4.067 6.666 7.31 9.39M167.963 95.365c17.762 12.628 35.953 22.982 54.922 35.568m7.581 7.527c-3.77-1.864-6.056-2.286-10.256.442m-.054-3.626c-.07-.532 1.766-5.472 4.646-6.464m.304-.49c2.301 3.84 2.576 6.076 5.117 8.103"/><path class="arrow" d="M168.852 97.065c18.695 10.843 35.403 23.325 51.729 35.323m6.914 5.333c-.716-.673-4.896-.9-7.869-.167m.17-.484c1.172-3.627 2.954-4.511 6.073-9.216m.212 1.012c.566 3.066 1.838 6.266 4.276 10.11M336.511 107.534c-12.652 10.91-30.145 17.49-43.755 25.892m-11.362 4.997c.28-4.432 5.563-5.814 6.464-9.144m-.001 1.124c3.21 2.073 3.915 7.205 4.698 7.957m-.797-1.067c-1.272-.276-4.379 1.386-9.271.842"/><path class="arrow" d="M339.553 109.637c-18.44 9.44-33.446 14.698-48.702 24.12m-8.106 5.836c1.086-4.18 2.114-6.946 5.31-10.119m.14-.024c3.586 3.063 3.417 6.87 4.132 8.677m.946.685c-4.662.503-7.483-1.654-11.298.173"/></svg>
A froghat.ca/2023/11/sourcehut-cloudflare-pages/systemd-chart.svg => froghat.ca/2023/11/sourcehut-cloudflare-pages/systemd-chart.svg +1 -0
@@ 0,0 1,1 @@
+<svg baseProfile="full" viewBox="0 0 512 192" xmlns="http://www.w3.org/2000/svg"><defs><style>text{fill:#1d2021;dominant-baseline:middle;font-family:sans-serif}path{fill:transparent;stroke:#1d2021}text.mono{font-family:monospace;font-size:90%}path.service{stroke-width:0;fill:#b16286;opacity:.2}.arrow,path{stroke-width:1px}</style></defs><path class="target" d="m150.953 8.411 4.842 15.328 200.46 2.617-.141-17.303z" style="stroke-width:0;fill:#d79921;opacity:.2"/><text text-anchor="middle" x="256" y="16.941">europan-materialist-cd.target</text><path class="service" d="m25.83 51.451 1.325 16.812 200.685 2.259-1.323-14z"/><text text-anchor="middle" x="128" y="62.118">europan-materialist-mirror.service</text><path class="service" d="m282.87 51.79-1.344 18.6 201.743 6.246-.219-22.63z"/><text text-anchor="middle" x="384" y="62.118">europan-materialist-deploy.service</text><text class="mono" text-anchor="middle" x="128" y="73.412">cd /mirror</text><text class="mono" text-anchor="middle" x="384" y="73.412">cd /deploy</text><text class="mono" text-anchor="middle" x="128" y="84.706">git push --mirror github</text><text class="mono" text-anchor="middle" x="384" y="84.706">git fetch --prune /mirror</text><text class="mono" text-anchor="middle" x="384" y="96">git checkout FETCH_HEAD</text><text class="mono" text-anchor="middle" x="384" y="107.294">podman build</text><path class="service" d="m154.919 145.217 3.434 15.303 202.027-1.22-2.93-14.29z"/><text text-anchor="middle" x="256" y="152.471">europan-materialist-fetch.service</text><text class="mono" text-anchor="middle" x="256" y="163.765">cd /mirror</text><text class="mono" text-anchor="middle" x="256" y="175.059">git fetch --prune srht</text><path class="arrow" d="M231.958 29.095c-18.93 3.82-35.89 10.292-54.802 16.117m-8.998 3.416c3.18-3.671 4.775-6.958 7.69-10.013m.798 2.44c.722 2.955 2.618 6.961 1.978 8.615m.665 1.026c-3.36-1.076-8.513-.912-10.63-2.348"/><path class="arrow" d="M230.118 29.42c-16.201 3.54-35.049 10.077-52.651 15.666m-9.985 2.985c4.723-4.926 7.541-4.737 9.264-6.257m-.088-.906c1.064 3.524 2.951 5.142 2.963 8.128m-.992.077c.464-.008-6.025.413-10.437-.901M281.885 29.48c16.289 3.764 30.18 10.086 46.305 14.962m9.609 4.672c-3.233-.293-6.174.271-11.01.07m.016.09c.368-3.387 1.36-7.743 2.165-10.407m3.652.984c.328 2.331 3.201 5.754 4.673 8.056"/><path class="arrow" d="M281.397 27.76c15.706 5.329 30.673 11.852 46.43 17.725m8.554 2.89c-1.872-.132-3.565-.473-10.82 1.896m.011-.1c1.612-2.153 5.317-6.794 5.019-9.561m-.033-1.785c-.567 4.884 4.067 6.666 7.31 9.39M167.963 95.365c17.762 12.628 35.953 22.982 54.922 35.568m7.581 7.527c-3.77-1.864-6.056-2.286-10.256.442m-.054-3.626c-.07-.532 1.766-5.472 4.646-6.464m.304-.49c2.301 3.84 2.576 6.076 5.117 8.103"/><path class="arrow" d="M168.852 97.065c18.695 10.843 35.403 23.325 51.729 35.323m6.914 5.333c-.716-.673-4.896-.9-7.869-.167m.17-.484c1.172-3.627 2.954-4.511 6.073-9.216m.212 1.012c.566 3.066 1.838 6.266 4.276 10.11M336.511 107.534c-12.652 10.91-30.145 17.49-43.755 25.892m-11.362 4.997c.28-4.432 5.563-5.814 6.464-9.144m-.001 1.124c3.21 2.073 3.915 7.205 4.698 7.957m-.797-1.067c-1.272-.276-4.379 1.386-9.271.842"/><path class="arrow" d="M339.553 109.637c-18.44 9.44-33.446 14.698-48.702 24.12m-8.106 5.836c1.086-4.18 2.114-6.946 5.31-10.119m.14-.024c3.586 3.063 3.417 6.87 4.132 8.677m.946.685c-4.662.503-7.483-1.654-11.298.173"/></svg>
