~sircmpwn/sr.ht-nginx

Add another AI scraper to robots.txt

This bot alledgedly respects robots.txt
Add /tarpit, block another bot
lists.sr.ht: adapt to new shared assets layout
hub.sr.ht: adapt to new share assets layout
hg.sr.ht: adapt to new shared assets layout
git.sr.ht: adapt to new shared assets layout
builds.sr.ht: adapt to new shared asset layout
meta.sr.ht: adapt to new shared asset layout
todo.sr.ht: adapt to new shared asset layout
paste.sr.ht: adapt to new shared asset layout
man.sr.ht: adapt to new shared assets layout
Expand robots.txt

Block commits and block a new SEO bot
robots.txt: expand OpenAI blocks

Fuck you, Sam Altman
sourcehut.conf: update our trusted CIDRs
sourcehut.conf: update $python to 3.12

Pending tomorrow's maintenance window
robots.txt: add facebook crawler
todo.sr.ht: update CSP
robots.txt: add bytespider
sourcehut.conf: update python
Next
Do not follow this link