~ctb/blog

ce2df1c2ef99636aa11175f946bbda9d819b1854 — Christopher Thomas Bohn 1 year, 10 months ago bb9f3c6 master 
Remove post
patch browse .tar.gz 
2 files changed, 1 insertions(+), 733 deletions(-)

M site/blog.txt
D site/blog/self-hosting-email.txt

M site/blog.txt => site/blog.txt +1 -1
@@ 1,4 1,4 @@
Blog
______________________________________________________________________

%/self-hosting-email
%/self-hosting-basics


D site/blog/self-hosting-email.txt => site/blog/self-hosting-email.txt +0 -732
@@ 1,732 0,0 @@
You've Got Mail
________________________________________________________________________________

There are plenty of guides all over the internet for how to setup a mail server.
Honestly, it's a bit overwhelming just how many there are. And they're all the
same: use your package manager to install bla, do bla hoopla, set foo and mv
bar... It's tedious, it's dumb, it isn't one-to-one. There are simpler ways. 

I am running a mailserver with exactly 51 packages and fewer than 2GB of space
used up. It idles at around 140MB RAM usage. It works very well. And it was
mindnumbingly simple, after I read a few manpages, read one of the package's
author's blog post on usage, and figured out how I had created an infinite
delivery loop that rendered the whole stack dead until I added a quick match
directive to a configuration file. 

People will tell you mail was hard, and that's just not true. And while it took
me four days to set everything up, that's mostly because I did a lot of
exploring in the meantime. All-in-all, it probably took around an afternoon to
do the core parts that mattered - and that includes building all the
dependencies *on the server* - a real genius would've scp'd them up. 

I don't intend on doing any in-depth explanations of what most of these parts
are, just know that you need/want them. This is essentially the minimum-viable
setup, with a bit of gravy to handle the spam. If you want some rough details on
most of this stuff, I'd recommend starting by looking here [1] - Linode offers a
great and compact little explainer of how the whole thing operates. It's concise
enough to get the picture. 

The basic stack is OpenSMTPD as our MTA, dovecot providing our IMAP and mailbox
management stuff, rspamd+redis for spam filtering and DKIM signing, mlmmj for
mailing list management (made it joyful), and finally bubger for archiving. We
make use of caddy to power our archive website, because we already had it for
our other sites. If you're not making your archives public, you don't need it.
We make light use of crond, certbot once a year (so install python on December
31st and then please, PLEASE uninstall it), and a service manager (busybox sv is
all we need).

The short answer of where you *have* to read if you just want a mailing list is:
you need an MTA, regardless.  That's what OpenSMTPD covers. If you just want a
mailing list, you just need this. If you want to replace Gmail/Protonmail/$mail,
then you'll want to run an IMAP server - that's what dovecot is for. 

Spam filtering is covered by rspamd, which requires redis. You'll want spam
filtering, because it's great. rspamd also covers DKIM signing our emails in a
really easy to configure way, so there's no good reason not to just do it.

All of these resources you need to refer to otherwise can be found here:
Core setup        [2]
Clarify OpenSMTPD [3]
mlmmj             [4]
bubger            [5]

Linode instituted a policy that ports 25, 465, and 587 are closed by default on
all new accounts. I'd suggest opening a ticket before continuing to get those
ports opened up - they usually respond within a few hours.
Or you can use iptables2 in community and do it yourself... who knows... [6]

This is going to be divied up into sections:
________________________________________________________________________________

- DNS records                                                              [1.0]
- OpenSMTPD configuration                                                  [2.0]
- mailing list configuration                                               [3.0]
- dovecot configuration                                                    [4.0]
- rspamd+redis configuration                                               [5.0]
- bubger configuration                                                     [6.0]


[1.0] DNS records
________________________________________________________________________________

domain.tld is whatever website you're setting up mail for - in my case, it was
k1sslinux.org. In your case, maybe it's goodtimesat5.com. Who knows. Just don't
assume domain.tld is a literal - it's a variable, fill in the blank.

Here are precisely the records you need for all this - even if you just want a
mailing list. An A record for your mail domain (mail.domain.tld), an MX record
pointing from domain.tld TO mail.domain.tld, and 3 TXT records; one for DMARC,
one for DKIM, and one for SPF. 

First, your hostname should be what you want the banner from OpenSMTPD to be
when it handshakes other servers - this banner should match the A record you
made (mail.domain.tld). Just do a quick 

# echo mail.domain.tld > /etc/hostname

You might want to add to /etc/hosts a line that reads

ServerIPAddress mail.domain.tld

Alternatively you can add the hostname to /etc/mail/mailname or add 'helo
mail.domain.tld' to the end of your outbound relay action in /etc/smtpd.conf,
but hostname is useless for the most part so there's no good reason not to just
set it there.

For DKIM, we need to generate a key. We can do that pretty easily:

# dkimpath=/etc/mail/dkim
# mkdir $dkimpath
# openssl genrsa -out $dkimpath/domain.tld.key 1024
# openssl rsa -in $dkimpath/domain.tld.key -pubout -out $dkimpath/domain.tld.pub
# cat $dkimpath/domain.cat.pub
# chmod 0440 $dkimpath/domain.tld.key

Copy the public key (that gross string after/before the -----***-----). We need
it. 

Open up your domain registrar's page to set your DNS records. Suffixes are
important here.

Make an A record for mail.domain.tld pointing to the external IP of your server

Make an MX record for domain.tld pointing to mail.domain.tld.

Make a TXT record for _dmarc.domain.tld pointing to:
v=DMARC1;p=none;pct=100;rua=mailto:postmaster@domain.tld;

// 'selector' below can be anything; I chose the date I made the key. We will
//  need this for later
Make a TXT record for _selector._domainkey.domain.tld, pointing to:
v=DKIM1;k=rsa;p=the public key that you copied from earlier

Make a TXT record for domain.tld that points to v=spf1 mx ~all

The first record does what we want. The rest is so that $BadEmailMonopoly
doesn't throw our email away when it gets to our recipient. 

Next you need to setup rDNS. This is just reverse DNS - if you give them an IP,
they can give you the domain name. On Linode, it's as easy as opening up your
Linode on the website, clicking the Network tab, and under the IP Addresses
block, click the ... in the row of the IPv4 address and choose 'edit rDNS'. Make
the rDNS for this address be mail.domain.tld
With rDNS, you will look more trustworthy. Spamsters and jerks don't usually
have the ability to set something like this, much like the rest of these records
we've set up. 

With all this work, I've only had my emails go to spam for two providers:
Protonmail and Tutanota. They're unkind to most of the internet. Half the
mailing lists I subscribe get marked as spam by Tutanota; they're just too
unreliable in my opinion. 
Gmail gave my emails flying colors :)

And just like that, our DNS records are all setup! Be aware that DNS records can
take up to two days to propogate. In my experience, it happens pretty quickly.
There are a couple ways to verify this. You can use https://mxtoolbox.com/ :
MX Lookup    domain.tld          -> to mail.domain.tld
DNS Lookup   mail.domain.tld     -> to Your Linode server IP
SPF Lookup   domain.tld          -> All green
DKIM Lookup  domain.tld:selector -> All green
DMARC Lookup _dmarc.domain.tld   -> Pass everything but DMARC Policy enabled

Useful test for later when we have SMTPD setup:
Test Email Server mail.domain.tld -> All green!

You can also install bind from community and use `host mail.domain.tld` to get
the ip, and then do `host` on that ip to get back mail.domaind.tld. If you do,
then rDNS is setup properly. You can use `dig {mail.}domain.tld` to get the rest
of the information mxtoolbox would give you.

Finally, we need some TLS certificates. Technically, Porkbun provides them for
me, and updates them for me too. But I didn't want to bother with this, so I
made my own! It's pretty easy if you opt to use something like certbot,
available via pip (I didn't want to package a billion python dependencies). Make
sure you have rust installed (because of some dumb upstream related decision
making, forcing rust to do SSL/TLS/Auth/Encryption things), and simply do:

# pip install certbot
# mkdir /var/www/mail.domain.tld 
# certbot certonly --standalone -d mail.domain.tld

Your important files made from this process are:
/etc/letsencrypt/live/mail.domain.tld/{fullchain,privkey}.pem

And that's it! The DNS setup is done. With all this scaffolding work, you can
basically all but guarantee that your emails will be delivered 1) successfully,
2) to an inbox. 


[2.0] OpenSMTPD configuration
________________________________________________________________________________

Now we move on to the lovely task of configuring OpenSMTPD!

OpenSMTPD had an update not too long ago which rendered older configuration
files incompatible with the newer version. This, I think, is a good thing; the
syntax is basically human readable, you only need one command to really make it
all work, it's a very well designed program (what else would I expect, thank you
for libressl/openssh/openntpd BSD folks). 

A second hurdle for figuring out how to configure MTAs in general is that if you
do a single thing wrong, you can end up with undeliverable mail. Or worse, an
infinite loopback of mail deliveries. Ask me how I know (luckily OpenSMTPD just
timed out instead of actually delivering a billion emails between two accounts
for eternity). 

Finally, of all the guides that exist in the ether of the Internet, nobody
really explains how you can properly serve mailing list emails && host your own.
All-in-all, it isn't hard. Spend a little time with the OpenSMTPD manpages to
familiarize yourself with the syntax and options - honestly, if you think to
yourself 'can I do this thing in these case and something else in others?', the
answer is almost certainly yes; it's a very thorough program, you can route,
filter, and send in basically any ways you could dream of. 

All that being said, here is a simple configuration file. Explanation of what it
does after:

# cat /etc/smptd.conf
________________________________________________________________________________
#       $OpenBSD: smtpd.conf,v 1.10 2018/05/24 11:40:17 gilles Exp $

# This is the smtpd server system-wide configuration file.
# See smtpd.conf(5) for more information.

# Setup PKI for our certs
pki domain.tld cert "/etc/letsencrypt/live/mail.domain.tld/fullchain.pem"
pki domain.tld key  "/etc/letsencrypt/live/mail.domain.tld/privkey.pem"

# This allows us to rewrap the body when we bounce it
# That way, it looks like it came from the original sender.
# Important for DMARC, DKIM to pass.
srs key "/etc/mail/dkim/domain.tld.key"

# All the filters we could ever want.
filter "rspamd" proc-exec "/usr/lib/opensmtpd/filter-rspamd"
filter "dyndns" phase connect match rdns regex \
    { '.*\.dyn\..*', '.*\.dsl\..*' } disconnect "550 no residential connections"
filter "rdns"   phase connect match !rdns   disconnect "550 no rDNS is so 80s"
filter "fcrdns" phase connect match !fcrdns disconnect "550 no FCrDNS is so 80s"

# The tables that define where any mail goes in any instance
# Passwords, real (system) users, domains, and virtual (not-system) users.
table passwd   file:/etc/mail/passwd
table aliases  file:/etc/mail/aliases
table domains  file:/etc/mail/domains
table virtuals file:/etc/mail/virtuals

# Listen for mail on eth0, enable TLS/PKI/filters/authentication
listen on lo
listen on eth0 port 25 tls pki "domain.tld" \
    filter {"dyndns", "rdns", "fcrdns", "rspamd" }
listen on eth0 port 587 tls-require pki "domain.tld" \
    auth &ltpasswd&gt filter "rspamd"

# Define some actions - receiving and relaying
action "receive"     maildir                      alias &ltaliases&gt
action "domain_mail" lmtp "/var/run/dovecot/lmtp" rcpt-to virtual &ltvirtuals&gt
action "send"        relay                        srs

## Send mail out when it hits
## Specifically, to me, OR to the mailing list.
match      from any for rcpt-to "user.name@domain.tld" action "domain_mail"
match      from any for domain  <domains>              action "receive"
match               for any                            action "send"
match auth from any for any                            action "send"
________________________________________________________________________________

The `pki...` lines define where our certificates are for TLS authentication.
The `srs...` line defines where our DKIM key is for signing our mails.
The `filter...` lines define what does filtering on what, and what to do. 
    My favorite is the disconnect messages. Feel free to change these to meme on
    those poor saps who want to connect to your server :)
The `table...` lines tell OpenSMTPD what files to open for &lttable&gt expansions.
The `listen...` lines define what ports we listen to for what types of
    connections. Additionally, we filter those connections based on the
    `filter...` lines, and authenticate emails by their passwords so random
    users can't just send mail to the universe.
The `action...` lines define what OpenSMTPD does - these are used by `match...`
    lines. We have a relay for sending mail out, and we use srs to say that
    these should be relayed with our DKIM key. We'll define a 'receive' action
    that delivers in the maildir format to our aliases, and we also have an
    action for when mail is delivered to a domain user, that sends the mail to
    dovecot over lmtp. Because this is a server and the users being delivered to
    don't have real accounts, we just deliver to virtual users.
The `match...` lines connect our listens to our actions. When any mail arrives,
    we send it. When any mail arrives for our domain, we receive it, which are
    then shipped off to our aliases. When mail is received for a specific user, 
    it gets delivered to them via dovecot. 
Be aware that the ordering for your `match...` lines matter. Mail coming in for
    user.name@domain.tld are *also* coming in for domain.tld, and so they will
    match the first match rule. Of course, user.name isn't a valid user in the
    alises file, and so the mail will be undeliverable.

There are many ways to set this up. Because I'm hosting this on a server and not
my own machine, I made myself a virtual user and put myself in the virtuals
file - all mail is actually delivered to a system user named vmail who we will
setup later. Our mailing list is actually just a trigger-on-receipt; OpenSMTPD
passes the mail off to mlmmj, and this action is in the aliases file.

It's quite possible this configuration is suboptimal, or even bad. If you find a
way to improve it, let me know! 

As for the files we've defined, do:

# touch /etc/mail/aliases
# touch /etc/mail/passwd

# cat >> /etc/mail/domains << EOF
domain.tld
mail.domain.tld
EOF

# cat >> /etc/mail/virtuals << EOF
abuse@domain.tld      user.name@domain.tld
hostmaster@domain.tld user.name@domain.tld
postmaster@domain.tld user.name@domain.tld
webmaster@domain.tld  user.name@domain.tld
user.name@domain.tld  vmail
EOF

# echo 'mail.domain.tld' > /etc/mail/mailname

Now we just need to add our user vmail who will act as the proxy through which
all of our mail is delivered to our fake users! 

addgroup -S vmail
adduser -S -D -H -h /var/vmail -s /bin/nologin -G vmail -g vmail vmail

All of the mail vmail delivers will eventually be delivered to:
/var/vmail/user.name@domain.tld/Mail

So if you want it to go somewhere else (for whatever FHS reasons you might
have), set the home directory for vmail to be somewhere else. Just make sure to
change the home directory of this new user to have the right permissions:
# chown vmail:vmail /var/vmail

We'll come back to the aliases file in the next section, and the passwd file
when we get to the dovecot section.


[3.0] mailing list configuration
________________________________________________________________________________

Setting up mlmmj is super simple! It has a lot of components that you'll
probably never have to interact with directly - it basically runs itself, while
you get to focus on the actual usability of your list! 

# mlmmj-make-ml

That's it. The script gives you a couple of prompts it will reference to
generate all of the files it needs. It defaults to putting the list in
/var/spool/mlmmj, which can be changed. It'll ask for your list name, domain
name, the owner's email, and the languages. It creates a collection of folders
and files in /var/spool/mlmmj/listname, and list management happens by editing,
adding, and removing these files. Check out the listname/text directory for some
useful files to edit, like FAQ! 

After you've made your list, you should help OpenSMTPD with getting access:

# chown -R smtpd:smptd /var/spoo/mail/listname

Next, we'll have to add the alias to OpenSMTPD for the list - which the script
kindly reminds you to do (in fact, it offers to add the entry for you!)

Add the following line anywhere to your /etc/mail/aliases file we made earlier:
listname "|/usr/bin/mlmmj-receive -L /var/spool/mlmmj/listname"

When OpenSMTPD gets an email, it follows the rules in the configuration file.
Our configuration tells it that, whenever it receives an email for domain.tld,
it should lookup the recipient in /etc/mail/aliases, and send that mail to
wherever it should go. In this case, OpenSMTPD is going execute a command,
the program mlmmj-receive, with an argument. It's pretty genius. 

Add a cronjob to automate list maintenance - it's a script that handles
auto-unsubscribing emails that have bounced too many times, resending emails
that bounced, and more. It's pretty hands off after that! The suggested time is
every two hours:

# crontab -e
0 */2 * * * "/bin/mlmmj-maintd -F -L /var/spool/mlmmj/mlmmj-test/"

You can do it any way you prefer though - maintd can even be daemonized, though
I haven't tried it...

I'd recommend looking at the mlmmj READMEs [6], they're absolutely packed with
good information. Some final things for /var/spool/mlmmj/listname/control:

The default digest timer is to go out once a week. If you want it to go out more
frequently, add a file named digestinterval whose only contents are the time
between digests, in seconds. 

If your list address ever changes, modify it in the listaddress file.

notmetoo if a file whose existence makes it so that when a user emails the list,
they don't get a copy of it back from the list. Though the documentation says
'attempts', so who knows how well it works...

If you run into errors involving mlmmj being unable to connect to localhost,
some issues might exist either with your hostname or your /etc/hosts file; the
default relay host for the list is set in relayhost, you can pass along the
correct relay IP address in there.

Repeat these steps as needed for the rest of your lists. 

And... That's basically it. email listname+help@domain.tld and if you get a
response back from your server, you did it all correctly! Otherwise, check your
logs for what the problem might be (and see all the lovely errors filling your
log from people assaulting your smtp connection). 

For troubleshooting, smtpd -n will verify the contents of your smtpd.conf. You
can run smtpd -Fv for verbose logging and keep it running in the foreground
incase the lines in the log get cut off. Finally, the most useful:
# smtpd -d -T rules
This will print a trace of what rule OpenSMTPD ends up matching to for any give
piece of mail. -T takes many arguments for better and more fine-grained
debugging; check the manpage!

Start your opensmtpd service and let the spam connections start rolling in.


[4.0] dovecot configuration
________________________________________________________________________________

dovecot is arguably the hardest part of all of this to configure. There is a TON
of documentation, but the features available and what they can do changes a lot
between even the most minor version bumps. So be ready for your configuration
to potentially break at some future time. For now, however, the following is
quite workable:

# cat /etc/dovecot/dovecot.conf
________________________________________________________________________________

# Some simple + sane SSL/TLS stuff
ssl = required
ssl_cert = &lt/etc/letsencrypt/live/mail.domain.tld/fullchain.pem
ssl_key  = &lt/etc/letsencrypt/live/mail.domain.tld/privkey.pem
ssl_min_protocol = TLSv1.2
ssl_prefer_server_ciphers = yes

# For special_use in our mailboxes
imap_capability = +SPECIAL-USE

# We have to make sure we start with system users
disable_plaintext_auth = no
first_valid_uid = 100

# Mail for our users; %n == user.name@domain.tld 
#                     %u == user.name
# OpenSMTPD is delivering in the maildir format, dovecot should follow
# suit. Mail is a subdir to put it in (in case ~ were in say, /home). 
mail_home=/var/vmail/%Lu
mail_location = maildir:~/Mail/:LAYOUT=fs

# Always watching, waiting
listen = *, ::

# lmpt for smptd
protocols = lmtp imap
service lmtp {
        unix_listener lmtp {
                user  = vmail
                group = vmail
        }
}
service imap-login {
        inet_listener imap {
                port = 143
        }
        inet_listener imaps {
                port = 993
        }
}

# Define our user database for auth lookups.
# Theoretically, this can be shared with OpenSMTPD a la passwd format
# If we were delivering mail for actual system users and no virtual users, we
# wouldn't actually need a passwd file aside from system /etc/passwd for either.
passdb {
        driver = passwd-file
    args = scheme=SHA512-CRYPT username_format=%Lu /etc/dovecot/users
}
userdb {
        driver = passwd-file
    args = username_format=%Lu /etc/dovecot/users
        override_fields = uid=vmail gid=vmail
}

# Define the imap protocol to use our sieve plugins
protocol imap {
    mail_plugins = $mail_plugins imap_sieve
}
# Our sieve plugins
plugin {
    sieve_plugins = sieve_imapsieve sieve_extprograms
        sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment
        
        imapsieve_mailbox1_name = Spam
        imapsieve_mailbox1_causes = COPY APPEND
        imapsieve_mailbox1_before = file:/usr/lib/dovecot/sieve/report-spam.sieve        

        imapsieve_mailbox2_name = *
        imapsieve_mailbox2_from = Spam
        imapsieve_mailbox2_causes = COPY
        imapsieve_mailbox2_before = file:/usr/lib/dovecot/sieve/report-ham.sieve
        
        imapsieve_mailbox3_name = INBOX
        imapsieve_mailbox3_causes = APPEND
        imapsieve_mailbox3_before = file:/usr/lib/dovecot/sieve/report-ham.sieve

        sieve_pipe_bin_dir = /usr/lib/dovecot/sieve
}

# Create mailboxes you expect to exist. Thanks, RFCs.
namespace inbox {
    # the namespace prefix isn't added again to the mailbox names.
    #prefix = INBOX.
    inbox = yes
        separator = /
    mailbox Archive {
        auto = create
        special_use = \Archive
    }
    mailbox Drafts {
        auto = create
        special_use = \Drafts
    }
    mailbox Sent {
        auto = subscribe # autocreate and autosubscribe the Sent mailbox
        special_use = \Sent
    }
    mailbox Spam {
        auto = create # autocreate Spam, but don't autosubscribe
        special_use = \Junk
    }
    mailbox Trash {
        auto = create
        special_use = \Trash
    }
}
________________________________________________________________________________

There are some comments throughout this config that loosely explain what's going
on. Basically it...
implements some basic SSL, 
sets up lmtp so OpenSMTPD can send your mail to and from dovecot,
sets up imap so you can get your mail on your phone,
define account usernames and passwords,
setup spam filtering sieves,
create the tradition mailbox folders you would expect of any email service

This is really the most involved portion of the whole setup, assuming you
actually want to dive into dovecot at all. It took me a while to come up with
all of this; I spent a lot of time paging through the wiki. Their documentation
is fine, if a little presumptuous. 

If we had regular old system users, we wouldn't have to do much else - all
account authentication would be done by looking at the /etc/passwd file. But
because we're special and don't want sensitive user data kept on our mail
server, we insist on virtual users to avoid the temptation of treating our
mailserver like our own PC. So we have to setup a passwords file. Generate some
password for your user to connect with this server for authentication.

# mkdir /etc/dovecot/users
# doveadm pw -s SHA512-CRYPT

You'll be prompted for that password with the second command. Copy the output
(except for the {SHA512-CRYPT} part), and add it to /etc/dovecot/users like so:

user.name@domain.tld:ThePasswordYouCopied::::::

Next, add the following to /etc/mail/passwd:
user.name@domain.tld ThePasswordYouCopied

And your authentication is basically handled. You can add this to whatever mail
app you fancy - your domain is mail.domain.tld, your user name is user.name, and
your password is whatever plaintext password you chose. ezclap

Start your dovecot service and you should be able to login and send fully
Gmail/Microsoft compliant emails to anyone and everyone! Protonmail, Tutanota,
and the like might complain, but they're special. We don't send mail to them.


[5.0] rspamd+redis configuration
________________________________________________________________________________

rspamd and redis have a ton of configuration possibilities, which makes me think
that they're overkill. But my server is idling at 140MB of RAM usage serving
three websites and all this mess, so it seems fine enough. 

Remember that selector from earlier in [1.0]? Yeah, we need it here.
I had to do the following for DKIM signing and spam filtering to work:

`domain {` and `selector =` are both literally 'domain' and 'selector'. 
# cat >> /etc/rspamd/local.d/dkim-signing.conf << EOF
allow_username_mismatch = true;

domain {
    domain.tld {
        path = "/etc/mail/dkim/domain.tld.key";
        selector = "selector";
    }
}
EOF

Additionally, we'll need to make sure that rspamd can read the DKIM keys we made
back at the beginning - otherwise you'll end up with intermittent timeout issues
as the daemon simply fails to do its job. 

# chown -R smtpd:rspamd /etc/mail/dkim

# cat >> /etc/rspamd/local.d/classifier-bayes.conf << EOF
servers = "127.0.0.1";
backend = "redis";
autolearn = true;
EOF

# cat >> /usr/lib/dovecot/sieve/report-ham.sieve << EOF
require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", "variables"];

if environment :matches "imap.mailbox" "*" {
  set "mailbox" "${1}";
}

if string "${mailbox}" "Trash" {
  stop;
}

if environment :matches "imap.user" "*" {
  set "username" "${1}";
}

pipe :copy "sa-learn-ham.sh" [ "${username}" ];
EOF

# cat >> /usr/lib/dovecot/sieve/report-spam.sieve << EOF
require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", "variables"];

if environment :matches "imap.user" "*" {
  set "username" "${1}";
}

pipe :copy "sa-learn-spam.sh" [ "${username}" ];
EOF

# cat >> /usr/lib/dovecot/sieve/sa-learn-ham.sh << EOF
#!/bin/sh
exec /usr/bin/rspamc -d "${1}" learn_ham
EOF

# cat >> /usr/lib/dovecot/sieve/sa-learn-spam.sh << EOF
#!/bin/sh
exec /usr/bin/rspamc -d "${1}" learn_spam
EOF

# sievec /usr/lib/dovecot/sieve/report-ham.sieve
# sievec /usr/lib/dovecot/sieve/report-spam.sieve
# chmod 755 /usr/lib/dovecot/sieve/sa-learn-ham.sh
# chmod 755 /usr/lib/dovecot/sieve/sa-learn-spam.sh

Restart your dovecot service and you should be good to go with spam detection!
rspamd will have to be trained a little bit before it becomes super smart, but
given time it'll learn. You can test it by moving an email to your spam folder
in your mail app of choice - you shold see in your rspamd logs about spam
training. So far I have yet to receive any spam, so I have no actual idea how
good it is; but some stranger on the internet told me it was good, so... shrug.


[6.0] bubger configuration
________________________________________________________________________________

bubger is literally a genius piece of software. I don't know why it's so hard to
find a program that just generates simple static pages of content that it
generates from a file hierarchy as simple as email, but it is. mlmmj-archiver is
ancient and requires... hypermail... Others require php, and I didn't feel like
packaging php for this project - just look at Alpine's pkgbuild for it. No.

bubger is just a simple C program that does heavy lifting very quickly. Checkout
its creator June's thought process behind it [7]. It'll need your plaintext
password in the environment or passed super securely to it somehow, and setting
all that up is on YOU. Good luck.

Setup == easy. I made a script that is executed by a cronjob twice a day.
Because I was working on this whole stack for a week and I just wanted to roll
it out and not worry about efficiency right now, the script just deletes the
entire served archive and replaces it with an updated version. bubger supports
doing incremental updates though, and you can make powerful use of rsync to do
it. 

For the dovecot config, I copied the Archive mailbox and renamed it to List
(feel free to be more creative). Sieve is a powerful spec that we can make
perfect use of here; whenever we get an email from the list, archive a copy of
it - that way, when bubger goes snooping, it only finds emails for the list. 

Add this to your plugins section in /etc/dovecot/dovecot.conf:
    imapsieve_mailbox4_name = *
    imapsieve_mailbox4_cause = APPEND
    imapsieve_mailbox4_before = file:/usr/lib/dovecot/sieve/mailing-list.sieve

Finally, the actual sieve itself:

# cat >> /usr/lib/dovecot/sieve/mailing-list.sieve << EOF
require ["fileinto", "copy", "imap4flags"];

if address :matches ["To", "Cc"] "listname@domain.tld" {
    fileinto :copy :flags "\\Seen" "List";
}
EOF

# sievec /usr/lib/dovecot/sieve/mailing-list.sieve

And restart the dovecot service. Wham, Bham, Thank You, Ma'am. 

As for the archive script, it's just something simple, cd's to where the static
files for the archive should go, the domain and username for the account, and
what folder to peak into for all those good, good convos:

bubger -C /var/www/archive.domain.tld user.name@domain.tld List

And that's basically it. Painless setup, hands off management. And the aesthetic
jives with the rest of the k1sslinux.org style, so it's a win-win!


________________________________________________________________________________

And that's all there is to it. I spent about a week reading a ton of stuff about
all of these things. I had never done a full-scale project like this before, so
it was a fun learning experience! I breezed over a ton of the more fine-grained
details that are essentially uneccessary if you 'just want something that
works' - this is all workable. But I highly recommend reading the documentation
for these projects - most of them are palatable and easy to read, and you'll
learn a lot along the way! 


[7.0] References
________________________________________________________________________________

[1] https://www.linode.com/docs/guides/running-a-mail-server/
[2] https://poolp.org/posts/2019-09-14/setting-up-a-mail-server-with-opensmtpd-dovecot-and-rspamd/
[3] https://www.opensmtpd.org/manual.html
[4] http://mlmmj.org/
[5] https://git.causal.agency/bubger/about/
[6] http://mlmmj.org/docs/readme/
[7] https://text.causal.agency/019-mailing-list.txt