~egtann/cup: README.md - sourcehut git

#cup

cup helps you deploy apps and services on remote servers with high-availability rolling and parallel deploys. It works alongside up to do so.

up requires an Upfile, but maintaining a bug-free Upfile on a large project is difficult. To solve this problem, cup automatically generates Upfiles for you, removing the tedious and error-prone boilerplate. cup generates scripts suitable for fast deploys in production environments. Even large projects deploy in barely any more time than it takes to rsync the directories.

Nothing needs to be installed on the remote host except ssh and rsync (but you can customize those if you really want).

#Manifest

Every project using cup must define a manifest file, which is a json file with the name of your service, such as my_web_app.json. Although this one is called my_web_app.json, yours can be named anything. Here's an example:

$ cat my_web_app.json
{
	"files": {
		"deploy/": {},
		"pf.conf": {
			"path": "/etc/pf.conf",
			"mod": "600",
			"own": "root:root"
		},
	},
	"stop": [
		"sudo systemctl stop \"my_app 2\""
	],
	"start": [
		"echo $MY_SECRET_ENV > env.ini",
		"sudo systemctl start \"my_app 2\"",
		"sleep 5 && $check_health"
	],
	"default": {
		"ssh": "ssh -J $jump",
		"rsync": "rsync -chazP -e 'ssh -J $jump'",
		"remote": "/home/$user/my_app",
		"user": "_daemon"
	},
	"vars": {
		"jump": "jump@10.0.0.4",
		"check_health": "curl -s --max-time 1 http://$server:80/health"
	}
}

There's several sections in the example above:

files: this lists the files and directories to sync with the remote host. By default, everything is placed in a namespaced folder in the user's home directory, so if my remote user on a Linux box is called app then the deploy folder will be synced recursively to /home/app/my_web_app/deploy. You can override this default by specifying path, mod, and/or own.
stop and start: these are the steps that up will complete after syncing all files. up will always run stop, then start. You can omit stop if unneeded. Notice that $check_health is defined in the vars section.
default: tells cup to change its default remote commands and file locations. Normally this is not necessary, as cup will pick smart defaults. Consider this an escape hatch if you use a jumpbox or have some other non-standard setup.
vars: define specific variables which will be substituted any time they're encountered in the manifest file.

Note that you can also pass in environment variables such as $ENV above, which is great for secrets. Vars not declared in the "vars" section will be left as-is, meaning that up will do the substitution for you.

To use cup with our above arrangement, you'd usually call:

$ cup -f my_web_app.json | up -f -

cup will read my_web_app.json to generate an Upfile, which is piped to up and executed. In the above case, my_web_app would be deployed to 10.0.0.1, and if all steps succeeded, it would move on to 10.0.0.2.

The generated Upfile rsyncs all files in a single command, changes file ownership and permissions as needed, and then runs the deploy steps as a single script, minimizing ssh and rsync connections to deploy as quickly as possible.

You can configure up per usual. In the following example we do a dry run before before deploying 2 servers per batch in parallel:

$ cup -f my_web_app.json | up -d -f -
$ cup -f my_web_app.json | up -n 2 -f -

Note the trailing - is required after -f, as it instructs up to read from stdin.

#Using cup

Typically cup and up will be called from a deploy script, which ensures processes such as compiling and cleanup happen only once per deploy, even if deploying to many servers. An example best-practice deploy script is below.

#!/usr/bin/env sh
set -efu

name=my-service
tmpdir=/tmp/$name

# compile our binary
make

# unlock secrets which we can include in our build
shh login
mkdir -p $tmpdir
shh get -n my_web_app/production/env > $tmpdir/$name.env
shh get -n my_web_app/production/sql_client_key > $tmpdir/sql_key.pem
shh get -n my_web_app/production/sql_password > $tmpdir/sql_pass

# do a rolling, zero-downtime deploy on the appropriate servers
cup -f $name.json > $tmpdir/Upfile
up -c $name -f $tmpdir/Upfile

# clean up build artifacts
rm -r $tmpdir
make clean

#Defaults

user: $UP_USER
group: $UP_USER
remote: default folder for remote files, usually /home/$user/$manifest/. This will be used when a file's remote is either not specified or isn't an absolute path. Note that $manifest will be the name of the manifest.json file, without the extension. For instance if we're using my_app.json as our manifest file, the default folder for files will /home/$user/my_app.
ssh: ssh $user@$server $command. $command will be replaced automatically.
rsync: rsync -chazP --del --chmod=700 $files $user@$server:$manifest/. $files will be replaced automatically.
mv: sudo cp -R
chown: sudo chown -R
chmod: sudo chmod -R
mkdir: sudo mkdir -p

#Debugging

Since cup minimizes round-trips by combining all commands on the server into a single ssh call, you'll get an error by default that's difficult to debug, since it highlights a single line doing many different things, and any of those generated cp or chmod or chowns could have failed.

To address this, cup includes a debug flag -v which will create a slower but debuggable Upfile. Each ssh statement will be executed on its own line and ssh connection. When anything goes wrong, you'll see the exact command that caused it.

Your workflow will usually involve run/modify/run the manifest file with cup -v, and once everything works, remove the debug flag and enjoy deploys that are as fast as possible.