~qeef/damn-deploy

60291d940480cdd30a1f9802bd5811db6bf3768c — Jiri Vlasak 5 months ago 9f081ee + a6c464c 
Merge branch 'move-to-sr.ht'
patch browse 
3 files changed, 193 insertions(+), 194 deletions(-)

M CHANGELOG.md
M README.md
M clients.yml

M CHANGELOG.md => CHANGELOG.md +41 -53
@@ 9,13 9,21 @@ The format is based on [Keep a Changelog][] and this project adheres to
[Keep a Changelog]: http://keepachangelog.com/
[Semantic Versioning]: http://semver.org/

[Unreleased][]
==============
Unreleased
==========

The repository source moved to https://sr.ht/~qeef/damn-project/.

Added
-----

- Domain and subdomain `.env` variables.
- Upgrade to `v0.8.0` section.

  **NOTE:** Please, follow the *Upgrade to v0.8.0* section of the [readme][].
  This version introduces two hard changes. First, the source repository was
  moved to https://sr.ht/~qeef/damn-project/. Second, the `damn-deploy`
  components were renamed (the directory structure.)

Changed
-------


@@ 31,18 39,18 @@ Removed
- Defaults used in `.env` variables. They are set in `.env` file anyway.


[0.7.0][] - 2020-10-17
======================
0.7.0 - 2020-10-17
==================

Upgrades
--------

- `damn_server` to [v0.6.1][s0.6.1].
- `damn_server` to `v0.6.1`.

  **NOTE:** You need to upgrade database content to make sure the statistics
  works. Please, see *Upgrade to `v0.7.0`* section of [readme][] file.
- `damn_client` to [v0.11.2][c0.11.2].
- `damn_jsonmanager` to [v0.1.5][m0.1.5].
- `damn_client` to `v0.11.2`.
- `damn_jsonmanager` to `v0.1.5`.

Added
-----


@@ 56,13 64,10 @@ Changed
- Markdown reformat.

[readme]: ./README.md
[s0.6.1]: https://gitlab.com/damn-project/damn_server/-/tags/v0.6.1
[c0.11.2]: https://gitlab.com/damn-project/damn_client/-/tags/v0.11.2
[m0.1.5]: https://gitlab.com/damn-project/damn_jsonmanager/-/tags/v0.1.5


[0.6.2][] - 2020-10-01
======================
0.6.2 - 2020-10-01
==================

Fixed
-----


@@ 70,16 75,16 @@ Fixed
- Update json manager version.


[0.6.1][] - 2020-10-01
======================
0.6.1 - 2020-10-01
==================

Fixed
-----
- Update server and client version.


[0.6.0][] - 2020-10-01
======================
0.6.0 - 2020-10-01
==================

Added
-----


@@ 97,8 102,8 @@ Changed
- Swich client to `v0.10.0` that support multiple languages.


[0.5.0][] - 2020-08-18
======================
0.5.0 - 2020-08-18
==================

Added
-----


@@ 106,8 111,8 @@ Added
- Docker image, docker compose for [WSChat](https://sr.ht/~qeef/wschat/).


[0.4.2][] - 2020-06-16
======================
0.4.2 - 2020-06-16
==================

Fixed
-----


@@ 115,8 120,8 @@ Fixed
- Set maximum of 100 workers due to maximum of 100 database connections.


[0.4.1][] - 2020-06-02
======================
0.4.1 - 2020-06-02
==================

Fixed
-----


@@ 124,8 129,8 @@ Fixed
- Upkeep script.


[0.4.0][] - 2020-04-29
======================
0.4.0 - 2020-04-29
==================

Added
-----


@@ 139,8 144,8 @@ Changed
- Update server to `v0.5.0`.


[0.3.0][] - 2020-02-01
======================
0.3.0 - 2020-02-01
==================

Added
-----


@@ 148,16 153,16 @@ Added
- Second docker compose configuration for running clients (client and manager).


[0.2.2][] - 2020-01-28
======================
0.2.2 - 2020-01-28
==================

Changed
-------

- Server update to version 0.4.0.

[0.2.1][] - 2020-01-14
======================
0.2.1 - 2020-01-14
==================

Fixed
-----


@@ 165,8 170,8 @@ Fixed
- Use UTC now() for default timestamps in database.


[0.2.0][] - 2020-01-13
======================
0.2.0 - 2020-01-13
==================

Changed
-------


@@ 174,8 179,8 @@ Changed
- Damn server version to v0.3.0.


[0.1.2][] - 2020-01-08
======================
0.1.2 - 2020-01-08
==================

Changed
-------


@@ 183,8 188,8 @@ Changed
- Update create database sql script.


[0.1.1][] - 2020-01-01
======================
0.1.1 - 2020-01-01
==================

Fixed
-----


@@ 204,20 209,3 @@ Added
- Docker compose file.
- Docker file for damn upkeep image.
- Traefik docker image, restrict damn server to https.


[Unreleased]: https://gitlab.com/damn-project/damn_deploy/compare/v0.7.0...master
[0.7.0]: https://gitlab.com/damn-project/damn_deploy/compare/v0.6.2...v0.7.0
[0.6.2]: https://gitlab.com/damn-project/damn_deploy/compare/v0.6.1...v0.6.2
[0.6.1]: https://gitlab.com/damn-project/damn_deploy/compare/v0.6.0...v0.6.1
[0.6.0]: https://gitlab.com/damn-project/damn_deploy/compare/v0.5.0...v0.6.0
[0.5.0]: https://gitlab.com/damn-project/damn_deploy/compare/v0.4.2...v0.5.0
[0.4.2]: https://gitlab.com/damn-project/damn_deploy/compare/v0.4.1...v0.4.2
[0.4.1]: https://gitlab.com/damn-project/damn_deploy/compare/v0.4.0...v0.4.1
[0.4.0]: https://gitlab.com/damn-project/damn_deploy/compare/v0.3.0...v0.4.0
[0.3.0]: https://gitlab.com/damn-project/damn_deploy/compare/v0.2.2...v0.3.0
[0.2.2]: https://gitlab.com/damn-project/damn_deploy/compare/v0.2.1...v0.2.2
[0.2.1]: https://gitlab.com/damn-project/damn_deploy/compare/v0.2.0...v0.2.1
[0.2.0]: https://gitlab.com/damn-project/damn_deploy/compare/v0.1.2...v0.2.0
[0.1.2]: https://gitlab.com/damn-project/damn_deploy/compare/v0.1.1...v0.1.2
[0.1.1]: https://gitlab.com/damn-project/damn_deploy/compare/v0.1.0...v0.1.1


M README.md => README.md +150 -139
@@ 14,6 14,14 @@ This project is published under [MIT License][].

[MIT License]: ./LICENSE

Discussion
----------

Use the mailing list for the discussion.

    ~qeef/damn-project@lists.sr.ht


How to deploy
=============



@@ 23,15 31,14 @@ server.

Then, you need to clone the damn deploy repository:

```
git clone https://gitlab.com/damn-project/damn_deploy.git
cd damn_deploy
```
    git clone https://git.sr.ht/~qeef/damn-deploy
    cd damn-deploy

The most common usecase is *autostart server and clients*. The following
The most common usecase is *autostart server, clients, and chat*. The following
sections consider *server* setup, setup for *clients* (client and manager),
*chat*, *autostart*, and *update*. Also, see *Damn upkeep* section for periodic
database upkeep scripts.
*chat*, and *autostart* with `systemd`. Then, there is a *How to upgrade*
section that discuss the upgrade/update process. Finally, see the *Damn upkeep*
section for the database periodic upkeep scripts.

Server setup
------------


@@ 58,16 65,12 @@ Server setup

3. Create `acme.json` file and set `600` permissions:

   ```
   touch acme.json
   chmod 600 acme.json
   ```
       touch acme.json
       chmod 600 acme.json

4. You may run the damn server now.

   ```
   docker-compose -f server.yml up
   ```
       docker-compose -f server.yml up

Setup for clients
-----------------


@@ 94,9 97,7 @@ In addition to *Server setup*, you may want to run a client and a manager, too.

2. You may run the damn server with clients now.

   ```
   docker-compose -f server.yml -f clients.yml up
   ```
       docker-compose -f server.yml -f clients.yml up

Setup for chat
--------------


@@ 109,95 110,75 @@ In addition to *Server setup*, you may want to run a chat server.

2. Run the chat server with the server:

   ```
   docker-compose -f server.yml -f chat.yml up
   ```
       docker-compose -f server.yml -f chat.yml up

   or also with clients:

   ```
   docker-compose -f server.yml -f clients.yml -f chat.yml up
   ```
       docker-compose -f server.yml -f clients.yml -f chat.yml up

Autostart with `systemd`
------------------------

I assume that the damn deploy repository is cloned to `/root/damn_deploy/`.
I assume that the damn deploy repository is cloned to `/root/damn-deploy/`.

1. Create systemd unit for `docker-compose`:

   ```
   cat << EOF >> /etc/systemd/system/damn.service
   [Unit]
   Description=Damn server app
   After=network.target docker.service
       cat << EOF > /etc/systemd/system/damn.service
       [Unit]
       Description=Damn server app
       After=network.target docker.service

   [Service]
   Type=simple
   WorkingDirectory=/root/damn_deploy
   ExecStart=/usr/bin/docker-compose -f /root/damn_deploy/server.yml -f /root/damn_deploy/clients.yml up
   ExecStop=/usr/bin/docker-compose -f /root/damn_deploy/server.yml -f /root/damn_deploy/clients.yml down
       [Service]
       Type=simple
       WorkingDirectory=/root/damn-deploy
       ExecStart=/usr/bin/docker-compose -f /root/damn-deploy/server.yml -f /root/damn-deploy/clients.yml -f /root/damn-deploy/chat.yml up
       ExecStop=/usr/bin/docker-compose -f /root/damn-deploy/server.yml -f /root/damn-deploy/clients.yml -f /root/damn-deploy/chat.yml down

   [Install]
   WantedBy=multi-user.target
   EOF
   ```
       [Install]
       WantedBy=multi-user.target
       EOF

   Remove ` -f /root/damn_deploy/clients.yml` when running server only.
   Remove ` -f /root/damn-deploy/clients.yml` when running server only.

2. Enable and run damn service:

   ```
   systemctl start damn.service
   systemctl enable damn.service
   ```
       systemctl start damn.service
       systemctl enable damn.service


How to update
=============
How to upgrade
==============

1. If use `systemd` autostart, stop the damn service first:

   ```
   systemctl stop damn.service
   ```
       systemctl stop damn.service

2. Stash the configuration:

   ```
   git stash
   ```
       git stash

3. Pull new version of the damn deploy repository:

   ```
   git pull
   ```
       git pull

4. Pop stashed changes (renew the configuration):

   ```
   git stash pop
   ```
       git stash pop

5. Maybe rebuild some docker images, but it depends on update changes:
5. Maybe rebuild some docker images, but it depends on the upgrade changes:

   ```
   docker-compose -f server.yml build --no-cache api
   docker-compose -f server.yml build --no-cache upkeep
   docker-compose -f clients.yml build --no-cache client
   docker-compose -f clients.yml build --no-cache manager
   docker-compose -f chat.yml build --no-cache chat
   ```
       docker-compose -f server.yml build --no-cache api
       docker-compose -f server.yml build --no-cache upkeep
       docker-compose -f clients.yml build --no-cache client
       docker-compose -f clients.yml build --no-cache manager
       docker-compose -f chat.yml build --no-cache chat

6. Start the damn service again:

   ```
   systemctl start damn.service
   ```
       systemctl start damn.service

Update database structure
-------------------------
Upgrade the database structure
------------------------------

The database structure is rarely to change but it may happen. For new
deployments, there is nothing to do.


@@ 206,7 187,7 @@ However, if you are upgrading from an existing database, you need to run the
upgrade script manually.

All the scripts related to the database structure are stored in `damndb`
directory, starting at `70_...`. Which script should be run when updating the
directory, starting at `70_...`. Which script should be run when upgrading the
database structure is always noted in the [changelog][].

Upgrade to `v0.6.0`


@@ 214,39 195,29 @@ Upgrade to `v0.6.0`

Stop the damn service and update the repository:

```
systemctl stop damn.service
git stash
git pull
git stash pop
```
    systemctl stop damn.service
    git stash
    git pull
    git stash pop

Run the database:

```
docker-compose up -d db
```
    docker-compose up -d db

Run the database upgrade script (you will be asked for the password stored in
`.env` file):

```
export DB_HOST=$(docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' damndb)
psql -h $DB_HOST -d damndb -U damnuser < damndb/71_areas_squares_stats.sql
```
    export DB_HOST=$(docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' damndb)
    psql -h $DB_HOST -d damndb -U damnuser < damndb/71_areas_squares_stats.sql

Stop the database container:

```
docker-compose down
```
    docker-compose down

Finally, there is no need to build the database container again. The database
is created only once anyway, so run the damn service again:

```
systemctl start damn.service
```
    systemctl start damn.service

Upgrade to `v0.7.0`
-------------------


@@ 254,16 225,79 @@ Upgrade to `v0.7.0`
When the upgrade is finished, update the `current_commits` in the database.
Connect to the database (the password is in `.env` file):

```
export DB_HOST=$(docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' damndb)
psql -h $DB_HOST -d damndb -U damnuser
```
    export DB_HOST=$(docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' damndb)
    psql -h $DB_HOST -d damndb -U damnuser

Run the update query:

    update current_commits set type='splitted' where message='The square was splitted';

Upgrade to `v0.8.0`
-------------------

This upgrade introduce change of the repository source and directories
renaming. The best is to stop the running service first:

    systemctl stop damn.service

Make sure all the containers are stopped:

    cd damn_deploy
    docker-compose -f docker-compose.yml stop
    docker-compose -f docker-compose-clients.yml stop
    docker-compose -f docker-compose-chat.yml stop
    cd ..

And then follow the *How to deploy* section (I assume the current directory is
`/root/`:)

    git clone https://git.sr.ht/~qeef/damn-deploy
    cd damn-deploy

Then copy secrets and configuration from `../damn_deploy/.env` to `.env` file.
(The `.env` changed a bit!) Also, update `traefik.yml` and copy
`../damn_deploy/acme.json` to `acme.json`.

The next step is to rewrite systemd config files. From the *Autostart with
`systemd`* section, run the command to *Create systemd unit for
`docker-compose`*:

Run update query:
    cat << EOF > /etc/systemd/system/damn.service
    ...

```
update current_commits set type='splitted' where message='The square was splitted';
```
And from the *Damn upkeep* section, run the command to *Create systemd unit and
timer for damn upkeep*:

    cat << EOF > /etc/systemd/system/damn_upkeep.service
    ...

Rebuild all the docker images with:

    docker-compose -f server.yml build --no-cache api
    docker-compose -f server.yml build --no-cache upkeep
    docker-compose -f clients.yml build --no-cache client
    docker-compose -f clients.yml build --no-cache manager
    docker-compose -f chat.yml build --no-cache chat

Copy old database volume to the new database volume:

    docker volume create --name damn-deploy_damndb-volume
    docker run --rm -v damn_deploy_damndb-volume:/from -v damn-deploy_damndb-volume:/to alpine ash -c 'cd /from ; cp -av . /to'

Remove `damndb` container and rebuild it. It does **nothing** to the database.

    docker rm damndb
    docker-compose -f server.yml build --no-cache db

Reload systemd units and start the service:

    systemctl daemon-reload
    systemctl start damn.service

Now, if everything works and everything is tested, you may remove old database
volume:

    docker volume rm damn_deploy_damndb-volume


Damn upkeep


@@ 277,51 311,28 @@ hours.

1. Create systemd unit and timer for damn upkeep:

   ```
   cat << EOF >> /etc/systemd/system/damn_upkeep.service
   [Unit]
   Description=Run damn upkeep
       cat << EOF > /etc/systemd/system/damn_upkeep.service
       [Unit]
       Description=Run damn upkeep

   [Service]
   WorkingDirectory=/root/damn_deploy
   ExecStart=/usr/bin/docker-compose -f /root/damn_deploy/server.yml run --rm upkeep
   EOF
   cat << EOF >> /etc/systemd/system/damn_upkeep.timer
   [Unit]
   Description=Run damn upkeep every 15 minutes
       [Service]
       WorkingDirectory=/root/damn-deploy
       ExecStart=/usr/bin/docker-compose -f /root/damn-deploy/server.yml run --rm upkeep
       EOF
       cat << EOF > /etc/systemd/system/damn_upkeep.timer
       [Unit]
       Description=Run damn upkeep every 15 minutes

   [Timer]
   OnCalendar=*:0/15
       [Timer]
       OnCalendar=*:0/15

   [Install]
   WantedBy=timers.target
   EOF
   ```
       [Install]
       WantedBy=timers.target
       EOF

2. Enable and run damn upkeep service and timer:

   ```
   systemctl enable damn_upkeep.timer
   systemctl enable damn_upkeep.service

   systemctl start damn_upkeep.timer
   ```

Contribute
==========

Use [OneFlow][] branching model and keep the [changelog][].

Write [great git commit messages][]:

1. Separate subject from body with a blank line.
2. Limit the subject line to 50 characters.
3. Capitalize the subject line.
4. Do not end the subject line with a period.
5. Use the imperative mood in the subject line.
6. Wrap the body at 72 characters.
7. Use the body to explain what and why vs. how.
       systemctl enable damn_upkeep.timer
       systemctl enable damn_upkeep.service

[OneFlow]: https://www.endoflineblog.com/oneflow-a-git-branching-model-and-workflow
[changelog]: ./CHANGELOG.md
[great git commit messages]: https://chris.beams.io/posts/git-commit/
       systemctl start damn_upkeep.timer


M clients.yml => clients.yml +2 -2
@@ 4,7 4,7 @@ services:
        build:
            context: client
            args:
                DAMN_SERVER: ${SERVER_SUB}.{DOMAIN_NAME}
                DAMN_SERVER: ${SERVER_SUB}.${DOMAIN_NAME}
                DAMN_CLIENT_REPO: ${DAMN_CLIENT_REPO}
                DAMN_CLIENT_VERSION: ${DAMN_CLIENT_VERSION}
                DEFAULT_LANG: ${DEFAULT_LANG}


@@ 27,7 27,7 @@ services:
        build:
            context: manager
            args:
                DAMN_SERVER: ${SERVER_SUB}.{DOMAIN_NAME}
                DAMN_SERVER: ${SERVER_SUB}.${DOMAIN_NAME}
                DAMN_MANAGER_REPO: ${DAMN_MANAGER_REPO}
                DAMN_MANAGER_VERSION: ${DAMN_MANAGER_VERSION}
        env_file: .env