~fkooman/vpn-documentation

23884a6c61419eeaaa3493876da2f241f7f48585 — Fran├žois Kooman 9 months ago 92a7b2c v2
remove files now part of development repo
2 files changed, 0 insertions(+), 406 deletions(-)

D development-setup.md
D development_setup.sh
D development-setup.md => development-setup.md +0 -279
@@ 1,279 0,0 @@
# Development Setup

This document will describe how to set up an Let's Connect! / eduVPN 
development environment for easy development and running it on your development 
system. 

This is **NOT** meant to be used as installation instructions! See the 
[deploy](index.md#deployment) instructions instead!

We assume you will be using either Fedora >= 33, or Debian >= 10. Other 
distributions will also work, but some minor details, regarding installation of 
the required software, will be different.

**NOTE**: if you try to build a package repository you need to be on the 
respective OS: for DEB packages you need to be on Debian, for Fedora/CentOS 
packages you need to be on Fedora.

## CentOS 7

When you are not running CentOS as your desktop OS, it is easiest to install a
VM with a desktop. In addition, install the required software (dependencies):

```
$ sudo yum -y install golang php-cli git composer php-date php-filter php-hash \
    php-json php-mbstring php-pcre php-pdo php-spl php-libsodium php-curl \
    php-gd unzip qrencode scdoc 
```

## Fedora >= 35

When you are not running Fedora as your desktop OS, it is easiest to install a
VM with a desktop. In addition, install the required software (dependencies):

```
$ sudo dnf -y install golang php-cli git composer php-date php-filter php-hash \
    php-json php-mbstring php-pcre php-pdo php-spl php-sodium php-curl php-gd \
    unzip scdoc qrencode
```

## Debian >= 11

When you are not running Debian as your desktop OS, it is easiest to install a
VM with a desktop. In addition, install the required software (dependencies):

```
$ sudo apt install curl git build-essential php-sqlite3 composer php-curl \
    php-xml php-cli unzip golang qrencode scdoc
```

## Installation

Download the `development_setup.sh` script from this repository and run it. It
will by default create a directory `${HOME}/Project/eduVPN-v2` under which 
everything will be installed. No `root` is required!

```bash
$ curl -L -O https://codeberg.org/eduVPN/documentation/raw/branch/v2/development_setup.sh
$ sh ./development_setup.sh
```

## Testing

Most projects have unit tests included, they can be run from the project 
folder, e.g.: 

    $ cd ${HOME}/Projects/eduVPN-v2/vpn-user-portal
    $ make test

## Using

A "launch" script is included to run the PHP built-in web server to be able
to easily test the portals.

    $ cd ${HOME}/Projects/eduVPN-v2
    $ sh ./launch.sh

Now with your browser you can connect to the user portal on 
`http://localhost:8082/`.

You can login with the users `foo` and password `bar` or `admin` with password 
`secret`.

## VPN Configuration

To generate the OpenVPN server configuration files:

    $ cd ${HOME}/Projects/eduVPN-v2/vpn-server-node
    $ php bin/server-config.php

The configuration will be stored in the `openvpn-config` folder.

## Modifying Code

If you want to modify any of your code, it makes sense to switch to using your
own "fork". We'll only consider the eduVPN/LC components here and not the 
underlying dependencies.

For example if you want to modify `vpn-user-portal` you can do the following, 
we assume you already have an empty `vpn-user-portal` repository created on 
GitHub:

```
$ mkdir ${HOME}/tmp && cd ${HOME}/tmp
$ git clone --bare https://git.sr.ht/~fkooman/vpn-user-portal
$ cd vpn-user-portal
$ git push --mirror git@github.com/fkooman/vpn-user-portal
```

You can add GitHub as a remote now to your project checkout of 
`vpn-user-portal`:

```
$ cd ${HOME}/Projects/eduVPN-v2/vpn-user-portal
$ git remote add github git@github.com/fkooman/vpn-user-portal
```

Now when you make changes to the code you can push them to your own repository
"fork"

```
$ git branch my-feature-branch
$ git checkout my-feature-branch
...
$ git commit -a -m 'implement feature X'
$ git push github my-feature-branch
```

If this works properly locally, you can send a "pull request" or 
"merge request" to the upstream project, or even create a package repository
containing your own version of `vpn-user-portal` if necessary! 

When working with RPM packages for CentOS/Fedora you can easily create a 
package from a commit. For DEB packages you need to make a release first. What
I am doing is developing on CentOS/Fedora and only when considered stable I'll
make a release after which I also create Debian packages.

## Creating RPM packages

First install all requirements on your Fedora system as described 
[here](https://git.sr.ht/~fkooman/builder.rpm).

So we assume you have your own fork of `vpn-user-portal` as shown above. Now 
how to create an RPM package for it in your own development repository?

The `vpn-user-portal` package also has an RPM package repository, we assume you 
already have an empty `vpn-user-portal.rpm` repository created on GitHub::

```
$ mkdir ${HOME}/tmp && cd ${HOME}/tmp
$ git clone --bare https://git.sr.ht/~fkooman/vpn-user-portal.rpm
$ cd vpn-user-portal.rpm
$ git push --mirror git@github.com/fkooman/vpn-user-portal.rpm
```

Add the new remote to `rpm/vpn-user-portal.rpm`:

```
$ cd ${HOME}/Projects/eduVPN-v2/rpm/vpn-user-portal.rpm
$ git remote add github git@github.com/fkooman/vpn-user-portal.rpm
```

Now you can modify the SPEC file under 
`vpn-user-portal.rpm/SPECS/vpn-user-portal.spec` to point to your Git 
repository and commit. 

Modify the following lines, point `%global git` to your Git commit:

```
%global git 925933279f8f4b99e242d4c154be7c6c55aa91e6

Version:    2.4.8
Release:    0.1%{?dist}

...

Source0: https://github.com/fkooman/vpn-user-portal/archive/%{git}.tar.gz

...

%changelog
* Tue Feb 09 2021 Fran├žois Kooman <fkooman@tuxed.net> - 2.4.8-0.1
- update to development package
```

Set `Version` to one higher than the current version.
The `0.1` in the `Release` field indicates it is a pre-release version. Also 
update the `%changelog` entry.

You can try to build this package locally:

```
$ rpmdev-setuptree
$ cd ${HOME}/Projects/eduVPN-v2/rpm/vpn-user-portal.rpm
$ spectool -g -R SPECS/vpn-user-portal.spec
$ cp SOURCES/* ${HOME}/rpmbuild/SOURCES
$ rpmbuild -bs SPECS/vpn-user-portal.spec
$ rpmbuild -bb SPECS/vpn-user-portal.spec
```

Then you can commit your changes:

```
$ git commit -a -m 'update to development package'
$ git push github
```

Once this is complete, you can continue and create your own package repository 
by following the instructions [here](https://git.sr.ht/~fkooman/builder.rpm). 
Make sure you update `REPO_URL_BRANCH_LIST` and point it to your fork of the 
`vpn-user-portal.rpm` package.

If all goes well, you'll end up with your own RPM repository with your fork
of the package in it!

## Software Releases

The following script is used to make official releases of the components as 
`tar.xz`. It signs the source code with both PGP (for the Debian packages) and
with [Minisign](https://jedisct1.github.io/minisign/) for Fedora/CentOS. 
Please setup your own PGP and Minisign keys before running this script.

`${HOME}/.local/bin/make_release`:

```
#!/bin/sh
PROJECT_NAME=$(basename "${PWD}")
PROJECT_VERSION=${1}
RELEASE_DIR="${PWD}/release"

if [ -z "${1}" ]
then
    # we take the last "tag" of the Git repository as version
    PROJECT_VERSION=$(git describe --abbrev=0 --tags)
    echo Version: "${PROJECT_VERSION}"
fi

mkdir -p "${RELEASE_DIR}"
if [ -f "${RELEASE_DIR}/${PROJECT_NAME}-${PROJECT_VERSION}.tar.xz" ]
then
    echo "Version ${PROJECT_VERSION} already has a release!"
    exit 1
fi

git archive --prefix "${PROJECT_NAME}-${PROJECT_VERSION}/" "${PROJECT_VERSION}" -o "${RELEASE_DIR}/${PROJECT_NAME}-${PROJECT_VERSION}.tar.xz"
gpg2 --armor --detach-sign "${RELEASE_DIR}/${PROJECT_NAME}-${PROJECT_VERSION}.tar.xz"
minisign -Sm "${RELEASE_DIR}/${PROJECT_NAME}-${PROJECT_VERSION}.tar.xz"
```

Make the script executable:

    $ chmod +x ${HOME}/.local/bin/make_release

In order to upload the release, I'm using the `sr.ht` API. I am storing this
file under `${HOME}/.local/bin/sr.ht_upload_release`. You'll need to update the
locations. For other source hosting platforms it will be different.

```
#!/bin/sh
#POST /api/:username/repos/:name/artifacts/:ref
#
#Attaches a file artifact to the specified ref.
#
#Note: this endpoint does not accept JSON. Submit your request as multipart/form-data, with a single field: "file".

V=$(git describe --abbrev=0 --tags)
B=$(cat ${HOME}/.config/sr.ht/api.key)
R=$(basename $(pwd))

for F in release/*${V}*; do
	curl -H "Authorization: Bearer ${B}" -F "file=@${F}" "https://git.sr.ht/api/~fkooman/repos/${R}/artifacts/${V}"
done
```

## Creating DEB packages

Debian packages currently only work from proper releases. Updating the Debian 
packages is described [here](https://git.sr.ht/~fkooman/nbuilder.deb). You can
also use your own `vpn-user-portal.deb` fork to get going. Follow the 
instructions there.

D development_setup.sh => development_setup.sh +0 -127
@@ 1,127 0,0 @@
#!/bin/sh

REPO_BRANCH=v2
BASE_DIR=${HOME}/Projects/eduVPN-${REPO_BRANCH}
GIT_HOST=https://git.sr.ht/
#GIT_HOST=git@git.sr.ht:

mkdir -p "${BASE_DIR}"
cd "${BASE_DIR}" || exit

# clone repositories (read-only)
git clone -b "${REPO_BRANCH}" ${GIT_HOST}~fkooman/vpn-lib-common
git clone -b "${REPO_BRANCH}" ${GIT_HOST}~fkooman/vpn-user-portal
git clone -b "${REPO_BRANCH}" ${GIT_HOST}~fkooman/vpn-server-api
git clone -b "${REPO_BRANCH}" ${GIT_HOST}~fkooman/vpn-server-node
git clone -b "${REPO_BRANCH}" ${GIT_HOST}~fkooman/vpn-documentation
git clone ${GIT_HOST}~fkooman/vpn-portal-artwork-eduVPN
git clone ${GIT_HOST}~fkooman/vpn-portal-artwork-LC
git clone ${GIT_HOST}~fkooman/vpn-ca
git clone -b v1 ${GIT_HOST}~fkooman/vpn-daemon
git clone -b v1 ${GIT_HOST}~fkooman/vpn-maint-scripts
git clone ${GIT_HOST}~fkooman/builder.rpm
git clone ${GIT_HOST}~fkooman/nbuilder.deb

# clone all RPM/DEB packages
mkdir -p rpm deb
for PACKAGE_NAME in vpn-daemon vpn-lib-common php-jwt php-oauth2-server php-otp-verifier php-secookie php-sqlite-migrate vpn-ca vpn-portal-artwork-LC vpn-portal-artwork-eduVPN vpn-server-api vpn-server-node vpn-user-portal vpn-maint-scripts php-saml-sp php-saml-sp-artwork-eduVPN; do
	git clone ${GIT_HOST}~fkooman/"${PACKAGE_NAME}".rpm rpm/"${PACKAGE_NAME}".rpm
	git clone ${GIT_HOST}~fkooman/"${PACKAGE_NAME}".deb deb/"${PACKAGE_NAME}".deb
done

######################################
# vpn-user-portal                    #
######################################
cd "${BASE_DIR}/vpn-user-portal" || exit
mkdir -p data
composer update

cat << 'EOF' > config/config.php
<?php
$baseConfig = include __DIR__.'/config.php.example';
$localConfig = [
    //'styleName' => 'eduVPN',
    //'styleName' => 'LC',
    'secureCookie' => false,
    'apiUri' => 'http://localhost:8008/api.php',
];
return array_merge($baseConfig, $localConfig);
EOF

php bin/init.php
php bin/generate-oauth-key.php
php bin/add-user.php --user foo   --pass bar
php bin/add-user.php --user admin --pass secret

# symlink to the official templates we have so we can easily modify and test
# them
mkdir -p web/css web/img web/fonts
for TPL in eduVPN LC
do
    ln -s "${BASE_DIR}/vpn-portal-artwork-${TPL}/views"  "views/${TPL}"
    ln -s "${BASE_DIR}/vpn-portal-artwork-${TPL}/locale" "locale/${TPL}"
    ln -s "${BASE_DIR}/vpn-portal-artwork-${TPL}/css"    "web/css/${TPL}"
    ln -s "${BASE_DIR}/vpn-portal-artwork-${TPL}/img"    "web/img/${TPL}"
    ln -s "${BASE_DIR}/vpn-portal-artwork-${TPL}/fonts"  "web/fonts/${TPL}"
done

######################################
# vpn-ca                             #
######################################
cd "${BASE_DIR}/vpn-ca" || exit
go build -o vpn-ca tuxed.net/vpn-ca/cmd/vpn-ca/...

######################################
# vpn-daemon                         #
######################################
cd "${BASE_DIR}/vpn-daemon" || exit
go build -o _bin/vpn-daemon vpn-daemon/*.go

######################################
# vpn-server-api                     #
######################################
cd "${BASE_DIR}/vpn-server-api" || exit
mkdir -p data
composer update

cat << EOF > config/config.php
<?php
\$baseConfig = include __DIR__.'/config.php.example';
\$localConfig = [
    'vpnCaPath' => '${BASE_DIR}/vpn-ca/vpn-ca',
];
return array_merge(\$baseConfig, \$localConfig);
EOF

php bin/init.php

######################################
# vpn-server-node                    #
######################################
cd "${BASE_DIR}/vpn-server-node" || exit
mkdir -p data openvpn-config
composer update
cat << 'EOF' > config/config.php
<?php
$baseConfig = include __DIR__.'/config.php.example';
$localConfig = [
    'apiUri' => 'http://localhost:8008/api.php',
];
return array_merge($baseConfig, $localConfig);
EOF

######################################
# launch script                      #
######################################
cat << 'EOF' | tee "${BASE_DIR}/launch.sh" > /dev/null
#!/bin/sh
(
    cd vpn-server-api || exit
    php -S localhost:8008 -t web &
)
(
    cd vpn-user-portal || exit
    php -S localhost:8082 -t web &
)
EOF
chmod +x "${BASE_DIR}/launch.sh"