654d9e0d5121f40b9e91178bd31006520bcc7829 — Jakub Pastuszek 1 year, 1 month ago c8f5aa4
work on linux backup post
1 files changed, 213 insertions(+), 0 deletions(-)

A content/linux-backup.md
A content/linux-backup.md => content/linux-backup.md +213 -0
@@ 0,0 1,213 @@
title = "TBD"
draft = true
date = "2022-10-06"

tags = []
categories = []

image = "../wip.jpg"
image_alt = "Work in progress"

# Backing up Linux devices using common tools

I run Linux on many devices.

# Requirements 

* Network storage
* Incremental backups
* Encrypted storage
* Flexible to old backups
* Compression

# Initial setup

## iSCSI

It is a protocol allowing for access to storage devices on block level over network.

Using this protocol we can export a volume from network attached storage to the device we are backing up.

For iSCSI the Linux device we are backing up needs to use Linux kernel with iSCSI support available (e.g. via module).
Most desktop Linux distributions will have such support but PostmarketOS (e.g. on PinePhone) does not support it.
For such device we can still use other device that has the support to perform the backup via SSH.

One downside of using block devices is that only one system can connect to iSCSI volume at the same time.
This will mean that only one divec can run backup at the same time to the same iSCSI volume.
One can create dedicated volumes for each device needing backup.


0. NAS: Prepare iSCSI LUN (volume) on your NAS device using _chap_ authentication
1. Install user land tools package: `open-iscsi`
2. Set the _chap_ login/password in `/etc/iscsi/iscsid.conf`
3. Discover targets where `$BACKUP_ISCSI_IP` is the IP address of your NAS:
sudo iscsiadm -m discovery -t sendtargets -p $BACKUP_ISCSI_IP
4. Login/open iSCSI device where `$BACKUP_ISCSI_IQN` is a string provide by your NAS for the iSCSI LUN:
sudo iscsiadm -m node -p $BACKUP_ISCSI_IP -T $BACKUP_ISCSI_IQN --login
5. Identify the iSCSI device device name:
  - Note the PATH as `$ISCSI_DEV`
6. Verify that you have correct device (size and no partitions)
sudo fdisk -l $ISCSI_DEV


To encrypt the backups will will LUKS.

1. Encrypt `$ISCSI_DEV` with LUSK
sudo cryptsetup luksFormat --type luks1 $ISCSI_DEV
  - Pay attention to the message
  - Set strong password (use password manager!)
  - Make sure you have access to the password in case your device needs to be restored
2. Open the LUKS device as `/dev/mapper/backup`:
sudo cryptsetup luksOpen $ISCSI_DEV backup
  - Confirm the device is correct
  - Enter the password when prompted
3. Verify that you have `/dev/mapper/backup` device available
sudo fdisk -l /dev/mapper/backup

Now we have `/dev/mapper/backup` block device. 
Any data written to it will be encrypted and stored on the iSCSI volume one your NAS.


The BTRFS is a modern Linux file system that is becoming a default choice for many distributions.

In this backup setup the main feature that we will use is ability to do snapshot that allow us to access
the old backup as a separate file system directory structure.

Another advantage that we can exploit here is support for transparent file compression.


1. Create BTRFS file system on the encrypted device
sudo mkfs.btrfs /dev/mapper/backup
  - Note the _UUID_ value reported as `$BACKUP_BTRFS_DEV_UUID`
2. Now you can use _UUID_ provided by this command to identify  your LUKS volume and BTRFS file system
next time you are accessing the backup:

## Mounting the files system and preparing subvolume

1. Create the mount point
sudo mkdir /mnt/backup
2. Mount the filesystem
sudo mount UUID=$BACKUP_BTRFS_DEV_UUID /mnt/backup
3. Fix ownership of the mount point to your user account
sudo chown $USER /mnt/backup
4. Prepare subvolume for the backups of your host
btrfs subvolume create /mnt/backup/$(hostname)
  - Using subvolume this way will allow you to reuse this backup volume for other hosts

Now you can verify you have the file systme ready:

df -h /mnt/backup/
ls -l /mnt/backup/  

## Finish

Now we can umount the file system and close LUKS and iSCSI deiveces.

sudo umount /mnt/backup
sudo cryptsetup close backup
sudo iscsiadm -m node -p $BACKUP_ISCSI_IP -T $BACKUP_ISCSI_IQN --logout

Vefiry no iSCSI sessions are present for the backup volume:

iscsiadm -m session

# Performing backup

## Mount the backup

## Rsync

Using `rsync` you can back up your local file system.


* snapshot of your local fs

NAME=`date +%Y%m%d%H%M%S`
btrfs subvolume snapshot / /.snapshots/$NAME
cd  /.snapshots
test -e latest && rm latest
ln -s $NAME latest

* rsync command

rsync -a -XHA --delete -h --stats \
--exclude home/hxd/.cache/mozilla/firefox \
--exclude home/hxd/Downloads \
/mnt/root/ /mnt/backup/morgana/


rsync --rsync-path=rsync -a -XHA --delete -h --info=progress2 --stats -F /.snapshots/$DATE/ /mnt/backup/$(hostname)

TODO: .rsync-filter files

## Backup snapshot

After successfull `rsync` run we need to create a snapshot on the backup device.
This way we will be able to access old file in the future.

btrfs subvolume snapshot /mnt/backup/$(hostname) /mnt/backup/$(hostname)-$(date +%Y-%m-%d_%H-%M-%S)

## Cleanup

After all is done we need to cleanly unmount the file syste, close LUKS and iSCSI device.