1d00917bd6b111282b87070a9700807c9be00859 — Michael Kelly 2 months ago c0043ac
LXC containers part 1

I have a draft of part 2 as well.
A content/posts/lxc-containers-on-debian-p1.md => content/posts/lxc-containers-on-debian-p1.md +205 -0
@@ 0,0 1,205 @@
title: LXC Containers on Debian, Part 1 (Setup)
date: 2023-09-05T07:57:58-07:00

# Why LXC?

I'm looking for something between an application container (such as what Docker
and Podman do well) and a full-fledged VM. I do all my work in KVM guests that
run on a server at home for isolation and ease of management. But it seems
inefficient to always virtualize everything, when all my guessts are Linux

So, I'll try replacing some of my long-lived KVM guests with LXC containers.

LXC is one of the original containerization toolsets on Linux. It's very
flexible, it focuses on whole-OS containers, and it can run nicely on the same
host as KVM.

# Setup

The two resources I found most helpful were:
* [LXC "getting started"
* [Debian LXC wiki page](https://wiki.debian.org/LXC)

However, there are a number of decisions you can make, and a few setup steps
that aren't well covered. Additionally, many pages on the internet that talk
about "LXC" are actually talking about [LXD](https://ubuntu.com/lxd). The goal
of this post is to supplement the authoritative docs above and describe a
complete, specific installation.

I chose to use unprivileged containers with a host-shared bridge setup. My host
is running Debian 11.7.

I'm running Debian containers as well. (Which container you run shouldn't make
a big difference, but there may be some image-specific tweaks you need to

I'll write a followup covering customizing/provisioning containers.

## Unprivileged Containers

The [LXC security wiki page](https://linuxcontainers.org/lxc/security/) takes a
strong stance that such containers can't ever be secure. I'd like the extra
isolation if I can get it.

This requires some id-mapping configuration in `~/.config/lxc/default.conf`,
`/etc/subuid`, and `/etc/subgid`.

Based on [LXC

Copy `/etc/lxc/default.conf` to `~/.config/lxc/default.conf`.

In `~/.config/lxc/default.conf`, add:
lxc.idmap = u 0 100000 65536
lxc.idmap = g 0 100000 65536

Full contents of `/etc/subuid`:

Full contents of `/etc/subgid`:

**`mkelly` is my username here.** Substitute yours.

## Networking: Host-Shared Bridge Setup

This is appropriate for containers you want to be accessible outside the host.
If you're also using the host for VMs with KVM (which I am), you can use the
same bridge for both KVM and LXC.

Setting up the bridge itself is well-documented elsewhere. You can follow the
[Debian wiki](https://wiki.debian.org/BridgeNetworkConnections) here. For
reference, my `/etc/network/interfaces` looks like this:
iface eno1 inet manual
auto br0
iface br0 inet static
  bridge_ports eno1
    bridge_stp off
    bridge_fd 0
    bridge_maxwait 5

This sets a static IP. `br0` is my bridge interface, `eno1` is my physical
interface. You'll have to follow more steps to integrate this with LXC.

In `~/.config/lxc/default.conf`:
lxc.net.0.type = veth
lxc.net.0.link = br0
lxc.net.0.flags = up

Network device quotas: In `/etc/lxc/lxc-usernet`, I put:
mkelly veth br0 10

**`mkelly` is my local username, `br0` is the name of my bridge
device.** You'll need to make both consistent with your setup.

## Apparmor

This is a big source of container startup failures and mysterious issues inside

**I used the `unconfined` Apparmor profile**, which allowed me to start
containers and avoided mysterious networking issues inside the containers once
they started.

In `~/.config/lxc/default.conf`:
lxc.apparmor.profile = unconfined

Using the `generated` profile did not work for me -- `systemd-networkd` was
unable to start on Debian containers. More info in [this Github

## Host Permissions

Before starting any containers: `chmod -R +x ~/.local/share/` -- otherwise, you
will get permission errors when trying to start containers.

## Creating a container

At this point, you should be able to create a container!

LXC comes with quite a few templates to install different Linux distros, but
[the distro-specific shell script templates are
deprecated](https://brauner.io/2018/02/27/lxc-removes-legacy-template-build-system.html), so I use only the `download` template, which downloads one of many pre-built images. It is plenty flexible for me.

Per [LXC instructions](https://linuxcontainers.org/lxc/getting-started/), I
wrap all LXC commands in `systemd-run` when interacting with unprivileged

As your non-root user, you can create a container called `container1`, which
will prompt for distro, release, and architecture:
systemd-run --user --scope -p "Delegate=yes" -- lxc-create -t download -n container1
I chose `debian` / `bookworm` / `amd64`. You can provide answers to all the
questions the `download` template asks interactively with by adding `-- --dist
$dist --release $release --arch $arch --variant $variant`

Then start the container:
systemd-run --user --scope -p "Delegate=yes" -- lxc-start container1

Then attach to the container to get a shell:
systemd-run --user --scope -p "Delegate=yes" -- lxc-attach container1

That's it!

With `ip a` you should see that you have network addresses:
root@container1:/# ip a
2: eth0@if28: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether a6:23:bf:d4:33:70 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet metric 1024 brd scope global dynamic eth0
       valid_lft 43149sec preferred_lft 43149sec

On a Debian container, you won't be able to ping anything until you run the
final container setup step below.

## Container Fixes

Running a Debian container, there was one strange fix required to allow
unprivileged users (in the container) to use `ping`:

# in the container
/sbin/setcap cap_net_raw+p /bin/ping

It looks like this is an issue with the process that creates LXC images, based
on reading through this [Proxmox forum

## Next Steps

Now that we can create containers, the next step for me is automatically
creating lightly-customized containers that I can ssh to as my personal user,
with a single shell command.

I'll cover that in the next part.

A content/posts/lxc-containers-on-debian-p2.md => content/posts/lxc-containers-on-debian-p2.md +38 -0
@@ 0,0 1,38 @@
title: LXC Containers on Debian, Part 2 (Provisioning)
date: 2023-09-05T08:20:05-07:00
draft: true

This is a followup to [part 1 on LXC containers]({{< ref
"lxc-containers-on-debian-p1.md" >}}). The beginning of that post explains why
I'm using LXC for this task specifically.

# Provisioning Options

There are a lot of different ways to provision containers.

* Custom images
    * with [distrobuilder](https://linuxcontainers.org/distrobuilder/docs/latest/tutorials/use/#create-an-image-for-lxc).
    * with [cloudinit](https://cloudinit.readthedocs.io/en/latest/index.html).
* Post-boot provisioning
    * with [Ansible](https://docs.ansible.com/). Or any
      other configuration management tool -- Chef or Salt would work fine, too.
    * with a shell script. This is undoubtedly the
      lowest-tech option, so that's what we're going to do first.

# Provisioning with a shell script

I'm not trolling you. There are some benefits to doing it this way:

* This is the lowest-common-denominator method, so it's actually pretty good as
  a demo. It's more useful to show someone an example in bash (which everyone
  in this field probably understands) than an example in a configuration
  management tool they don't use.
* We get to show some of the very interesting quirks of `lxc-attach`, which
  actually play in our favor here. It's good to know them to avoid significant
  confusion if you're trying to automate simple one-off changes later.
* We're just trying to get a usable container -- not set up an app for a
  production environment -- so our needs are actually pretty minimal. I
  wouldn't set up a Kubernetes node this way.
* It's a nice demo of what you can do with absolutely no setup.