Build Environment Manager
also get the exception
import traceback

refs

master
browse log

clone

read-only
https://git.sr.ht/~charles/bem
read/write
git@git.sr.ht:~charles/bem

You can also use your local clone with git send-email.

BEM -- Build Environment Manger

A simple tool to manage build environments based on Docker.

In short, BEM intends to reduce to a minimum the friction required for non-experts to use Docker for interactive development work.

Whenever you run bem [some command], it searches your current and any parent directories for a BEMfile, which is an INI file with configuration information. It will then build a Dockerfile (either in the same directory as BEMfile, or somwhere else if configured by CLI argument or the BEMfile), and execute your command within the resulting container.

BEM will automatically perform user account translation, and map (i.e. bind mount) your project directory (usually the location of the BEMfile) into the Docker container at the same absolute path as on your host system. BEM will also carry over your environment, as well as set up X11 access.

BEM is intended to make it easy to manage dependencies in a language agnostic, re-producible way that is accessible to non-experts (especially those who are unfamiliar with Docker).

Installation

python3 ./setup.py install

Usage

See bem --help.

Project Status

BEM is currently at a pre-alpha level of quality, and should not be used in production environments.

Goals

  • Be easy and simple to use for users in a way that does not require them to be familiar with Docker.
  • Operate on source files and build artifacts on the host filesystem, to avoid extra difficulty with moving data to and from Docker containers.
  • Be fast and convenient for those who know Docker already to manage build environments with.
  • Allow re-producible build environments to be used across many different systems, such as developer workstations and CI.

Non-Goals

BEM explicitly does not attempt to solve the following problems:

  • Security -- in short, you should not use BEM to run code you would not otherwise run.
    • In somewhat mode detail, BEM may mitigate certain types of attacks. It's probably roughly as safe as running code in a chroot as root. This is ostensibly safer than running code directly as root on the host OS, but if you intend to use BEM for security-related purposes, you better be very familiar with the security implications of Docker containers.
  • Replace docker-compose -- BEM is not intended for managing long-running Docker containers (i.e. containing servers), nor is it intended to orchestrating multiple Docker containers together.
  • Replace Kubernetes -- BEM is not intended as a cluster manager, nor for solving resource allocation or workload distribution.
  • Replace AppImage or other distribution or runtime package solutions.

FAQ

Why Not Just Use Docker?

Many developers are unfamiliar with Docker, since it is not directly relevant to their job or expertise. BEM seeks to enable these people to benefit from the positives of Docker without having to spend too much time learning new tools.

Additionally, BEM acts as a set of sane defaults for using Docker in an interactive context. Think oh-my-zsh or SpaceEmacs.

Why Not Kubernetes / Docker-Compose or Another Off The Shelf Tool?

None of these tools are optimized for convenient for interactive use while developing locally. In particular, these types of tools tend to be focused on running one or more long-running server processes, usually in a non-interactive fashion; BEM aims to do the opposite. Additionally, they (subjectively) require much more expertise to set up and use.

What Was the Motivation Behind BEM?

First, I wanted a convenient way to keep different build environments separated from one another. However, I don't often write code that could be described as a server process, usually write code that runs as a monolithic process (i.e. not as a set of networked services), and I don't want to have to go through the process of docker build, docker run every single time i want to do anything in my build environment.

Secondly, I hope to create a tool which is useful for other researches in my lab to help them solve the problem of dependency management. Many of these people are very smart, but also far too busy to learn to use a tool like Docker. BEM hopes to make it easier for us to collaborate on projects without every project member being deeply familiar with Docker.

Further, BEM aims to work with existing build systems, testing processes, and so on. If a project already builds with ./configure ; make, just running bem ./configure \; make will allow it to build in just the same way as before, without requiring any effort to port over (aside from writing a Dockerfile describing the program's Dependencies).

Why Not LXC/systemd-nspawn/FreeBSD Jails/Some Other Container Manager

I am not familiar with those systems. Docker is portable (relatively speaking) and widely used and understood. There may be other container management systems that could be used instead.

Isn't Running Everything Through Docker Slow?

Not really. It definitely adds to start-up time, especially if the image needs to be rebuilt. However, starting cold from an image that has already been built and cached, BEM only adds 1-2 seconds of additional start up time.

Configuration

Configuration Keys

The following keys may be specified in the [BEM] section of the BEMfile. Values are strings if not otherwise noted. Values may include other configuration keys for substitution using the syntax {key}. This makes it easy to, for example, specify a file relative to the project root.

  • minimum_version -- minimum BEM version required to use this BEMfile (default: current version)
  • dockerfile -- path to Dockerfile BEM should use (default: {bem_file_dir}/Dockerfile)
  • projectdir -- path to the directory BEM should treat as the project's top-level (default: {bem_file_dir})
  • nonineractive -- boolean indicating BEM should run in non-interactive mode (default: False) -- non-interactive mode is WiP and has several problems at the moment.
  • logfile -- when running in non-interactive mode, output should be written here; ignored in interactive mode (default: /dev/null)
  • flush_interval -- number of seconds between flushes to the logfile
  • noenvtamper -- set to True to prevent BEM from tampering with your environment, such as setting BEM_PROJECT and overriding PS1. Environment overrides are still honored. (default: False)
  • noenvpassthrough -- set to True to prevent BEM from passing through the host environment into the container. (default: False).
  • x11 -- set to True to bind mount in /tmp/.X11-unix. (default: True)
  • squashmethod -- specify how the user account should be squashed. (default: bind)
    • bind -- bind-mount in /etc/passwd, /etc/shadow, and /etc/group read-only, and set the UID within the container to the UID running BEM.
    • none -- do nothing and set the UID within the container to 0 (root).
  • enable_ptrace -- set to True to add the SYS_PTRACE capability to the container. (default: True)
  • network_mode -- the desired Docker network mode (default: host)

The following config keys should not be changed, but may be used via for substitution/indirection.

  • version -- the running BEM version
  • bem_file_dir -- parent directory of the current BEMfile

Configuration keys beginning with a # are reserved for internal use and should not be used for substitutions, nor overridden.

Environment Overrides

The section [environment] in the BEMfile may be used to specify environment variables that should be defined within the container. Such values will still be set even if noenvtamper or noenvpassthrough are asserted. Values placed in [indirectenvironment] will be expanded using the substitution syntax used for config values. If the same name exists in both [environment] and [indirectenvironment], the latter takes precedence.

Volmes

Any sections prefixed with volume_ must specify the following keys, and will be used to create bound volumes. These values do not support indirection (yet).

  • host_path -- path on the host system
  • container_path -- path in the container
  • mode -- ro or rw

Known Issues

  • Exit codes from processes within the container are not passed back up to the caller of BEM.
  • Although both stderr and stdout are returned, they are collectively squashed into stout.