#Org-mode test infrastructure

Org-mode comes already with a Makefile that defines make test.

This functionality now needs to be brought to automatic CI servers (even if it's only used as convinience function).

On top of that many advanced features like org-babel need many prerequisites installed, that most people will not have on their machines.

This repository tried to address those two issues:

#Sourcehut builds

This repository contains SourceHut (sr.ht) build manifests to test latest Org against the three latest stable versions of GNU Emacs (26, 27, 28) in Debian (oldstable, stable, unstable. Those manifests are stored in the .builds folder.

The .builds directory also contains other manifests for performing tests when a change is pushed on this repository.

The .builds_maintainers directory contains the same manifests except that warning and errors are sent to maintainers only, not to the ~bzg/org-build-failures@lists.sr.ht mailing list.

#Reproducible local builds

The local folder help to execute tests in predefined environments. It tries to match the builds that are done on [https://builds.sr.ht] locally by using podman as container engine.

• Dockerfile.debian.emacs26 Environment for emacs 26 on debian
• Dockerfile.debian.emacs27 Environment for emacs 27 on debian
• Dockerfile.debian.emacs28 Environment for emacs 28 on debian
• Dockerfile.ubuntu Environment for emacs 26, 27, 28 (parametrized via build-arg) on ubuntu
• build.sh (entry point into the image to run make, used inside the container)
• init.el (GNU Emacs configuration and test dependencies)
• rakefile.rb ((rake)[https://github.com/ruby/rake]) file to build and run the images). Please run rake -T to see all available tasks. In general its build|run|emacs|test:OS|EMACS_VERSION.
• run.sh (entry point to run Emacs ready for interactive testing, used inside the container)

#Usage

Because the org-mode-tests repository is separated from the org-mode sources, those need to be linked into the org-mode-tests folder at org-mode (e.g. ln -s ~/org-mode . when in the org-mode-tests directory). Then see rake -T for available tasks.

#Images

The images create by the rakefile are all named org-mode-build-image:OS:EMACS_VERSION.

#Environment Variables

Some environment variables are passed to the container to fine tune the tests:

• ORG_TEST_ERROR_ON_WARN (true|false) is used to fail the build also on warnings (e.g. byte-compile warnings).
• ORG_TEST_REGEX (regex) is used to run only a subset of tests.

#Rules

Org tests are run from the orgmode.org. A cronjob calls the mk/run-tests.sh script which triggers the builds.

Here are the rules:

• If the repository didn't change, don't run the tests.

• If the repository changed, report new test failures to the org-build-failures mailing list.

• If the repository changed and the last test against a specific Emacs version failed, report new failures against this Emacs version to the test maintainers.

#TODO

• Describe the requirements for each .yml file more precisely.

#License

Code in this repository is released under the GNU General Public License 3.0 or any later version of it (see COPYING).