~bzg/org-mode-tests

Run Org tests
mk/run-tests.sh: Complete previous commit
mk/run-tests.sh: Ensure to find the hut executable
mk/run-tests.sh: Sync tags and badges names

refs

master
browse  log 

clone

read-only
https://git.sr.ht/~bzg/org-mode-tests
read/write
git@git.sr.ht:~bzg/org-mode-tests

You can also use your local clone with git send-email.

#Org-mode test infrastructure

Org-mode comes already with a Makefile that defines make test.

This functionality now needs to be brought to automatic CI servers (even if it's only used as convinience function).

On top of that many advanced features like org-babel need many prerequisites installed, that most people will not have on their machines.

This repository tried to address those two issues:

#Sourcehut builds

This repository contains SourceHut (sr.ht) build manifests to test latest Org against the three latest stable versions of GNU Emacs (26, 27, 28) in Debian (oldstable, stable, unstable. Those manifests are stored in the .builds folder.

The .builds directory also contains other manifests for performing tests when a change is pushed on this repository.

The .builds_maintainers directory contains the same manifests except that warning and errors are sent to maintainers only, not to the ~bzg/org-build-failures@lists.sr.ht mailing list.

#Reproducible local builds

The local folder help to execute tests in predefined environments. It tries to match the builds that are done on [https://builds.sr.ht] locally by using podman as container engine.

#Usage

Because the org-mode-tests repository is separated from the org-mode sources, those need to be linked into the org-mode-tests folder at org-mode (e.g. ln -s ~/org-mode . when in the org-mode-tests directory). Then see rake -T for available tasks.

#Images

The images create by the rakefile are all named org-mode-build-image:OS:EMACS_VERSION.

#Environment Variables

Some environment variables are passed to the container to fine tune the tests:

  • ORG_TEST_ERROR_ON_WARN (true|false) is used to fail the build also on warnings (e.g. byte-compile warnings).
  • ORG_TEST_REGEX (regex) is used to run only a subset of tests.

#Rules

Org tests are run from the orgmode.org. A cronjob calls the mk/run-tests.sh script which triggers the builds.

Here are the rules:

  • If the repository didn't change, don't run the tests.

  • If the repository changed, report new test failures to the org-build-failures mailing list.

  • If the repository changed and the last test against a specific Emacs version failed, report new failures against this Emacs version to the test maintainers.

#TODO

  • Describe the requirements for each .yml file more precisely.

#License

Code in this repository is released under the GNU General Public License 3.0 or any later version of it (see COPYING).