Published

forgejo-runner image with mkosi

Codeberg doesn't have much CI capacity so I bring my own action runners to run workflows in my Codeberg repositories. I use mkosi to build a generic system image containing the runner as well as all dependencies I need for CI, which I can then deploy on any system (e.g. spare laptops at home, hosted cloud VMs, etc.) that uses systemd. This post shows the image definition.

Base image

The base image is pretty straight-forward. We build a discoverable disk image based on Archlinux, configure timezone, locale, hostname, and keymap, and install a basic set of packages:

[Output]
ImageId=forgejo-runner
Format=disk
CompressOutput=true

[Distribution]
Distribution=arch

[Content]
Bootable=false
Timezone=UTC
Keymap=us
Locale=C.UTF-8
Hostname=forgejo-runner-????-????
Packages=
    # Base packages
    base
    base-devel
    # pacman-offline to auto-update the runner system
    pacman-offline
    # The forgejo runner daemon itself
    forgejo-runner
    # node 24, as required by actions
    nodejs-lts-krypton
    # Git, obviously
    git
    # Extra dependencies for the actual CI jobs
    rustup
    npm
    # ... whatever you need

We also need a preset file to enable the pacman offline timer for auto-updates as well as the forgejo runner service. We also disable homed while at it, because we don't need user management in a single-purpose disk image. The preset file goes into the default extra tree below mkosi.extra where mkosi automatically picks it up and includes it in the image.

$ cat mkosi.extra/usr/local/lib/systemd/system-preset/10-forgejo-runner.preset
disable systemd-homed.service
disable systemd-homed-activate.service
enable forgejo-runner.service
enable pacman-offline-prepare.timer

Disk layout

By default, mkosi tries to minimize the size of all partitions in the disk image. However, we do need a large /var for toolchains, caches, build outputs, etc., so we adjust the partitions created in the disk image with two repart configuration files:

mkosi.repart/10-root.conf:

[Partition]
Type=root
Format=btrfs
CopyFiles=/
Minimize=guess

mkosi.repart/20-var.conf:

[Partition]
Type=var
CopyFiles=/var:/
Format=btrfs
SizeMinBytes=10G
SizeMaxBytes=10G

Runner configuration

Next we initialize a default configuration for the forgejo-runner:

$ mkdir mkosi.extra/etc/forgejo-runner
$ curl https://code.forgejo.org/forgejo/runner/raw/branch/main/internal/pkg/config/config.example.yaml \
   > mkosi.extra/etc/forgejo-runner/config.yaml

The defaults are good; we just need to adapt them slightly to remove some example values, reduce the job timeout, and pick some more reasonable directories:

diff --git a/mkosi.extra/etc/forgejo-runner/config.yaml b/mkosi.extra/etc/forgejo-runner/config.yaml
index 5addac4..420c43d 100644
--- a/mkosi.extra/etc/forgejo-runner/config.yaml
+++ b/mkosi.extra/etc/forgejo-runner/config.yaml
@@ -25,15 +25,13 @@ runner:
   capacity: 1
   # Extra environment variables to run jobs.
   envs:
-    A_TEST_ENV_NAME_1: a_test_env_value_1
-    A_TEST_ENV_NAME_2: a_test_env_value_2
   # Extra environment variables to run jobs from a file.
   # It will be ignored if it's empty or the file doesn't exist.
   env_file: .env
   # The timeout for a job to be finished.
   # Please note that the Forgejo instance also has a timeout (3h by default) for the job.
   # So the job could be stopped by the Forgejo instance if it's timeout is shorter than this.
-  timeout: 3h
+  timeout: 1h
   # The timeout for the runner to wait for running jobs to finish when
   # shutting down because a TERM or INT signal has been received.  Any
   # running jobs that haven't finished after this timeout will be
@@ -92,7 +90,7 @@ cache:
   #
   # If empty, the cache data will be stored in $HOME/.cache/actcache.
   #
-  dir: ""
+  dir: "/var/lib/forgejo-runner/cache"
   #
   #######################################################################
   #
@@ -191,4 +189,4 @@ container:
 host:
   # The parent directory of a job's working directory.
   # If it's empty, $HOME/.cache/act/ will be used.
-  workdir_parent:
+  workdir_parent: /var/lib/forgejo-runner/act

Register runner from credentials

We also need to register the runner with Codeberg (or any other Forgejo instance). We could do this by hand: open a shell in the image and run forgejo-runner register. However, it's a lot more convenient to do this automatically when booting the image. Systemd has a concept of credentials which we can use to inject registration data into the image when booting. Inside the image we can then automatically register the runner if it's not registered yet.

For this purpose we add a small one-shot service mkosi.extra/usr/local/lib/systemd/system/forgejo-runner-register.service:

[Unit]
Description=Forgejo Runner Registration
After=network-online.target
Wants=network-online.target
ConditionPathExists=!/var/lib/forgejo-runner/.runner

ConditionCredential=forgejo.runner.name
ConditionCredential=forgejo.runner.instance
ConditionCredential=forgejo.runner.labels
ConditionCredential=forgejo.runner.token

[Service]
Type=oneshot
RemainAfterExit=true
ExecStart=/usr/local/bin/forgejo-runner-register-from-credentials
User=forgejo-runner
WorkingDirectory=/var/lib/forgejo-runner

ImportCredential=forgejo.runner.*

This service only runs when the runner is not registered (ConditionPathExists) and all secrets for registration are available (ConditionCredential). It imports all registration credentials from the service manager and calls a small script at mkosi.extra/usr/local/bin/forgejo-runner-register-from-credentials which simply callsforgejo-runner register with the credential values:

#!/usr/bin/bash
set -Euo pipefail
exec /usr/bin/forgejo-runner register \
    --no-interactive \
    --name "$(systemd-creds cat forgejo.runner.name)" \
    --instance "$(systemd-creds cat forgejo.runner.instance)" \
    --labels "$(systemd-creds cat forgejo.runner.labels)" \
    --token "$(systemd-creds cat forgejo.runner.token)"

Then we arrange for this service to be started before the actual forgejo-runner with a small

[Unit]
After=forgejo-runner-register.service
Requires=forgejo-runner-register.service
ConditionFileNotEmpty=/var/lib/forgejo-runner/.runner
ConditionFileNotEmpty=/etc/forgejo-runner/config.yaml

Build and launch

We build the image with mkosi and import the resulting forgejo-runner.raw.zst as a machine image:

$ importctl import-raw --class=machine \
   forgejo-runner.raw.zst codeberg-runner

Now we obtain a registration token and boot the image manually to pass credentials to register the runner:

$ systemd-nspawn --boot --link-journal=try-guest -U \
   --machine codeberg-runner \
   --set-credential=forgejo.runner.instance:https://codeberg.org \
   --set-credential=forgejo.runner.name:foo-runner \
   --set-credential=forgejo.runner.labels:foo:host \
   --set-credential=forgejo.runner.token:THE_TOKEN

From now on we can use machinectl to boot the runner, e.g. machinectl start codeberg-runner.

Launching and registering a new runner is now a matter of importctl, systemd-nspawn, and machinectl, and I can launch a new runner within minutes. We can now spin up an e.g. Alma Linux VM on Hetzner cloud and get a runner up and running within a few minutes, all by hand, even without heavy automation by e.g. terraform, and dispose of it again when the runner is not needed any more which makes for a simple and cost-effective way to do CI on Codeberg.