systemd as a build system

systemd eats the world

SEAN K.H. LIAO

systemd as a build system

systemd eats the world

build system

So a build system (for a CI process) is usually a collection of program executions that have various dependency requirements, which come out to be a directed acyclic gragh (DAG). You know what manages processes and has a rich dependency graph solver? systemd.

notes

execution

Type=oneshot most processes in CI run a single execution, ie they aren't persistently running in the background. Using oneshot also means you can have multiple ExecStart= lines, running one after the other (replacing && in shell scripting).

ExecStart=/path/to/exe where your main processes go. -/path/to/exe ignores failures. ExecStartPre= isn't very useful in the oneshot scenario.

ExecStartPost= only runs on successful completion, while ExecStopPost= always runs.

ordering

There are 2 main ways to order dependencies:

OnSuccess=, OnFailure=: This way you start with a root unit, and say which ones to trigger afterwards in a feed forward manner. It's an imperative chain that limits the possibility of reuse.

After= + Requires: Using these 2 you specify the end target, and the dependencies it has on previous units. works out which ones it needs to run to get to the end result. Requires= triggers a service to start successfully, while After= waits for a service to complete. Wants= doesn't enforce successful exit of the dependency.

There are also the inverse Before=, RequiredBy, and WantedBy, but they really only have an effect when everything is installed+enabled.

.target files are useful when since with the graph method you need to know the end state, and it's unlikely your last step is very memorable. It's also a good place to attach the notification handlers with OnSuccess= and OnFailure. Unfortunately, they only run once (unless explicitly targeted by a systemctl start command, so you may want to consider replacing them with a dummy service that just echos something).

passing values

Sometimes you need to pass values between the different steps. You could: write out config files for later programs or write key=value environment values to be passed in via EnvironmentFile (and optionally used as flags/args via ExecStart=/bin/exe $SOME_ENV).

instances

While you may think that you can reuse config with unit@.service and template common executions, it may be less useful than it appears, as you have no good way of specifying the dependencies before it.

isolation

Unfortunately, there's no way to say "run all these units in a single chroot". So your options are probably:

example

 1# clone.service
 2[Unit]
 3Description=clone git repo
 4[Service]
 5Type=oneshot
 6WorkingDirectory=/tmp
 7ExecStart=/usr/bin/git clone https://github.com/example/repo
 8
 9# setup.target
10[Unit]
11Description=setup
12Requires=clone.service
13After=clone.service
14
15---
16# build.service
17[Unit]
18Description=build code
19Requires=setup.target
20After=setup.target
21[Service]
22Type=oneshot
23WorkingDirectory=/tmp/repo
24ExecStart=/usr/bin/mkdir -p bin
25ExecStart=/usr/bin/go build -o bin/ ./...
26
27---
28# test.service
29[Unit]
30Description=test code
31[Service]
32Type=oneshot
33WorkingDirectory=/tmp/repo
34ExecStart=/usr/bin/go test ./...
35
36---
37# vet.service
38[Unit]
39Description=vet code
40[Service]
41Type=oneshot
42WorkingDirectory=/tmp/repo
43ExecStart=/usr/bin/go vet ./...
44
45---
46# build.target
47[Unit]
48Description=build and test
49Requires=build.service vet.service test.service
50After=build.service vet.service test.service
51OnSucess=post-success.service
52OnFailure=notify-failure.service
53
54---
55# post-success.service
56[Unit]
57Description=notify on success
58[Servuce]
59ExecStart=/usr/bin/curl ...
60
61---
62# notify-failure.service
63[Unit]
64Description=notify on failure
65[Service]
66ExecStart=/usr/bin/slack-notify ...