SEANK.H.LIAO

k8s ordered container execution

run your steps in order

k8s ordered container execution

In kubernetes, pods have 2 places where you can specify containers to run: initContainers and containers. initContainers start in the order they're specified and only after the previous has finished (unless it's a 1.28 sidecar with restartPolicy: Always). Once that's done, containers start in the order they're specified (is this an implementation detail leaking?) without depending on each other for lifecycle.

So if you're creating a task runner and have an ordered list of steps to execute, you might think: oh i'll just put it all in initContainers. Which might be fine... until you decide that what you actually need is an graph (DAG) of steps, rather than a linear list.

This is where something like the tekton project's entrypoint come in. An executor command is copied / mounted into every step, and wraps the actual command. It writes to a specified file when the actual command exits, and waits for files from previous steps to be present before starting its command. This does have a slight disadvantage: after the first step is complete, the container exits (remember to set restartPolicy: Never), and the pod enters into NotReady state, even though for our purposes, it's executing as expected.

Usage is slightly verbose:

 1apiVersion: v1
 2kind: Pod
 3metadata:
 4  name: tekton-ordered-0
 5  namespace: default
 6  labels:
 7    experiment: ordered
 8spec:
 9  restartPolicy: Never
10  initContainers:
11    # copy itself to a volume that can be mounted in every container
12    - name: copy-entry
13      image: gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/entrypoint:v0.52.0
14      command:
15        - /ko-app/entrypoint
16        - cp
17        - /ko-app/entrypoint
18        - /ordered/bin/entrypoint
19      volumeMounts:
20        - name: bin # shared volume with executor
21          mountPath: /ordered/bin
22  containers:
23    - name: one
24      image: alpine
25      command:
26        - /ordered/bin/entrypoint # our entrypoint command
27        - -post_file=/ordered/steps/one # write to here on exit
28        - -termination_path=/ordered/steps/termintaion # termination log, nicer k8s status
29        - -entrypoint=sh # actual thing to be executed
30        - -- # everything after this is passed to the subprocess
31        - -c
32        - |
33          echo step one
34          echo hello world          
35      terminationMessagePath: /ordered/steps/termination # termination log
36      terminationMessagePolicy: File
37      volumeMounts:
38        - name: bin # shared volume with executor
39          mountPath: /ordered/bin
40        - name: steps # shared volume for step status
41          mountPath: /ordered/stepsered/steps
42
43    - name: two
44      image: alpine
45      command:
46        - /ordered/bin/entrypoint
47        - -wait_file=/ordered/steps/one # wait for file from previous step to be present before proceeding
48        - -post_file=/ordered/steps/two
49        - -termination_path=/ordered/steps/termintaion
50        - -entrypoint=sh
51        - --
52        - -c
53        - |
54          echo step two
55          echo fizz buzz          
56      terminationMessagePath: /ordered/steps/termination
57      terminationMessagePolicy: File
58      volumeMounts:
59        - name: bin
60          mountPath: /ordered/bin
61        - name: steps
62          mountPath: /ord
63
64    - name: three
65      image: alpine
66      command:
67        - /ordered/bin/entrypoint
68        - -wait_file=/ordered/steps/two
69        - -post_file=/ordered/steps/three
70        - -termination_path=/ordered/steps/termintaion
71        - -entrypoint=sh
72        - --
73        - -c
74        - |
75          echo step three
76          echo lorem ipsum          
77      terminationMessagePath: /ordered/steps/termination
78      terminationMessagePolicy: File
79      volumeMounts:
80        - name: bin
81          mountPath: /ordered/bin
82        - name: steps
83          mountPath: /ordered/steps
84  volumes:
85    - name: bin
86      emptyDir: {}
87    - name: steps
88      emptyDir: {}