blog

SEAN K.H. LIAO

argocd

We've settled on Argo CD for our continuous deployments. Some other parts of our stack: helm for packaging kubernetes manifests and helm-gcs (bash version) for a private helm repository.

doesn't work

So a colleague was tasked with setting it up. Everything worked more or less as expected, well almost everything, they said the helm-gcs plugin didn't work. Sprint planning comes around and now there's a ticket for solving the issue with 3 options: fixing it as an upstream bug (linking to #4439), using an alternate chart repository, and something else i forgot. Side note: an option would also be to expose the bucket as a static site. Noises were made about it being a difficult problem, I ask if they tried the other helm-gcs and it was promptly dumped on me.

debugging

Ah, what do we do? Try to reproduce the problem of course.

What next? Try to see what's happening.

That was a dead end, time to get my hands dirty

The helm-gcs we use is a bash script that wraps gsutil which is itself written in python. Argo itself is written in Go, so the resources were sized accordingly, request/limit of 25/50m cpu and 64/128Mi memory. Python is hungry and fat.

I believe my original proposition of using the other helm-gcs (written in Go) might also have worked, but wouldn't have uncovered this issue of resource starvation.