Engineering Blog

Today I Learned: GitLab Fleeting Edition

In collaboration with Puzzle ITC's Yannik Dällenbach, I have recently worked to publish fleeting-plugin-cloudscale. With this plugin it is possible to dynamically scale Gitlab runner instances on cloudscale.ch.

While the bulk of the work is all thanks to Yannik's efforts, I had the chance to integrate his work into our open source landscape. This is what I learned.

GitLab Fleeting Works Well

Originally, GitLab used the Docker machine driver, to offer autoscaled runner instances. This tool has been deprecated for years. The alternative takes the form of a Go plugin.

With only a few short functions, Yannik was able to implement the necessary logic to launch, list, and cleanup runner instances. It's easy to write such a framework in a way that makes things cumbersome. GitLab has avoided that pitfall.

The GitLab runner process uses the plugin to react to tasks popping up in the CI pipeline, ensuring there is always enough capacity when it is actually needed.

Fleeting Plugin Packaging Is Different

I figured that publishing this plugin would involve a container image. This turned out to be right, but how GitLab does it was a bit unexpected:

Using a bespoke fleeting-artifact command, we build a container image, given a set of architecture-specific binaries. Go's cross-compilation story is top-notch, so providing these binaries is easy, but it is unexpected to not use generic container image build tools.

I think the approach has its merits, but I might have saved some time, had I known about it sooner. I only realized my container images did not work, after I built them using ko.

Go 1.24 Tools

This was my first project where I got to try out Go 1.24's go tool command. It is used to integrate tools related to the development of the project into dependency tracking. In our case, this enabled me to integrate go-releaser and the mentioned fleeting-artifact.

To install these tools, I ran the following:

go get -tool -modfile tool.mod github.com/goreleaser/goreleaser/v2@latest
go get -tool -modfile tool.mod gitlab.com/gitlab-org/fleeting/fleeting-artifact/cmd/fleeting-artifact
go mod tidy -modfile tool.mod

This makes these tools available as follows:

go tool -modfile tool.mod fleeting-artifact
go tool -modfile tool.mod goreleaser

What I like about this approach, though it is a bit verbose, is the fact that I can run the exact same tool locally, and in the CI, and across all our workstations.

Dev-tooling should be versioned as well, to keep its results stable, and go tool accomplishes that.

Zizmor

I like linters, static analyzers, language-servers, and so on. Anything that supplements my knowledge with community-wisdom. I didn't know about Zizmor before, but I rarely write GitHub workflows, and I figured: Someone must have written a validator for these.

Indeed someone has: Zizmors goal is to protect the user from the following (and more):

• Template injection vulnerabilities, leading to attacker-controlled code execution.
• Accidental credential persistence and leakage.
• Excessive permission scopes and credential grants to runners.
• Impostor commits and confusable git references.

It's not precisely a validator, or at least I did not come to rely on it as such, but it pointed out some potential security issues with my GitHub workflows, and I learned some newer directives I missed before.

As a result: The GitHub workflows in-place for fleeting-cloudscale-plugin are now hardened and checked for issues on each commit.

Conclusion

Adopting and packaging someone else's hard work was a somewhat new experience for me. With most of the groundwork already laid out, I was able to spend more time on packaging and testing, which the final result reflects I think.

If you want to check out our plugin, see:

https://github.com/cloudscale-ch/fleeting-plugin-cloudscale

If you want to demo it, use our Ansible playbook that configures a GitLab instance, adds a Gitlab runner, the fleeting-plugin-cloudscale plugin, and optionally a distributed S3 cache, all in one go:

https://github.com/cloudscale-ch/gitlab-runner

You'll be presented with a GitLab instance where CI jobs automatically work, scale, and share their cache. It's a great demo, and a good starting point to introduce GitLab into your own organization. Everything inside our infrastructure, far away from Five Eyes.

Wenn du uns Kommentare oder Korrekturen mitteilen möchtest, kannst du unsere Engineers unter engineering-blog@cloudscale.ch erreichen.