How do I automate Kubernetes deployment YAML without relying on :latest?

I have a repository with a Kubernetes deployment YAML. Pipelines run on each commit that builds and pushes an image into our repository, versioned with the commit (eg. my_project:bm4a83). Then I'm updating the deployment image

my_project:bm4a83

kubectl set image deployment/my_deployment my_deployment=my_project:bm4a83.

kubectl set image deployment/my_deployment my_deployment=my_project:bm4a83

This works, but I also want to keep the rest of the deployment YAML specification in version control.

I thought I could just keep it in the same repository, but that means my changes that may only be infrastructure (eg, changing replicas) triggers new builds, without code changes.

replicas

What felt like it made the most sense was keeping the deployment YAML in a totally separate repository. I figured I can manage all the values from there, independently from actual code changes. The only problem with that is the image key would need to be kept up to date. The only way around that, is working with some floating :latest-type version, but I don't really think that's ideal.

image

:latest

What's a sensible workflow for managing this? Am I missing something entirely?

2 Answers
2

What's a sensible workflow for managing this? Am I missing something entirely?

Some of the answer depends on the kind of risk you're trying to drive down with any process you have in place. If it's "the cluster was wiped out by a hurricane and I need to recover my descriptors," then Heptio Ark is a good solution for that. If the risks are more "human-centric," then IMHO you will have to walk a very careful line between locking down all the things and crippling the very agile, empowering, tools that kubernetes provides to a team. A concrete example of that model running up against your model is: what happens when a developer edits a Deployment but does not (remember|know) to update the descriptor in the repo? So do you revoke the edit rights? Use some diff-esque logic to detect a changed in-cluster config?

To speak to something you said specifically: it is a highly suboptimal idea to commit a descriptor change just to resize a (Deployment|ReplicationController|StatefulSet). Separately, a well-built CI pipeline would also understand if no buildable artifact changed and bail out (either early, or not even triggering a build, if the CI tool is that smart).

(Deployment|ReplicationController|StatefulSet)

Finally, if you do want to carry on with the current situation, then the best practice I can think of is textual replacement right before applying a descriptor:

$ grep "image: " the-deployment.yml
image: example.com/something:#CI_PIPELINE_IID#
$ sed -i'' -e "s/#CI_PIPELINE_IID#/${CI_PIPELINE_IID}/" the-deployment.yml
$ kubectl apply -f the-deployment.yml


so that the copy in the repo remains textually pristine, and also isn't inadvertently actually applied since it won't actually result in a runnable Deployment.

but I also want to keep the rest of the deployment YAML specification in version control.

Yes, you want to do that. Putting everything under version control is a good practice to achieve immutable infrastructure.

If you want the deployment to have a separate piece of metadata (for whatever auditing / change tracking reason), why can't you just leverage the Kubernetes metadata block?

metadata

metadata:
name: my_deployment
commit: bm4a83


Then you inject such information through Jinja, Ruby ERBs, Go Templates, etc.

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.