After traveling for a long time I started playing with tech again. I started building a crypto currency trading app for the Kraken Exchange API. The resulting app can be downloaded here: expo.io/@morriz/krakenfx-react-native.
But then I started playing with Kubernetes again, and started working on mostack: a stack with Kubernetes best practices. This was a hard and long road past obscure pitfalls and learnings. Some I just have to give back in the hope you may avoid them.
Drone CI/CD.
To automate software building we need a CI/CD build system. I chose to go with Drone, as I like the simplicity of working with docker containers, and it’s open source and not SaaS. But Drone uses Docker in Docker (dind) and that gave me the following problem:
Drone starts the host docker container running the dind with a custom network. Probably for good reasons, but this makes it impossible to resolve any cluster ips from known kubernetes service names.
I needed to docker push to a locally running docker-registry service, as well as make kubectltell the api server to update deployments. Since there is no way around this, I had to use the host docker socket and manually instrument the wiring of the plugins. Including the custom dns settings. Please see the .drone.yml in the morriz/nodejs-demo-api how I did that. For more information around my dns related issues see my posts in the drone discourse .
Helm
The biggest challenge in k8s userland is the deployment of the manifests. Ideally one would like to have a uniform approach to apply the entire new desired cluster state in one go. Preferably automated after a git push to the cluster repo. For now I chose to experiment with Helm, which allows me to make one root ‘Chart’ (the name they use for a ‘package’) for the entire cluster, with app subcharts that describe the components running on the cluster. But somehow the Helm people have decided to use a ‘Tiller’, which is an agent pod listening to the helm client. Supposedly it helps in managing the cluster, but the logician in me says it goes against the unidirectional flow of stateless architectures. I wanted to avoid running the agent, and luckily the ‘template’ helm plugin lets met do that. You can install it with helm plugin install https://github.com/technosophos/helm-template. Now we can just apply the entire application state (from the root folder) like this: helm template -r mostack . | kubectl apply -f -
Another downside to using helm is the fact that I can’t deploy subcharts in their own namespaces. But that option might come in the future.
Happy helming!