3 ms·
No problem. I wrote Matsuri as a way to manage this. https://github.com/matsuri-rb/matsuri https://github.com/matsuri-rb/matsuri This was written with K8S 1.1
by hosh 10y ago
No problem.
I wrote Matsuri as a way to manage this. https://github.com/matsuri-rb/matsuri https://github.com/matsuri-rb/matsuri This was written with K8S 1.1 in mind, and later somewhat updated to 1.4.
It was primarily intended to solve:
(1) Consolidate interfaces for working with dev and production (or other environments such as staging, demo, etc.)
(2) It implements this by being a manifest generator and feeding the manifest into `kubectl`.
This was designed and written before ConfigSet, so some of what Matsuri solved is now solvable with ConfigSet.
Matsuri is written in Ruby, and so it takes advantages of class inheritance and method override to achieve what it does. The class definitions used generate the manifest is broken down into composable chunks, which can be overridden at any point.
In this way, I can have a base template for say, a web pod for dev, staging, and production. In dev, I would then method-override what is returned for volume mounts and volumes. In that way, I can specify source mounting and SSH agent sock mounting for dev ... but not for staging or production. In another example, I can use method override to augment the environment variables passed into the container in order to capture the uid/gid so that the container can reconfigure itself to use those.
There are helper methods to generate things like, correct format of ports or volumes. None of those are necessary if you already know how to format it.
Likewise, since a manifest is just a class definition, replication controller, replicasets, and deployments all reference a pod and use those to create the pod template spec.
I then have an extra resource called an "app" that does not map to K8S. They define a bundle of K8S resources or other apps that can be converged together. That allows someone to quickly setup a dev environment with all the moving pieces. Those also include hooks for you to do things like, define the command for shelling (with correct uid/gid) or console (which depends; redis console is invoked differently than a rails console). This allows a consolidation ... for example, to log into my dev rails console, I would do something like:
bin/dev console myapp
And to do that on production:
bin/production console myapp
(Powerful, but I suspect there will be some asks around security here).
I had flirted with the idea of allowing plugins or packages ... and then Helm was announced. I figured more people will use that than Matsuri. But one possibility might be to get Matsuri to work with Helm.
There are a lot of rough spots with Matsuri. The command-line interface could be done a lot better (right now, it assumes you will always be in the root of your platform repo). There are no documentation or examples for Matsuri. It does not implement all of the resources. It also helps a lot if you know Ruby.
Infrastructure is not my full-time role on my team. Being a small team and early stage, we have had to wear multiple hats. I had not had the energy or time to document and promote this. If you want to chat about this further, feel free to email me at hosheng.hsiao at gmail.
- hosh 10y agoOn the stack-specific modules -- we're using Rails. The gems are installed on a host mount. None of us are using an IDE that needs to reference where it is, but if we did, we could point it to that path. One of the headaches we run into comes from using Alpine as a baseimage. They are compiled against muslc instead of libc. You always have to install gems from inside the container -- but we can do that with host mounting and then executing the install from inside the container. That hasn't been a problem until we tried to install google-api gem and found out grpc doesn't play well with muslc.
- fredsir 10y agoThanks for the write-up!