4 ms·

I am seeing K8s ecosystem repeating Borg's own history of configuration churns. It's just there seems so many configuration tools for K8s now, which seems a muc

I am seeing K8s ecosystem repeating Borg's own history of configuration churns. It's just there seems so many configuration tools for K8s now, which seems a much worse situation than Borg (started with only one configuration tool).

ithkuil 8y ago
Here is one of those: https://github.com/ksonnet/kubecfg https://github.com/ksonnet/kubecfg
- atombender 8y ago
  For those who are as confused as me why Kubecfg exists in the Ksonnet org on Github, when they also have Ksonnet itself, I found this explanation: https://blog.heptio.com/the-next-chapter-for-ksonnet-1dcbbad30cb https://blog.heptio.com/the-next-chapter-for-ksonnet-1dcbbad.... Short summary: They're different but somewhat complementary tools. Kubecfg was made by Bitnami, Ksonnet by Heptio. Kubecfg uses Jsonnet and is less opinioned than Ksonnet, but (as far as I can see) can also use Ksonnet.
  - guslees 8y ago
    The fork is my fault. I agree it's confusing. I wrote (and still maintain) kubecfg. Heptio joined the project and started adding lots of great stuff. Eventually it was clear they wanted to take it in a direction that was different to the original borgcfg-like vision. I suggested we split that new functionality into a different tool so we could keep exploring both directions without trying to mash both into the same cli flags. Hence ksonnet/kubecfg and ksonnet/ksonnet. They both use jsonnet internally, generate the same k8s resources in the end, and have that common code heritage, so have many similarities. ksonnet/ksonnet has a bunch of extra ("rails-like") tooling to hold your hand while you generate jsonnet, and assumes it uses the k.libsonnet library (https://github.com/ksonnet/ksonnet-lib https://github.com/ksonnet/ksonnet-lib). ksonnet/kubecfg is much dumber and really just conceptually `jsonnet | kubectl apply`. In particular kubecfg avoids having any opinion about which jsonnet libraries you use. We use it extensively with https://github.com/bitnami-labs/kube-libsonnet https://github.com/bitnami-labs/kube-libsonnet, but you can also use it with https://github.com/ksonnet/ksonnet-lib https://github.com/ksonnet/ksonnet-lib or anything else that is valid jsonnet.
    - atombender 8y ago
      Thanks for the explanation. Is Kubecfg like Helm in that it will apply a "destructive" diff (i.e. delete resources belonging to a deploy that are not part of the new deploy)? I took a look at Ksonnet a while back, and it seemed to have way too many bells and whistles. In particular, I did not particularly like the sheer amount of files, in something like three separate folders, that you have to maintain for each project. I love the idea of declaring a schema that you're generating data for, but Ksonnet seems to have adopted a fairly complex structure to accomplish this. We use Helm right now, though purely as a templating engine and destructive deployer of charts that we store in the same repo as the app itself. The package management isn't useful for our own apps, and really only gets in the way. We're looking at alternatives that don't use text-based templating and come with slightly higher-level concepts of releases. Right now, we wrap Helm with our own little CLI so that we can, for example, automatically build all the template inputs from a set of (defaults, environment-specific stuff, local overrides). Our tool also presents a diff if you want, and records things like git revision info and deployer name in the annotations. Thus, when you ask our tool to deploy something, it can look at what the currently deployed revision is, then do a "git log" to find what's changed, display that history with nice colours on the terminal, etc. before deploying. All things that, in my opinion, a Kubernetes deploy tool should have.
      - ithkuil 8y ago
        (EDIT: the nick rang a bell: thanks for ktail!) kubecfg has the --gc-tag flag, which you explicitly pass so that it can know which of the existing resources in the k8s API server used to be created by this set of kubecfg maintained config (this "system", this "application", it's up to you to decide the grain of your modelling) and thus be able to delete the resources are are no longer output by the evaluation of the current configs. This catches the cases where you delete resources but also when you rename resources, or when you "move" them between namespaces. It's implemented without any in-cluster component (no "tiller") by simply setting the GC tag as a label on the resource. Kubecfg also implements diff between local config a d deployed state. As for the amount of files: it's up to you. Kubecfg is not opinionated on how you lay out your config. Unlike helm, it doesn't require you to know in advance which values you want to parameterize (and thus out in a values.yaml) since it's trivial to override any value with jsonnet. The k8s API and its data model is the only thing you have to learn. There are some helpers available to help you structure larger configs, e.g. https://github.com/bitnami-labs/kube-libsonnet/blob/master/kube.libsonnet https://github.com/bitnami-labs/kube-libsonnet/blob/master/k... . While this lib also provides some 'macros' to help you build common entities like services and deployments, IMHO the main benefit is the mapping of "foo_: {key: val, ...}" to "foo: [val, ...]". The former is much more friendlier when you have to override values with jsonnet rather than having to depend on array ordering.

ithkuil 8y ago

Here is one of those: https://github.com/ksonnet/kubecfg https://github.com/ksonnet/kubecfg

atombender 8y ago
For those who are as confused as me why Kubecfg exists in the Ksonnet org on Github, when they also have Ksonnet itself, I found this explanation: https://blog.heptio.com/the-next-chapter-for-ksonnet-1dcbbad30cb https://blog.heptio.com/the-next-chapter-for-ksonnet-1dcbbad.... Short summary: They're different but somewhat complementary tools. Kubecfg was made by Bitnami, Ksonnet by Heptio. Kubecfg uses Jsonnet and is less opinioned than Ksonnet, but (as far as I can see) can also use Ksonnet.
- guslees 8y ago
  The fork is my fault. I agree it's confusing. I wrote (and still maintain) kubecfg. Heptio joined the project and started adding lots of great stuff. Eventually it was clear they wanted to take it in a direction that was different to the original borgcfg-like vision. I suggested we split that new functionality into a different tool so we could keep exploring both directions without trying to mash both into the same cli flags. Hence ksonnet/kubecfg and ksonnet/ksonnet. They both use jsonnet internally, generate the same k8s resources in the end, and have that common code heritage, so have many similarities. ksonnet/ksonnet has a bunch of extra ("rails-like") tooling to hold your hand while you generate jsonnet, and assumes it uses the k.libsonnet library (https://github.com/ksonnet/ksonnet-lib https://github.com/ksonnet/ksonnet-lib). ksonnet/kubecfg is much dumber and really just conceptually `jsonnet | kubectl apply`. In particular kubecfg avoids having any opinion about which jsonnet libraries you use. We use it extensively with https://github.com/bitnami-labs/kube-libsonnet https://github.com/bitnami-labs/kube-libsonnet, but you can also use it with https://github.com/ksonnet/ksonnet-lib https://github.com/ksonnet/ksonnet-lib or anything else that is valid jsonnet.
  - atombender 8y ago
    Thanks for the explanation. Is Kubecfg like Helm in that it will apply a "destructive" diff (i.e. delete resources belonging to a deploy that are not part of the new deploy)? I took a look at Ksonnet a while back, and it seemed to have way too many bells and whistles. In particular, I did not particularly like the sheer amount of files, in something like three separate folders, that you have to maintain for each project. I love the idea of declaring a schema that you're generating data for, but Ksonnet seems to have adopted a fairly complex structure to accomplish this. We use Helm right now, though purely as a templating engine and destructive deployer of charts that we store in the same repo as the app itself. The package management isn't useful for our own apps, and really only gets in the way. We're looking at alternatives that don't use text-based templating and come with slightly higher-level concepts of releases. Right now, we wrap Helm with our own little CLI so that we can, for example, automatically build all the template inputs from a set of (defaults, environment-specific stuff, local overrides). Our tool also presents a diff if you want, and records things like git revision info and deployer name in the annotations. Thus, when you ask our tool to deploy something, it can look at what the currently deployed revision is, then do a "git log" to find what's changed, display that history with nice colours on the terminal, etc. before deploying. All things that, in my opinion, a Kubernetes deploy tool should have.
    - ithkuil 8y ago
      (EDIT: the nick rang a bell: thanks for ktail!) kubecfg has the --gc-tag flag, which you explicitly pass so that it can know which of the existing resources in the k8s API server used to be created by this set of kubecfg maintained config (this "system", this "application", it's up to you to decide the grain of your modelling) and thus be able to delete the resources are are no longer output by the evaluation of the current configs. This catches the cases where you delete resources but also when you rename resources, or when you "move" them between namespaces. It's implemented without any in-cluster component (no "tiller") by simply setting the GC tag as a label on the resource. Kubecfg also implements diff between local config a d deployed state. As for the amount of files: it's up to you. Kubecfg is not opinionated on how you lay out your config. Unlike helm, it doesn't require you to know in advance which values you want to parameterize (and thus out in a values.yaml) since it's trivial to override any value with jsonnet. The k8s API and its data model is the only thing you have to learn. There are some helpers available to help you structure larger configs, e.g. https://github.com/bitnami-labs/kube-libsonnet/blob/master/kube.libsonnet https://github.com/bitnami-labs/kube-libsonnet/blob/master/k... . While this lib also provides some 'macros' to help you build common entities like services and deployments, IMHO the main benefit is the mapping of "foo_: {key: val, ...}" to "foo: [val, ...]". The former is much more friendlier when you have to override values with jsonnet rather than having to depend on array ordering.