5 ms·

Your comment would be more useful if it shared any hands-on experience with Operators instead of the usual ranting. I think you've made your general opinion on

Your comment would be more useful if it shared any hands-on experience with Operators instead of the usual ranting. I think you've made your general opinion on containers and Kubernetes pretty clear already.

cookiecaper 9y ago
First, I don't think anyone notices or cares when I post specifically, so it's hard to really feel like my "general opinion" is well-known enough to not talk about it anymore. I'm no Joel Spolsky over here! Second, I feel like it's valid to point out that Operators are not really just a method of "packaging", in a post that tries to make it sound like Operators are just a small bit of YAML or metadata. You're writing real, non-trivial Go code that tells Kubernetes explicitly how to deploy and manage the lifecycles of specific types of applications. At least until now with the "Operator Framework", there wasn't really even anything that firmly defined an Operator as an Operator; it's just what some people called their Go code that manipulated k8s's object handling and lifecycle internals. But, if you insist, here's one operator I've worked with: https://github.com/coreos/prometheus-operator https://github.com/coreos/prometheus-operator . This is from CoreOS itself. Here's a patch I submitted about a year ago: https://github.com/coreos/prometheus-operator/pull/289 https://github.com/coreos/prometheus-operator/pull/289 . This required updating the way the software handled HTTP response codes in one of its "watcher" daemons (because all packaging methods need those, right?), and fixing the order of operations in the bootstrap scripts. Some more general info about this repo: $ du -sh prometheus-operator/.git 51M prometheus-operator/.git The repo size is 51M. $ git rev-list --count master 1716 There have been almost 2000 commits. $ cloc --vcs=git --exclude-dir vendor,example,contrib . 290 text files. 278 unique files. 121 files ignored. github.com/AlDanial/cloc v 1.74 T=0.82 s (295.4 files/s, 48117.2 lines/s) ------------------------------------------------------------------------------- Language files blank comment code ------------------------------------------------------------------------------- Go 50 1581 1392 20622 JSON 9 0 0 6276 YAML 132 260 885 4164 Markdown 30 792 0 2957 Bourne Shell 17 65 58 257 make 1 34 2 91 Python 1 10 5 40 TOML 1 11 20 30 Dockerfile 2 8 0 27 ------------------------------------------------------------------------------- SUM: 243 2761 2362 34464 ------------------------------------------------------------------------------- It appears there are over 20k lines of Go code after excluding vendor libraries and the example and contrib directories (arguably, contrib should've been included). I dunno, it just feels a little disingenuous, to me, to say that something that involves this much code is just a "packaging method" for a normal application. "Sure, just write an operator to package that up" like it's comparable to a package.json manifest or something. It's not! You need custom daemons that watch files to make sure that your k8s deployment stays in sync, and then you need to exert very meticulous and specific control over Kubernetes' behavior to make things work well. I think it's demonstrative that it takes north of 20k lines of Go code to package an application for deployment on Kubernetes. What do you think? ------------- EDIT: And one clarification: my opinion on containers as such is probably not well-known, since you're conflating it with my opinion on Kubernetes. I like containers conceptually (who wouldn't?) and I run several of them through LXC: NAME STATE AUTOSTART GROUPS IPV4 IPV6 UNPRIVILEGED axxxx-dev STOPPED 0 - - - false gentoo-encoder STOPPED 0 - - - false jeff-arch-base-lxc STOPPED 0 - - - false jeff-crypto RUNNING 0 - xxx.xxx.xx.xxx - false jeff-ffmpeg STOPPED 0 - - - false jeff-netsec STOPPED 0 - - - false jeff-ocr STOPPED 0 - - - false localtorrents-lxc RUNNING 0 - xxx.xxx.xx.xxx - false nim-dev STOPPED 0 - - - false plex-2018 RUNNING 1 - xxx.xxx.xxx.xxx - true unifi STOPPED 0 - - - true I believe this is the kind of thing people actually want. Highly efficient, thin "VMs" that are easy to manage and run as independent systems without requiring the resource commitment. There is a good place for Kubernetes in probably about 1% of deployments where it's used. Most other people are just trying to run something like LXC, but they're confused because everyone who is critical of k8s drops to -4 and gets HN's mods after them. :)
- justinsaccount 9y ago
  > It appears there are over 20k lines of Go code after excluding vendor libraries and the example and contrib directories (arguably, contrib should've been included). close.. $ find -name '*.go'|grep -v ./vendor|xargs wc -l|sort -n|tail 507 ./test/e2e/alertmanager_test.go 540 ./pkg/client/monitoring/v1/types.go 562 ./pkg/alertmanager/operator.go 643 ./pkg/prometheus/statefulset.go 719 ./pkg/prometheus/promcfg.go 760 ./test/e2e/prometheus_test.go 835 ./pkg/client/monitoring/v1/zz_generated.deepcopy.go 1152 ./pkg/prometheus/operator.go 11410 ./pkg/client/monitoring/v1/openapi_generated.go 24526 total Deleting the auto generate api files and ignoring test/ gives ------------------------------------------------------------------------------- Language files blank comment code ------------------------------------------------------------------------------- JSON 9 0 0 6276 Go 31 1020 1065 6028 which is quite a bit off from 20k. > I dunno, it just feels a little disingenuous, to me, to say that something that involves this much code is just a "packaging method" for a normal application. What is disingenuous is to call the prometheus operator, that deploys an entire monitoring stack a "normal application" The operator sets up monitoring on all the nodes, runs all the server components including configuring grafana and setting up dashboards. Meanwhile, $ cloc puppet-prometheus-5.0.0 puppet-grafana-4.2.0 192 text files. 157 unique files. 48 files ignored. github.com/AlDanial/cloc v 1.76 T=1.09 s (131.9 files/s, 10534.0 lines/s) -------------------------------------------------------------------------------- Language files blank comment code -------------------------------------------------------------------------------- Ruby 43 425 108 2477 Puppet 32 132 1747 2280 Markdown 6 494 0 1201 ERB 20 105 0 925 YAML 35 0 200 459 JSON 4 0 0 329 Bourne Shell 2 28 130 198 Bourne Again Shell 2 28 56 176 -------------------------------------------------------------------------------- SUM: 144 1212 2241 8045 -------------------------------------------------------------------------------- Which is that large and doesn't do ANY of the things that the prometheus operator does. > I believe this is the kind of thing people actually want. Highly efficient, thin "VMs" that are easy to manage and run as independent systems without requiring the resource commitment. Or, people want to use k8s so they can run entire clusters of machines as a single consistent system and take advantage of things like rolling deployments and self healing applications.

cookiecaper 9y ago

First, I don't think anyone notices or cares when I post specifically, so it's hard to really feel like my "general opinion" is well-known enough to not talk about it anymore. I'm no Joel Spolsky over here! Second, I feel like it's valid to point out that Operators are not really just a method of "packaging", in a post that tries to make it sound like Operators are just a small bit of YAML or metadata. You're writing real, non-trivial Go code that tells Kubernetes explicitly how to deploy and manage the lifecycles of specific types of applications. At least until now with the "Operator Framework", there wasn't really even anything that firmly defined an Operator as an Operator; it's just what some people called their Go code that manipulated k8s's object handling and lifecycle internals. But, if you insist, here's one operator I've worked with: https://github.com/coreos/prometheus-operator https://github.com/coreos/prometheus-operator . This is from CoreOS itself. Here's a patch I submitted about a year ago: https://github.com/coreos/prometheus-operator/pull/289 https://github.com/coreos/prometheus-operator/pull/289 . This required updating the way the software handled HTTP response codes in one of its "watcher" daemons (because all packaging methods need those, right?), and fixing the order of operations in the bootstrap scripts. Some more general info about this repo: $ du -sh prometheus-operator/.git 51M prometheus-operator/.git The repo size is 51M. $ git rev-list --count master 1716 There have been almost 2000 commits. $ cloc --vcs=git --exclude-dir vendor,example,contrib . 290 text files. 278 unique files. 121 files ignored. github.com/AlDanial/cloc v 1.74 T=0.82 s (295.4 files/s, 48117.2 lines/s) ------------------------------------------------------------------------------- Language files blank comment code ------------------------------------------------------------------------------- Go 50 1581 1392 20622 JSON 9 0 0 6276 YAML 132 260 885 4164 Markdown 30 792 0 2957 Bourne Shell 17 65 58 257 make 1 34 2 91 Python 1 10 5 40 TOML 1 11 20 30 Dockerfile 2 8 0 27 ------------------------------------------------------------------------------- SUM: 243 2761 2362 34464 ------------------------------------------------------------------------------- It appears there are over 20k lines of Go code after excluding vendor libraries and the example and contrib directories (arguably, contrib should've been included). I dunno, it just feels a little disingenuous, to me, to say that something that involves this much code is just a "packaging method" for a normal application. "Sure, just write an operator to package that up" like it's comparable to a package.json manifest or something. It's not! You need custom daemons that watch files to make sure that your k8s deployment stays in sync, and then you need to exert very meticulous and specific control over Kubernetes' behavior to make things work well. I think it's demonstrative that it takes north of 20k lines of Go code to package an application for deployment on Kubernetes. What do you think? ------------- EDIT: And one clarification: my opinion on containers as such is probably not well-known, since you're conflating it with my opinion on Kubernetes. I like containers conceptually (who wouldn't?) and I run several of them through LXC: NAME STATE AUTOSTART GROUPS IPV4 IPV6 UNPRIVILEGED axxxx-dev STOPPED 0 - - - false gentoo-encoder STOPPED 0 - - - false jeff-arch-base-lxc STOPPED 0 - - - false jeff-crypto RUNNING 0 - xxx.xxx.xx.xxx - false jeff-ffmpeg STOPPED 0 - - - false jeff-netsec STOPPED 0 - - - false jeff-ocr STOPPED 0 - - - false localtorrents-lxc RUNNING 0 - xxx.xxx.xx.xxx - false nim-dev STOPPED 0 - - - false plex-2018 RUNNING 1 - xxx.xxx.xxx.xxx - true unifi STOPPED 0 - - - true I believe this is the kind of thing people actually want. Highly efficient, thin "VMs" that are easy to manage and run as independent systems without requiring the resource commitment. There is a good place for Kubernetes in probably about 1% of deployments where it's used. Most other people are just trying to run something like LXC, but they're confused because everyone who is critical of k8s drops to -4 and gets HN's mods after them. :)

justinsaccount 9y ago
> It appears there are over 20k lines of Go code after excluding vendor libraries and the example and contrib directories (arguably, contrib should've been included). close.. $ find -name '*.go'|grep -v ./vendor|xargs wc -l|sort -n|tail 507 ./test/e2e/alertmanager_test.go 540 ./pkg/client/monitoring/v1/types.go 562 ./pkg/alertmanager/operator.go 643 ./pkg/prometheus/statefulset.go 719 ./pkg/prometheus/promcfg.go 760 ./test/e2e/prometheus_test.go 835 ./pkg/client/monitoring/v1/zz_generated.deepcopy.go 1152 ./pkg/prometheus/operator.go 11410 ./pkg/client/monitoring/v1/openapi_generated.go 24526 total Deleting the auto generate api files and ignoring test/ gives ------------------------------------------------------------------------------- Language files blank comment code ------------------------------------------------------------------------------- JSON 9 0 0 6276 Go 31 1020 1065 6028 which is quite a bit off from 20k. > I dunno, it just feels a little disingenuous, to me, to say that something that involves this much code is just a "packaging method" for a normal application. What is disingenuous is to call the prometheus operator, that deploys an entire monitoring stack a "normal application" The operator sets up monitoring on all the nodes, runs all the server components including configuring grafana and setting up dashboards. Meanwhile, $ cloc puppet-prometheus-5.0.0 puppet-grafana-4.2.0 192 text files. 157 unique files. 48 files ignored. github.com/AlDanial/cloc v 1.76 T=1.09 s (131.9 files/s, 10534.0 lines/s) -------------------------------------------------------------------------------- Language files blank comment code -------------------------------------------------------------------------------- Ruby 43 425 108 2477 Puppet 32 132 1747 2280 Markdown 6 494 0 1201 ERB 20 105 0 925 YAML 35 0 200 459 JSON 4 0 0 329 Bourne Shell 2 28 130 198 Bourne Again Shell 2 28 56 176 -------------------------------------------------------------------------------- SUM: 144 1212 2241 8045 -------------------------------------------------------------------------------- Which is that large and doesn't do ANY of the things that the prometheus operator does. > I believe this is the kind of thing people actually want. Highly efficient, thin "VMs" that are easy to manage and run as independent systems without requiring the resource commitment. Or, people want to use k8s so they can run entire clusters of machines as a single consistent system and take advantage of things like rolling deployments and self healing applications.