6 ms·

> You may be familiar with Operators from the concept’s introduction in 2016. An Operator is a method of packaging, deploying and managing a Kubernetes applicat

> You may be familiar with Operators from the concept’s introduction in 2016. An Operator is a method of packaging, deploying and managing a Kubernetes application. "Operators", as introduced in 2016, were just bespoke Go programs that communicated with Kubernetes internals in a pretty low-level way. You were writing special-case plugins for Kubernetes, but they didn't want to make it sound that way, because I guess that just doesn't sound hip or devopsy. This branding exercise worked out for CoreOS -- Red Hat just bought them. This whole space is massively infused with bullshit. It's because all of these companies want to make money selling you cloud stuff, because it's profitable to rent computers at 3-5x the TCO. Google especially is hungry to claw back the lead in the cloud space from Amazon, and it's not hard to conceive why Kubernetes doesn't seem to work without fuss anywhere except GKE, or to understand the massive marketing dollars that Google is pumping into this whole Kubernetes farce (and for the record, Google seems to consider HN an important platform for k8s PR; I've been censured after too many Googlers found my k8s-skeptical posts "tedious"). Anyway, I guess that's neither here nor there. Just annoyed at what is by now the totally conventional status quo of overhyped empty promises made by people who seem more like ignorant promoters and fanboys than serious engineers. This "Operator Framework" seems to be the same concept of Operators, just with additional library support for the plugins -- err, "Operators". It may be a good improvement, will have to research more.

gtirloni 8y ago
Your comment would be more useful if it shared any hands-on experience with Operators instead of the usual ranting. I think you've made your general opinion on containers and Kubernetes pretty clear already.
- cookiecaper 8y ago
  First, I don't think anyone notices or cares when I post specifically, so it's hard to really feel like my "general opinion" is well-known enough to not talk about it anymore. I'm no Joel Spolsky over here! Second, I feel like it's valid to point out that Operators are not really just a method of "packaging", in a post that tries to make it sound like Operators are just a small bit of YAML or metadata. You're writing real, non-trivial Go code that tells Kubernetes explicitly how to deploy and manage the lifecycles of specific types of applications. At least until now with the "Operator Framework", there wasn't really even anything that firmly defined an Operator as an Operator; it's just what some people called their Go code that manipulated k8s's object handling and lifecycle internals. But, if you insist, here's one operator I've worked with: https://github.com/coreos/prometheus-operator https://github.com/coreos/prometheus-operator . This is from CoreOS itself. Here's a patch I submitted about a year ago: https://github.com/coreos/prometheus-operator/pull/289 https://github.com/coreos/prometheus-operator/pull/289 . This required updating the way the software handled HTTP response codes in one of its "watcher" daemons (because all packaging methods need those, right?), and fixing the order of operations in the bootstrap scripts. Some more general info about this repo: $ du -sh prometheus-operator/.git 51M prometheus-operator/.git The repo size is 51M. $ git rev-list --count master 1716 There have been almost 2000 commits. $ cloc --vcs=git --exclude-dir vendor,example,contrib . 290 text files. 278 unique files. 121 files ignored. github.com/AlDanial/cloc v 1.74 T=0.82 s (295.4 files/s, 48117.2 lines/s) ------------------------------------------------------------------------------- Language files blank comment code ------------------------------------------------------------------------------- Go 50 1581 1392 20622 JSON 9 0 0 6276 YAML 132 260 885 4164 Markdown 30 792 0 2957 Bourne Shell 17 65 58 257 make 1 34 2 91 Python 1 10 5 40 TOML 1 11 20 30 Dockerfile 2 8 0 27 ------------------------------------------------------------------------------- SUM: 243 2761 2362 34464 ------------------------------------------------------------------------------- It appears there are over 20k lines of Go code after excluding vendor libraries and the example and contrib directories (arguably, contrib should've been included). I dunno, it just feels a little disingenuous, to me, to say that something that involves this much code is just a "packaging method" for a normal application. "Sure, just write an operator to package that up" like it's comparable to a package.json manifest or something. It's not! You need custom daemons that watch files to make sure that your k8s deployment stays in sync, and then you need to exert very meticulous and specific control over Kubernetes' behavior to make things work well. I think it's demonstrative that it takes north of 20k lines of Go code to package an application for deployment on Kubernetes. What do you think? ------------- EDIT: And one clarification: my opinion on containers as such is probably not well-known, since you're conflating it with my opinion on Kubernetes. I like containers conceptually (who wouldn't?) and I run several of them through LXC: NAME STATE AUTOSTART GROUPS IPV4 IPV6 UNPRIVILEGED axxxx-dev STOPPED 0 - - - false gentoo-encoder STOPPED 0 - - - false jeff-arch-base-lxc STOPPED 0 - - - false jeff-crypto RUNNING 0 - xxx.xxx.xx.xxx - false jeff-ffmpeg STOPPED 0 - - - false jeff-netsec STOPPED 0 - - - false jeff-ocr STOPPED 0 - - - false localtorrents-lxc RUNNING 0 - xxx.xxx.xx.xxx - false nim-dev STOPPED 0 - - - false plex-2018 RUNNING 1 - xxx.xxx.xxx.xxx - true unifi STOPPED 0 - - - true I believe this is the kind of thing people actually want. Highly efficient, thin "VMs" that are easy to manage and run as independent systems without requiring the resource commitment. There is a good place for Kubernetes in probably about 1% of deployments where it's used. Most other people are just trying to run something like LXC, but they're confused because everyone who is critical of k8s drops to -4 and gets HN's mods after them. :)
  - justinsaccount 8y ago
    > It appears there are over 20k lines of Go code after excluding vendor libraries and the example and contrib directories (arguably, contrib should've been included). close.. $ find -name '*.go'|grep -v ./vendor|xargs wc -l|sort -n|tail 507 ./test/e2e/alertmanager_test.go 540 ./pkg/client/monitoring/v1/types.go 562 ./pkg/alertmanager/operator.go 643 ./pkg/prometheus/statefulset.go 719 ./pkg/prometheus/promcfg.go 760 ./test/e2e/prometheus_test.go 835 ./pkg/client/monitoring/v1/zz_generated.deepcopy.go 1152 ./pkg/prometheus/operator.go 11410 ./pkg/client/monitoring/v1/openapi_generated.go 24526 total Deleting the auto generate api files and ignoring test/ gives ------------------------------------------------------------------------------- Language files blank comment code ------------------------------------------------------------------------------- JSON 9 0 0 6276 Go 31 1020 1065 6028 which is quite a bit off from 20k. > I dunno, it just feels a little disingenuous, to me, to say that something that involves this much code is just a "packaging method" for a normal application. What is disingenuous is to call the prometheus operator, that deploys an entire monitoring stack a "normal application" The operator sets up monitoring on all the nodes, runs all the server components including configuring grafana and setting up dashboards. Meanwhile, $ cloc puppet-prometheus-5.0.0 puppet-grafana-4.2.0 192 text files. 157 unique files. 48 files ignored. github.com/AlDanial/cloc v 1.76 T=1.09 s (131.9 files/s, 10534.0 lines/s) -------------------------------------------------------------------------------- Language files blank comment code -------------------------------------------------------------------------------- Ruby 43 425 108 2477 Puppet 32 132 1747 2280 Markdown 6 494 0 1201 ERB 20 105 0 925 YAML 35 0 200 459 JSON 4 0 0 329 Bourne Shell 2 28 130 198 Bourne Again Shell 2 28 56 176 -------------------------------------------------------------------------------- SUM: 144 1212 2241 8045 -------------------------------------------------------------------------------- Which is that large and doesn't do ANY of the things that the prometheus operator does. > I believe this is the kind of thing people actually want. Highly efficient, thin "VMs" that are easy to manage and run as independent systems without requiring the resource commitment. Or, people want to use k8s so they can run entire clusters of machines as a single consistent system and take advantage of things like rolling deployments and self healing applications.
TheIronYuppie 8y ago
I'm sorry you've been down voted. FWIW, I've asked you several times for feedback, but you've never responded. We'd love to be better! Aronchick (at) google.com Disclosure: I work at Google on Kubeflow
- moderation 8y ago
  cookiecaper covers his lack of response to k8s threads in his bio at https://news.ycombinator.com/user?id=cookiecaper https://news.ycombinator.com/user?id=cookiecaper. There is an email if you want to contact him directly.
- cookiecaper 8y ago
  Minor correction: I've replied a couple of times on HN and gotten no response. You once clarified that this is because you don't check HN often enough to reply before the deadline, which is fine. I do admit that you once sent me an email and that I didn't reply to it.
n0w 8y ago
In a world where we are trying to codify more and more of the operational side of our applications, I personally look at operators as a better alternative to the configuration management systems we use to configure generic operating systems today. We're moving away from an imperative configuration/operational model to a declarative one. While these operators target applicatings running in k8s I could imagine them being created to manage applications running elsewhere as well.
- cookiecaper 8y ago
  I agree with your conclusions that the world is becoming more declarative and that we need to converge on an agnostic system to make these declarations. However, you might've been trained incorrectly, perhaps as a joke, because you've assigned properties to the opposite solutions! Configuration management like Ansible or Salt are agnostic modules that can be plumbed under the hood to work with any platform: use Ansible's "copy" module and it can be implemented under the hood as cp, rsync, passenger pigeon, whatever. You can use the same playbook against any target, including targets that run on orchestration platforms like Kubernetes. Kubernetes Operators, on the other hand, are tightly linked to Kubernetes internals. They are not declarative; they require explicit instruction on how to manage your application's lifecycle, implemented in a statically-typed programming language. Indeed, if your system doesn't need to muck around in the Kubernetes internals, you don't create an "Operator", you just use the pre-baked object types like Service, Deployment, etc. So I agree in principle, but Kubernetes is the opposite of what you've expressed here.
atombender 8y ago
Operators are just a pattern, not a technology. I think they chose "operator" over the more traditional "controller" because the latter can be quite simple, whereas an operator is potentially a combination of several things, including CRDs, API extensions, and controllers. For example, an operator might start different controllers depending on what cloud it's deploying to. It's a useful distinction; if someone says "I'm using this operator for X", I instantly know what they mean. FWIW, I'm one of those who remember your name, simply because you pop up in every Kubernetes discussion with a predictably contrarian, long-winded opinion. I don't know what you're getting out of it. In this case, you're not wrong — but the curmudgeonly, somewhat tone deaf way that you go about it isn't very nice, which probably explains the downvotes.
- cookiecaper 8y ago
  >FWIW, I'm one of those who remember your name, simply because you pop up in every Kubernetes discussion with a predictably contrarian, long-winded opinion. There are several k8s conversations on HN each day. I skip most of them. > I don't know what you're getting out of it. I get a lot of useful feedback out of it. Most of the posts I make on HN are about trying something out and gauging the response, because I want to learn from it. Sometimes I get a complete correction, which is great, because I stop believing something that's wrong. More often I get minor shifts in my personal POV, perspective on what arguments are effective and which aren't, the pitfalls/commmon counterarguments to specific positions, good feedback on tone / interpretation, and lots of other valuable information. Also, once in a while I make a nice personal connection. I'll note for the record that response on k8s/containers is mixed. There are certainly a substantial number of threads where I'm at the bottom, but there are also threads where I make essentially the same arguments and score pretty well, along with a few supporting comments from people who say they don't get it either. HN's responses generally seem to be signaled by the tone of the thread and headline, and the preponderance of existing responses. If the groundwork is laid with a positive outlook, negative comments will usually have a hard time, and vice-versa. Also, my arguments are usually not purely repetitive, even if they have the same core message (because same core things remain relevant). I have never talked about Operators on HN before. They came up and they're a good example of how people pretend that Kubernetes is more production-ready than it is, by obfuscating things like "you have to write special programs to teach Kubernetes how to deploy and manage your applications because the YAML configuration interface they tout isn't good for complex cases", behind the much hipper "Build a Kubernetes Operator, then you'll be cloudified and Dockerized out the wazoo!!" I admit that I find a culture focused on this type of hype to be grating and immature, and as a sign of its inability to really bring substantial improvement to the table. I don't think I feel this way about anything new in general, I just think it's a reaction to a progressively-worsening engineering deficit in the "devops" field. I hope that I can learn whether this is right or wrong as time goes on. > In this case, you're not wrong — but the curmudgeonly, somewhat tone deaf way that you go about it isn't very nice, which probably explains the downvotes. Yeah, so this is a great example of why I continue to post about this. Most people would consider this subject matter very dry, but Kubernetes is something that people imbue with much more personal identification than is typical for infrastructure orchestration projects. Where are the CloudFoundry Diego disciples (paging jacques_chester ;) )? It's important to post and learn the pressure points, and if there is any argument or circumvention that is effective against that identity imprint. I'm still trying to learn, so I continue to post and draw feedback from the community. I appreciate your participation in teaching me thus far. :)

gtirloni 8y ago

Your comment would be more useful if it shared any hands-on experience with Operators instead of the usual ranting. I think you've made your general opinion on containers and Kubernetes pretty clear already.

cookiecaper 8y ago
First, I don't think anyone notices or cares when I post specifically, so it's hard to really feel like my "general opinion" is well-known enough to not talk about it anymore. I'm no Joel Spolsky over here! Second, I feel like it's valid to point out that Operators are not really just a method of "packaging", in a post that tries to make it sound like Operators are just a small bit of YAML or metadata. You're writing real, non-trivial Go code that tells Kubernetes explicitly how to deploy and manage the lifecycles of specific types of applications. At least until now with the "Operator Framework", there wasn't really even anything that firmly defined an Operator as an Operator; it's just what some people called their Go code that manipulated k8s's object handling and lifecycle internals. But, if you insist, here's one operator I've worked with: https://github.com/coreos/prometheus-operator https://github.com/coreos/prometheus-operator . This is from CoreOS itself. Here's a patch I submitted about a year ago: https://github.com/coreos/prometheus-operator/pull/289 https://github.com/coreos/prometheus-operator/pull/289 . This required updating the way the software handled HTTP response codes in one of its "watcher" daemons (because all packaging methods need those, right?), and fixing the order of operations in the bootstrap scripts. Some more general info about this repo: $ du -sh prometheus-operator/.git 51M prometheus-operator/.git The repo size is 51M. $ git rev-list --count master 1716 There have been almost 2000 commits. $ cloc --vcs=git --exclude-dir vendor,example,contrib . 290 text files. 278 unique files. 121 files ignored. github.com/AlDanial/cloc v 1.74 T=0.82 s (295.4 files/s, 48117.2 lines/s) ------------------------------------------------------------------------------- Language files blank comment code ------------------------------------------------------------------------------- Go 50 1581 1392 20622 JSON 9 0 0 6276 YAML 132 260 885 4164 Markdown 30 792 0 2957 Bourne Shell 17 65 58 257 make 1 34 2 91 Python 1 10 5 40 TOML 1 11 20 30 Dockerfile 2 8 0 27 ------------------------------------------------------------------------------- SUM: 243 2761 2362 34464 ------------------------------------------------------------------------------- It appears there are over 20k lines of Go code after excluding vendor libraries and the example and contrib directories (arguably, contrib should've been included). I dunno, it just feels a little disingenuous, to me, to say that something that involves this much code is just a "packaging method" for a normal application. "Sure, just write an operator to package that up" like it's comparable to a package.json manifest or something. It's not! You need custom daemons that watch files to make sure that your k8s deployment stays in sync, and then you need to exert very meticulous and specific control over Kubernetes' behavior to make things work well. I think it's demonstrative that it takes north of 20k lines of Go code to package an application for deployment on Kubernetes. What do you think? ------------- EDIT: And one clarification: my opinion on containers as such is probably not well-known, since you're conflating it with my opinion on Kubernetes. I like containers conceptually (who wouldn't?) and I run several of them through LXC: NAME STATE AUTOSTART GROUPS IPV4 IPV6 UNPRIVILEGED axxxx-dev STOPPED 0 - - - false gentoo-encoder STOPPED 0 - - - false jeff-arch-base-lxc STOPPED 0 - - - false jeff-crypto RUNNING 0 - xxx.xxx.xx.xxx - false jeff-ffmpeg STOPPED 0 - - - false jeff-netsec STOPPED 0 - - - false jeff-ocr STOPPED 0 - - - false localtorrents-lxc RUNNING 0 - xxx.xxx.xx.xxx - false nim-dev STOPPED 0 - - - false plex-2018 RUNNING 1 - xxx.xxx.xxx.xxx - true unifi STOPPED 0 - - - true I believe this is the kind of thing people actually want. Highly efficient, thin "VMs" that are easy to manage and run as independent systems without requiring the resource commitment. There is a good place for Kubernetes in probably about 1% of deployments where it's used. Most other people are just trying to run something like LXC, but they're confused because everyone who is critical of k8s drops to -4 and gets HN's mods after them. :)
- justinsaccount 8y ago
  > It appears there are over 20k lines of Go code after excluding vendor libraries and the example and contrib directories (arguably, contrib should've been included). close.. $ find -name '*.go'|grep -v ./vendor|xargs wc -l|sort -n|tail 507 ./test/e2e/alertmanager_test.go 540 ./pkg/client/monitoring/v1/types.go 562 ./pkg/alertmanager/operator.go 643 ./pkg/prometheus/statefulset.go 719 ./pkg/prometheus/promcfg.go 760 ./test/e2e/prometheus_test.go 835 ./pkg/client/monitoring/v1/zz_generated.deepcopy.go 1152 ./pkg/prometheus/operator.go 11410 ./pkg/client/monitoring/v1/openapi_generated.go 24526 total Deleting the auto generate api files and ignoring test/ gives ------------------------------------------------------------------------------- Language files blank comment code ------------------------------------------------------------------------------- JSON 9 0 0 6276 Go 31 1020 1065 6028 which is quite a bit off from 20k. > I dunno, it just feels a little disingenuous, to me, to say that something that involves this much code is just a "packaging method" for a normal application. What is disingenuous is to call the prometheus operator, that deploys an entire monitoring stack a "normal application" The operator sets up monitoring on all the nodes, runs all the server components including configuring grafana and setting up dashboards. Meanwhile, $ cloc puppet-prometheus-5.0.0 puppet-grafana-4.2.0 192 text files. 157 unique files. 48 files ignored. github.com/AlDanial/cloc v 1.76 T=1.09 s (131.9 files/s, 10534.0 lines/s) -------------------------------------------------------------------------------- Language files blank comment code -------------------------------------------------------------------------------- Ruby 43 425 108 2477 Puppet 32 132 1747 2280 Markdown 6 494 0 1201 ERB 20 105 0 925 YAML 35 0 200 459 JSON 4 0 0 329 Bourne Shell 2 28 130 198 Bourne Again Shell 2 28 56 176 -------------------------------------------------------------------------------- SUM: 144 1212 2241 8045 -------------------------------------------------------------------------------- Which is that large and doesn't do ANY of the things that the prometheus operator does. > I believe this is the kind of thing people actually want. Highly efficient, thin "VMs" that are easy to manage and run as independent systems without requiring the resource commitment. Or, people want to use k8s so they can run entire clusters of machines as a single consistent system and take advantage of things like rolling deployments and self healing applications.

TheIronYuppie 8y ago

I'm sorry you've been down voted. FWIW, I've asked you several times for feedback, but you've never responded. We'd love to be better! Aronchick (at) google.com Disclosure: I work at Google on Kubeflow

moderation 8y ago
cookiecaper covers his lack of response to k8s threads in his bio at https://news.ycombinator.com/user?id=cookiecaper https://news.ycombinator.com/user?id=cookiecaper. There is an email if you want to contact him directly.
cookiecaper 8y ago
Minor correction: I've replied a couple of times on HN and gotten no response. You once clarified that this is because you don't check HN often enough to reply before the deadline, which is fine. I do admit that you once sent me an email and that I didn't reply to it.

n0w 8y ago

In a world where we are trying to codify more and more of the operational side of our applications, I personally look at operators as a better alternative to the configuration management systems we use to configure generic operating systems today. We're moving away from an imperative configuration/operational model to a declarative one. While these operators target applicatings running in k8s I could imagine them being created to manage applications running elsewhere as well.

cookiecaper 8y ago
I agree with your conclusions that the world is becoming more declarative and that we need to converge on an agnostic system to make these declarations. However, you might've been trained incorrectly, perhaps as a joke, because you've assigned properties to the opposite solutions! Configuration management like Ansible or Salt are agnostic modules that can be plumbed under the hood to work with any platform: use Ansible's "copy" module and it can be implemented under the hood as cp, rsync, passenger pigeon, whatever. You can use the same playbook against any target, including targets that run on orchestration platforms like Kubernetes. Kubernetes Operators, on the other hand, are tightly linked to Kubernetes internals. They are not declarative; they require explicit instruction on how to manage your application's lifecycle, implemented in a statically-typed programming language. Indeed, if your system doesn't need to muck around in the Kubernetes internals, you don't create an "Operator", you just use the pre-baked object types like Service, Deployment, etc. So I agree in principle, but Kubernetes is the opposite of what you've expressed here.

atombender 8y ago

Operators are just a pattern, not a technology. I think they chose "operator" over the more traditional "controller" because the latter can be quite simple, whereas an operator is potentially a combination of several things, including CRDs, API extensions, and controllers. For example, an operator might start different controllers depending on what cloud it's deploying to. It's a useful distinction; if someone says "I'm using this operator for X", I instantly know what they mean. FWIW, I'm one of those who remember your name, simply because you pop up in every Kubernetes discussion with a predictably contrarian, long-winded opinion. I don't know what you're getting out of it. In this case, you're not wrong — but the curmudgeonly, somewhat tone deaf way that you go about it isn't very nice, which probably explains the downvotes.

cookiecaper 8y ago
>FWIW, I'm one of those who remember your name, simply because you pop up in every Kubernetes discussion with a predictably contrarian, long-winded opinion. There are several k8s conversations on HN each day. I skip most of them. > I don't know what you're getting out of it. I get a lot of useful feedback out of it. Most of the posts I make on HN are about trying something out and gauging the response, because I want to learn from it. Sometimes I get a complete correction, which is great, because I stop believing something that's wrong. More often I get minor shifts in my personal POV, perspective on what arguments are effective and which aren't, the pitfalls/commmon counterarguments to specific positions, good feedback on tone / interpretation, and lots of other valuable information. Also, once in a while I make a nice personal connection. I'll note for the record that response on k8s/containers is mixed. There are certainly a substantial number of threads where I'm at the bottom, but there are also threads where I make essentially the same arguments and score pretty well, along with a few supporting comments from people who say they don't get it either. HN's responses generally seem to be signaled by the tone of the thread and headline, and the preponderance of existing responses. If the groundwork is laid with a positive outlook, negative comments will usually have a hard time, and vice-versa. Also, my arguments are usually not purely repetitive, even if they have the same core message (because same core things remain relevant). I have never talked about Operators on HN before. They came up and they're a good example of how people pretend that Kubernetes is more production-ready than it is, by obfuscating things like "you have to write special programs to teach Kubernetes how to deploy and manage your applications because the YAML configuration interface they tout isn't good for complex cases", behind the much hipper "Build a Kubernetes Operator, then you'll be cloudified and Dockerized out the wazoo!!" I admit that I find a culture focused on this type of hype to be grating and immature, and as a sign of its inability to really bring substantial improvement to the table. I don't think I feel this way about anything new in general, I just think it's a reaction to a progressively-worsening engineering deficit in the "devops" field. I hope that I can learn whether this is right or wrong as time goes on. > In this case, you're not wrong — but the curmudgeonly, somewhat tone deaf way that you go about it isn't very nice, which probably explains the downvotes. Yeah, so this is a great example of why I continue to post about this. Most people would consider this subject matter very dry, but Kubernetes is something that people imbue with much more personal identification than is typical for infrastructure orchestration projects. Where are the CloudFoundry Diego disciples (paging jacques_chester ;) )? It's important to post and learn the pressure points, and if there is any argument or circumvention that is effective against that identity imprint. I'm still trying to learn, so I continue to post and draw feedback from the community. I appreciate your participation in teaching me thus far. :)