5 ms·

What would be a good example where an operator would make sense?

jhoelzel 4y ago
- creating databases for your app on the fly. - scaling up and down applications because of time instead of demand. or based on non metric based actions - Extending kubernetes to understand your workload - Automating configuration and management of complex applications - Managing legacy applications that cannot be easily containerized or migrated to the cloud. if you love k8s youll love operators the list is endless!
- dilyevsky 4y ago
  With respect, being “in love” with a technology is not a good way to go about it - it leads to tunnel vision
  - jhoelzel 4y ago
    i know what you mean, but i am doing this for more than 20 years now. From bare-metal over openstack to serverless, i have pretty much provisioned all of them. Kubernetes is more like a way of doing things than a technology. Basically APIs all the way down. and thus the operators and controllers do deserve love. Im not saying that you need an operator to change the dipers of a baby, but as far as a stack goes, k8s is the best i have ever worked with.
debarshri 4y ago
There is whole list of public operators that you can find in operator hub [1]. [1] https://operatorhub.io/ https://operatorhub.io/
debarshri 4y ago
A good example from my perspective is when you are delivering an application as 3rd party vendor and you wish to automate lot of operational stuff like backup, scaling based on events, automating stuff based on cluster events. It starts becoming very valuable. I am sure there are many more use cases for.
- jrockway 4y ago
  I would not write an operator to do any of these things. To me an "operator" strongly implies the existence of a CRD and the need to manage it. So for autoscaling, HPA/VPA are built into k8s. Backups should be an application-level feature; when the "take a backup" RPC or time arrives, take a backup and dump it in configured object storage. Automating stuff based on cluster events also doesn't require an operator; call client.V1().Whatever().Watch and do what you need to do. The only moderately justifiable operator I've ever seen is cert-manager. Even then, one wonders what it would be like if it just updated a Secret every 3 months based on a hard-coded config passed to it, and skipped the CRDs.
remram 4y ago
An operator operates something, e.g. it actively makes changes. If you want to deploy an application, a Helm Chart is the correct way. It will allow you to have deterministic deployment, that you can duplicate multiple times in your cluster, and you can dry-run it and see the generated manifests. An operator is needed when you can't just deploy and forget about it. An example is the Prometheus operator, which will track annotations created by users to configure the scraping configuration of your Prometheus instances. Another example is cert-manager, which gets certificates into secrets based on Certificate and Ingress objects, renews them automatically before expiry, and does that by creating ingresses picked up by your ingress controller. The advantage of an operator is that it will react to stuff happening in the cluster. The drawback is that it reacts to stuff happening, potentially doing unexpected things because changes happen at any time and you can't dry-run them. Another drawback is that they are usually global, so you can't run multiple versions at the same time for different namespaces (mainly because custom resource definitions are global). Unfortunately many people think packaging an application = creating an operator, and that operator does nothing a chart couldn't do.
- mdaniel 4y ago
  > that operator does nothing a chart couldn't do. Or is can be actively harmful when they don't do any error checking whatsoever, causing it to be less accurate that `helm template` would be. Related, it's also one more thing to monitor because it can decide to start vomiting errors for whatever random reason
- stasmo 4y ago
  The CockRoach DB example in the article is a perfect example of an unnecessary CRD. Acquiring certificates within an Kubernetes cluster is a common requirement for lots of applications and there are lots of solutions out there. Is it really necessary to spend time writing your own operator? Now you have a second helm chart and an operator to maintain. Now you have to explain to people which chart to use. You could get rid of the non-operator chart but now I have operators within the cluster acquiring certificates in 5 or 6 different ways. Do I have to configure the credentials for 6 operators so they can make Route53 DNS challenge records? Edit: maybe we could shift left and ask the app developers to add certificate acquisition directly into the app source.
EdwardDiego 4y ago
I worked on an operator that manages Kafka in K8s. If you want to upgrade the brokers in a Kafka cluster, you generally do a rolling upgrade to ensure availability. The operator will do this for you, you just update the version of the broker in the CR spec, it notices, and then applies the change. Likewise, some configuration options can be applied at runtime, some need the broker to be restarted to be applied, the operator knows which are which, and will again manage the process of a rolling restart if needed to apply the change. You can also define topics and users as custom resources, so have a nice Gitops approach to declaring resources.
- cagmz 4y ago
  Is there an open source version of this?
  - gunnarmorling 4y ago
    Not the OP, but Strimzi (strimzi.io/) is an open-source operator for running Kafka on Kubernetes. Disclaimer: In my past job, I've worked at Red Hat, who are sponsoring Strimzi
  - EdwardDiego 4y ago
    Strimzi :)
spenczar5 4y ago
Operators make sense when you need to automatically modify resources in response to changes in the cluster's state. An example that has come up for me is an operator for a Kafka Schema Registry. This is a service that needs some credentials in a somewhat obscure format so it can communicate very directly with a Kafka broker. If the broker's certificates (or CA) are modified, then the Schema Registry needs to have new credentials generated, and needs to be restarted. But the registry shouldn't (obviously) have direct access to the broker's certificates. Instead, there's a more-privileged subsystem which orchestrates that dance; that's the operator.
sleepybrett 4y ago
kubernetes itself is a collection of controllers/operators. It takes manifests like pods and uses that information to create the workload in your container runtime on a node with the resources it needs.

- creating databases for your app on the fly. - scaling up and down applications because of time instead of demand. or based on non metric based actions - Extending kubernetes to understand your workload - Automating configuration and management of complex applications - Managing legacy applications that cannot be easily containerized or migrated to the cloud. if you love k8s youll love operators the list is endless!

dilyevsky 4y ago
With respect, being “in love” with a technology is not a good way to go about it - it leads to tunnel vision
- jhoelzel 4y ago
  i know what you mean, but i am doing this for more than 20 years now. From bare-metal over openstack to serverless, i have pretty much provisioned all of them. Kubernetes is more like a way of doing things than a technology. Basically APIs all the way down. and thus the operators and controllers do deserve love. Im not saying that you need an operator to change the dipers of a baby, but as far as a stack goes, k8s is the best i have ever worked with.

debarshri 4y ago

There is whole list of public operators that you can find in operator hub [1]. [1] https://operatorhub.io/ https://operatorhub.io/

debarshri 4y ago

A good example from my perspective is when you are delivering an application as 3rd party vendor and you wish to automate lot of operational stuff like backup, scaling based on events, automating stuff based on cluster events. It starts becoming very valuable. I am sure there are many more use cases for.

jrockway 4y ago
I would not write an operator to do any of these things. To me an "operator" strongly implies the existence of a CRD and the need to manage it. So for autoscaling, HPA/VPA are built into k8s. Backups should be an application-level feature; when the "take a backup" RPC or time arrives, take a backup and dump it in configured object storage. Automating stuff based on cluster events also doesn't require an operator; call client.V1().Whatever().Watch and do what you need to do. The only moderately justifiable operator I've ever seen is cert-manager. Even then, one wonders what it would be like if it just updated a Secret every 3 months based on a hard-coded config passed to it, and skipped the CRDs.

remram 4y ago

An operator operates something, e.g. it actively makes changes. If you want to deploy an application, a Helm Chart is the correct way. It will allow you to have deterministic deployment, that you can duplicate multiple times in your cluster, and you can dry-run it and see the generated manifests. An operator is needed when you can't just deploy and forget about it. An example is the Prometheus operator, which will track annotations created by users to configure the scraping configuration of your Prometheus instances. Another example is cert-manager, which gets certificates into secrets based on Certificate and Ingress objects, renews them automatically before expiry, and does that by creating ingresses picked up by your ingress controller. The advantage of an operator is that it will react to stuff happening in the cluster. The drawback is that it reacts to stuff happening, potentially doing unexpected things because changes happen at any time and you can't dry-run them. Another drawback is that they are usually global, so you can't run multiple versions at the same time for different namespaces (mainly because custom resource definitions are global). Unfortunately many people think packaging an application = creating an operator, and that operator does nothing a chart couldn't do.

mdaniel 4y ago
> that operator does nothing a chart couldn't do. Or is can be actively harmful when they don't do any error checking whatsoever, causing it to be less accurate that `helm template` would be. Related, it's also one more thing to monitor because it can decide to start vomiting errors for whatever random reason
stasmo 4y ago
The CockRoach DB example in the article is a perfect example of an unnecessary CRD. Acquiring certificates within an Kubernetes cluster is a common requirement for lots of applications and there are lots of solutions out there. Is it really necessary to spend time writing your own operator? Now you have a second helm chart and an operator to maintain. Now you have to explain to people which chart to use. You could get rid of the non-operator chart but now I have operators within the cluster acquiring certificates in 5 or 6 different ways. Do I have to configure the credentials for 6 operators so they can make Route53 DNS challenge records? Edit: maybe we could shift left and ask the app developers to add certificate acquisition directly into the app source.

EdwardDiego 4y ago

I worked on an operator that manages Kafka in K8s. If you want to upgrade the brokers in a Kafka cluster, you generally do a rolling upgrade to ensure availability. The operator will do this for you, you just update the version of the broker in the CR spec, it notices, and then applies the change. Likewise, some configuration options can be applied at runtime, some need the broker to be restarted to be applied, the operator knows which are which, and will again manage the process of a rolling restart if needed to apply the change. You can also define topics and users as custom resources, so have a nice Gitops approach to declaring resources.

cagmz 4y ago
Is there an open source version of this?
- gunnarmorling 4y ago
  Not the OP, but Strimzi (strimzi.io/) is an open-source operator for running Kafka on Kubernetes. Disclaimer: In my past job, I've worked at Red Hat, who are sponsoring Strimzi
- EdwardDiego 4y ago
  Strimzi :)

spenczar5 4y ago

Operators make sense when you need to automatically modify resources in response to changes in the cluster's state. An example that has come up for me is an operator for a Kafka Schema Registry. This is a service that needs some credentials in a somewhat obscure format so it can communicate very directly with a Kafka broker. If the broker's certificates (or CA) are modified, then the Schema Registry needs to have new credentials generated, and needs to be restarted. But the registry shouldn't (obviously) have direct access to the broker's certificates. Instead, there's a more-privileged subsystem which orchestrates that dance; that's the operator.

sleepybrett 4y ago

kubernetes itself is a collection of controllers/operators. It takes manifests like pods and uses that information to create the workload in your container runtime on a node with the resources it needs.