4 ms·
I'll start by saying that I think knowledge for knowing layers underneath the application is fading in some circles, and that makes me sad. Having been a front
by llama052 2y ago
I'll start by saying that I think knowledge for knowing layers underneath the application is fading in some circles, and that makes me sad.
Having been a frontend guy some 10+ years ago, into a network engineer, then infrastructure engineering and now SRE. The amount of people on both sides of the developer circle and operations circle that do not want to understand what's going on is mind boggling.
I was around when VMs were hot, when treating them as long living pets was just toil that operations dealt with. The collection of shell scripts to make that toil go away was nice. Then puppet, ansible and the like.
Now we are in the golden ages of Kubernetes and orchestration platforms. We have a set of standards for how things can be operated. The terms are obfuscated sure, but the core concepts are still the same underneath the abstraction.
I agree that platform engineering is a good place to be, and honestly it needs to be understood more by all parties including executives. They were bought and sold cloud on the idea that it's all managed, but that cannot be further from the truth, wrinkles will show as scale grows and your use cases progress in any environment, at home or in the cloud.
Unfortunately good platform teams often aren't seen. A good platform just works, metrics just exist, logs just work, tracing just works out of the box. Things don't often go down. It's really only visible when things fail. If you do a great job implementing a self service platform you're often met with executives wondering why you're there because the cloud does it all!
Applications are highly visible to all, but so are the layers underneath and they all work together if done correctly, I wish that was more understood.
For context, I'm currently running multiple environments of Kubernetes, on premise and in cloud. Our team prides itself on using open source solutions utilizing the operator model. Prometheus, Thanos, Loki, Tempo, Istio, Cert-Manager, Strimzi Kafka, Flink operator, Otel collector etc. We do billions of requests a month and TBs of bandwidth with microservices. Have at a minimum 4 9's of uptime, and our cost footprint is extremely small. This comes from a 4 man platform team that also handles on call for all applications, security, cloud budget, and operations. It's not impossible.
I guess I can't emphasize enough that understanding what the orchestration systems, the tooling and the stack are trying to do makes everything easier. As a developer you can understand your constraints and limitations. You can build off of known barriers. As an operations or platform engineer you can build things that don't require constant babysitting or toil.. you can save hundreds of thousands of dollars not offloading your observability to data dog or the like, you can make an impact. The technology is already here.
- hitsurume 2y agoI'm interested in knowing more about how you guys implemented the operator model and decided on those tools. Was there a book or anything that was helpful in all of this?
- llama052 2y agoNothing specific I've read that helped us with this, our goal was finding the easiest path to entry to get the tooling we desire in our platform. I've debated writing it up and posting it somewhere, maybe I should. There's so many ways of doing things now that it's quite overwhelming sometimes.
- hitsurume 2y agoYea exactly, thats why I asked haha.
- wyclif 2y agoI came here to ask this question as well!