5 ms·

I thought maybe I was just getting old and crusty, but this really does capture my primary complaint with K8s - everything that isn't standard, is so special an

I thought maybe I was just getting old and crusty, but this really does capture my primary complaint with K8s - everything that isn't standard, is so special and unrelated to the underlying technology that you need to relearn how K8's interacts with every layer of abstraction just to have half a chance of it working.

MuffinFlavored 4y ago
> everything that isn't standard Could you give an example? I thought k8s and its YAML/practices was the standard?
- jahsome 4y ago
  I've always felt the standards are more about organization and common practices, whereas the implementation is still the wild west. Unfortunately in my experience, abstraction, even if structured, doesn't solve the main issue: it turns out the engineering operation of infra is still pretty hard. At least you know where to look to find the chaos though.
  - MuffinFlavored 4y ago
    > whereas the implementation is still the wild west. in my own experience, i would've thought "ok, let me aptinstall k8s and then apply some yaml to it so i can avoid going to the cloud for a massive k8s bill for almost no reason" standing up kubeadm on a single master + worker node setup on a VM (like a droplet or EC2) is basically like, extremely non-standard + frowned upon then, does anybody actually write pure k8s yaml? or do they write jsonnet/kustomize/helm charts? do they apply with kubectl or the helm CLI or the argocd CLI?
    - sophiabits 4y ago
      > then, does anybody actually write pure k8s yaml? You can think of plain k8s yaml as a sort of machine code for k8s. You _can_ write yaml by hand, but it’s not generally how things are done outside of initially learning how k8s works. The tools you’ve mentioned all make working with k8s resources a lot more pleasant. Something simple like parameterizing the name of a secret instead of hardcoding it in N different yaml files saves a lot of time if you ever need to refactor, and being able to provide those values via a “standard” CLI tool makes automated deployments a lot easier compared to hacking together some yq commands. helm in particular tracks the history of any charts you’ve installed, and has a very simple “helm rollback” command which can get you back to the last working version of your application if things end up going wrong. The exact tools used by each team will differ; you could use any of them. But no one in my experience is writing plain yaml manifests and kubectl applying them in production contexts.
- Topgamer7 4y ago
  He means part of the k8s standard. Once you get to the non-standard, cloud specific configuration, things go haywire. For example try to configure a udp connection in k8s on aws.
  - MuffinFlavored 4y ago
    oh, like azure k8s vs aws k8s vs gcp k8s? each has their own way to like... attach a volume for block storage or something? i thought that was all abstracted away/standardized? > Once you get to the non-standard, cloud specific configuration what's a common use case for when this is hit?
- baq 4y ago
  YAML practices… YAML is not a language, it’s a syntax tree. It pretends to be configuration and what it really becomes is some kind of a gimped template engine which people put layers and layers of code generators on top of to manage the complexity. XML was better and I’m willing to die on this hill.
  - Alifatisk 4y ago
    Yea, gotts agree on this one. XML hits the sweat spot.
  - jalk 4y ago
    Schemas and includes are great features, but you will have to defend the syntax hill on your own - imo the syntax is neither human nor machine friendly
    - baq 4y ago
      Oh don’t get me wrong I don’t like xml. I just don’t like yaml so much more.
- salamander014 4y ago
  Use the service resource in Kubernetes to order a load balancer in different clouds. You need different tags, they function and are configured differently, and you still need to learn how it works in your cloud. Contrast this with the contract* K8 tries to sell you, which is "describe the intent and we'll figure it out for you" and you'll realize the cloud providers bunged up this whole thing. That's not to say K8 is perfect, but if you think for even a second that wiring up resources outside your kubernetes cluster will be lower-friction, kubernetes is falling short of it's goals.

MuffinFlavored 4y ago

> everything that isn't standard Could you give an example? I thought k8s and its YAML/practices was the standard?

jahsome 4y ago
I've always felt the standards are more about organization and common practices, whereas the implementation is still the wild west. Unfortunately in my experience, abstraction, even if structured, doesn't solve the main issue: it turns out the engineering operation of infra is still pretty hard. At least you know where to look to find the chaos though.
- MuffinFlavored 4y ago
  > whereas the implementation is still the wild west. in my own experience, i would've thought "ok, let me aptinstall k8s and then apply some yaml to it so i can avoid going to the cloud for a massive k8s bill for almost no reason" standing up kubeadm on a single master + worker node setup on a VM (like a droplet or EC2) is basically like, extremely non-standard + frowned upon then, does anybody actually write pure k8s yaml? or do they write jsonnet/kustomize/helm charts? do they apply with kubectl or the helm CLI or the argocd CLI?
  - sophiabits 4y ago
    > then, does anybody actually write pure k8s yaml? You can think of plain k8s yaml as a sort of machine code for k8s. You _can_ write yaml by hand, but it’s not generally how things are done outside of initially learning how k8s works. The tools you’ve mentioned all make working with k8s resources a lot more pleasant. Something simple like parameterizing the name of a secret instead of hardcoding it in N different yaml files saves a lot of time if you ever need to refactor, and being able to provide those values via a “standard” CLI tool makes automated deployments a lot easier compared to hacking together some yq commands. helm in particular tracks the history of any charts you’ve installed, and has a very simple “helm rollback” command which can get you back to the last working version of your application if things end up going wrong. The exact tools used by each team will differ; you could use any of them. But no one in my experience is writing plain yaml manifests and kubectl applying them in production contexts.
Topgamer7 4y ago
He means part of the k8s standard. Once you get to the non-standard, cloud specific configuration, things go haywire. For example try to configure a udp connection in k8s on aws.
- MuffinFlavored 4y ago
  oh, like azure k8s vs aws k8s vs gcp k8s? each has their own way to like... attach a volume for block storage or something? i thought that was all abstracted away/standardized? > Once you get to the non-standard, cloud specific configuration what's a common use case for when this is hit?
baq 4y ago
YAML practices… YAML is not a language, it’s a syntax tree. It pretends to be configuration and what it really becomes is some kind of a gimped template engine which people put layers and layers of code generators on top of to manage the complexity. XML was better and I’m willing to die on this hill.
- Alifatisk 4y ago
  Yea, gotts agree on this one. XML hits the sweat spot.
- jalk 4y ago
  Schemas and includes are great features, but you will have to defend the syntax hill on your own - imo the syntax is neither human nor machine friendly
  - baq 4y ago
    Oh don’t get me wrong I don’t like xml. I just don’t like yaml so much more.
salamander014 4y ago
Use the service resource in Kubernetes to order a load balancer in different clouds. You need different tags, they function and are configured differently, and you still need to learn how it works in your cloud. Contrast this with the contract* K8 tries to sell you, which is "describe the intent and we'll figure it out for you" and you'll realize the cloud providers bunged up this whole thing. That's not to say K8 is perfect, but if you think for even a second that wiring up resources outside your kubernetes cluster will be lower-friction, kubernetes is falling short of it's goals.