5 ms·

>> abandon technology like Kubernetes I think a lot of Kubernetes hate is misplaced. It is a great piece of software engineering, well supported and runs every

>> abandon technology like Kubernetes I think a lot of Kubernetes hate is misplaced. It is a great piece of software engineering, well supported and runs everywhere. You certainly don't always need it but don't create a bunch of random bash scripts running all of the place instead of learning how to use it.

zer00eyz 2y ago
Its not. The whole ecosystem around it is an example of Conways law and and a Google product. None of the people using it are google. Google, also, runs its own hardware. Shockingly it is a great product if you rent hardware, autoscaling is autospending. No one knows what a feature costs any more because its all just a big bucket your pouring money into for amazon to have 30 percent margin on. We need operations people again, we need to stop using containers as bags for shitty "software" ... Do actual engineering.
- milkglass 2y ago
  Where's all the good operations people at nowadays? Have worked with numerous cloud native engineers that do not have good foundational knowledge.
  - llama052 2y ago
    I think that's the issue, it's not the tooling. Kubernetes is great... unfortunately people are awful at unpacking the underlying layers.
  - betaby 2y ago
    > cloud native engineers That's the problem. Yes, I'm the old man yelling at cloud, but indeed 'kids this days' don't know Linux, they know how to provision things from Terraform. In fact it feels like logging in via ssh and checking process with with `strace` is a lost art. Checking PCAP? That's a black magic!
    - sadops 2y ago
      This is the elephant in the room: all "new" technology isn't fundamentally new. Terraform is just curl with state management for lots of different websites, Andible is just a YAML to Bash converter, and so on. If you have good fundamentals, not only are these things easy, they're also incredibly frustrating, because you can easily see their limitations. You can always tell how experienced someone is by how well they know what a given tool is actually doing under the hood. People who exhibit shock and awe will just mess up your codebase, because they have no clue how the machine actually works. No, I want to hire the grumpy, cynical greybeards, because they know their fundamentals, and they're immune to shiny bullshit. There's no substitute for skill and experience, despite what the modern tech discourse says. "Anyone can program!" And yet, we have threads like this, where we find out, SHOCKER, that the best codebases are the ones maintained by like... four senior developers, half of whom contributed to several Internet RFCs, and half of whom used to work at Bell Labs. I feel like we are through the looking glass.
      - deleted 2y ago
        [deleted]
      - udev4096 2y ago
        Ansible is a lot more than just "YAML to Bash". For instance, Bash scripts are not idempotent. Ansible playbooks are. Sure, you can hack together a way to ssh and run commands very easily but it will break at some point
        p_l 2y ago
        Ansible playbooks are as idempotent as bash scripts. You need to include checks when writing them so they actually are idempotent, just like with bash script, and they happily leak idempotency-breaking details like "restarting a service" works only if a) service currently runs b) because "restart" is a command, not expected state of "the service was restarted and is now running
        bigstrat2003 2y ago
        Bash scripts are as idempotent as you want to make them, so I think the comparison holds.
  - Aeolun 2y ago
    > Where's all the good operations people at nowadays? Still in the same place they were 20 years ago. They’ve just been lost in the swamp of their peers that know nothing.
- llama052 2y ago
  Kubernetes is actually extremely popular all around the world. Chickfila if I recall correctly deploy it in every single store! A lot of big dinosaur corporations are implementing it actively. Unfortunately VMs or Kubernetes or whatever tooling is still going to suck if you have shitty people using them.
  - chris_wot 2y ago
    VMs are going to suck if you use VMWare and they hike the price four fold.
- whoknowsidont 2y ago
  >The whole ecosystem around it is an example of Conways law This is such an inaccurate take. >we need to stop using containers as bags Containers and container orchestration are a NEEDED and REQUIRED piece of the technology stack in the current reality. That doesn't mean people need to be ignorant of the details that make them work. As someone who's been around for a while we are in a better place than when we had "dedicated operations people" that just gatekept everything. What you hate are teams just parroting what other people are doing with the tech, not the tech itself. But let me tell you, if you're going to have teams of people pretend to know what they're doing wrapping it in a standard "bag" sure does make it a hell of a lot fucking easier to unfuck when things go wrong.
  - zer00eyz 2y ago
    > Containers and container orchestration are a NEEDED and REQUIRED piece of the technology stack in the current reality. If you're deploying NODEjs apps to the cloud, they sure are. > if you're going to have teams of people pretend to know what they're doing wrapping it in a standard "bag" sure does make it a hell of a lot fucking easier to unfuck when things go wrong. No one looks in the bag, they just roll back, the bag is opaque, the bag breeds more bags, less trnasparency and higher costs. Step back and read what you wrote, if you described any other relationship in your life in these terms, your friends would be having an intervention. The codependent circle jerk has enabled so much bad behavior. The tooling you're advocating for is an enabler. I want App Store like software roll outs. The infrastructure we have is kind of like that, if you don't look too close. We need to start doing the hard work to make it not suck. We need to stop pretending that this is a good place and put in the work to make it better. Otherwise its never going to get untucked...
    - whoknowsidont 2y ago
      >If you're deploying NODEjs apps to the cloud, they sure are. I don't use nodejs. >No one looks in the bag I sure have. Plenty of times. >if you described any other relationship in your life in these terms, your friends would be having an intervention. For having a standard tool that you know how to use? That doesn't seem correct and it certainly isn't proportional. >I want App Store like software roll outs. Okay it sounds like you want K8's.
weitendorf 2y ago
It's basically a prototype that industry ran away with. It leaks implementation details everywhere and pushes way too many config options up to the developer. Because industry ran away with it before it could be good, it takes projects like Cilium to push it in the right direction, but those take forever to get adopted and are really hard because they don't live in the product itself. You'd need like 10 more Ciliums to make Kubernetes actually good, and they'll never all get adopted. The tech is a dead end that demonstrates some nice tech/patterns but got overhyped. It's like the Hadoop of containers
- llama052 2y ago
  Can you clarify what you mean by "leaks implementation details everywhere"? I like to think of kubernetes as a big orchestration platform that you can choose to use what you need. If an ingress and pods work then use that, otherwise extend an throw an operator up for what you need (it likely already exists). Cilium for instance is great for that, so is Istio and the like. They aren't hard you just have to understand networking... which is nearly the same energy of running it on another orchestration tool or raw on a network device.
  - weitendorf 2y ago
    You shouldn’t have to think about all the implementation details of your deployment target. There shouldn’t be platform engineers or Kubernetes experts. Nobody should be writing YAML, or getting paid to set up Istio. Nobody should have to learn the Kubernetes architecture or know about the kubelet or EBPF. The tools should be simple enough with good defaults that application developers just click a button and have their code run somewhere. Right now platform engineers fill that gap, because the underlying tech doesn’t. IMO you are thinking too much like an engineer if you are saying you “just have to understand networking”. Why? It’s always better when the problem gets solved in a way that allows you to not think of it too much. Right now SREs and the platform team do that for application developers because Kubernetes only does it halfway. Infrastructure/devops/SRE is a pure means to an end of getting actual applications (the ultimate source of all the value in software) to run. It’s an obstacle. Right now the obstacle is bigger than it needs to be
physicles 2y ago
Agreed. If we didn’t use kubernetes, we’d have to reimplement a bunch of its features. Back when we were getting started I tried docker swarm because it was supposed to be simpler, but I had weird issues with its networking. The best part is multiple environments. I can run our full stack on my laptop with k3s and one call to make. We use the same yaml (with kustomize) for all three cloud environments. It feels pretty boring, which is fantastic.
- weitendorf 2y ago
  The argument against Kubernetes is not that you should go backwards from it, but that there is a huge need for something better than it. Many big tech companies have platforms that are better than it which aren’t properly (if at all) externally productized. And the fact that many medium companies have “platform teams” configuring Kubernetes and gluing together basically the same set of tools (source, build, test, release, ops, obs) together in basically the same way is a huge smell that something better is needed. Basically, doing things the right way is actually a big operational/engineering/monetary burden for most companies that just want to write applications. And K8s is a big part of that
  - stackskipton 2y ago
    Big tech companies are willing to invest in those platforms and mold them to work like they want to work. Few other companies do so and Kubernetes becomes the way to get big tech like platform. I do agree it could be better but it's Un opinioned nature also let's these companies shape it in a way that matches where they are. Also, probably biggest issues with it only come up if you are not in managed Kubernetes which many are.
    - physicles 2y ago
      K8s does solve a huge set of problems. But I agree with GP that all the non-application-specific parts on top, especially observability, are a huge time sink. The industry could use a Rails for the cloud. We built ours from the ground up and we’re in a good place now, but we kind of had to reverse engineer things like logging requirements from security questionnaires. I suppose these are things that consultants and saas companies can charge a lot of money for, so there’s no desire to open source it.
udev4096 2y ago
Whoever has worked with bash scripts would know that relying on a bunch of bash scripts for your infrastructure is extremely naive. At the end of the day, Kubernetes is just a tool with complexities in place for companies with larger workloads
28304283409234 2y ago
Many things are 'great pieces of [software] engineering'. That does not mean they are a fit solution to any problem. Many times a script or simple ansible playbook, or docker-compose is just a better fit.

zer00eyz 2y ago

Its not. The whole ecosystem around it is an example of Conways law and and a Google product. None of the people using it are google. Google, also, runs its own hardware. Shockingly it is a great product if you rent hardware, autoscaling is autospending. No one knows what a feature costs any more because its all just a big bucket your pouring money into for amazon to have 30 percent margin on. We need operations people again, we need to stop using containers as bags for shitty "software" ... Do actual engineering.

milkglass 2y ago
Where's all the good operations people at nowadays? Have worked with numerous cloud native engineers that do not have good foundational knowledge.
- llama052 2y ago
  I think that's the issue, it's not the tooling. Kubernetes is great... unfortunately people are awful at unpacking the underlying layers.
- betaby 2y ago
  > cloud native engineers That's the problem. Yes, I'm the old man yelling at cloud, but indeed 'kids this days' don't know Linux, they know how to provision things from Terraform. In fact it feels like logging in via ssh and checking process with with `strace` is a lost art. Checking PCAP? That's a black magic!
  - sadops 2y ago
    This is the elephant in the room: all "new" technology isn't fundamentally new. Terraform is just curl with state management for lots of different websites, Andible is just a YAML to Bash converter, and so on. If you have good fundamentals, not only are these things easy, they're also incredibly frustrating, because you can easily see their limitations. You can always tell how experienced someone is by how well they know what a given tool is actually doing under the hood. People who exhibit shock and awe will just mess up your codebase, because they have no clue how the machine actually works. No, I want to hire the grumpy, cynical greybeards, because they know their fundamentals, and they're immune to shiny bullshit. There's no substitute for skill and experience, despite what the modern tech discourse says. "Anyone can program!" And yet, we have threads like this, where we find out, SHOCKER, that the best codebases are the ones maintained by like... four senior developers, half of whom contributed to several Internet RFCs, and half of whom used to work at Bell Labs. I feel like we are through the looking glass.
    - deleted 2y ago
      [deleted]
    - udev4096 2y ago
      Ansible is a lot more than just "YAML to Bash". For instance, Bash scripts are not idempotent. Ansible playbooks are. Sure, you can hack together a way to ssh and run commands very easily but it will break at some point
      - p_l 2y ago
        Ansible playbooks are as idempotent as bash scripts. You need to include checks when writing them so they actually are idempotent, just like with bash script, and they happily leak idempotency-breaking details like "restarting a service" works only if a) service currently runs b) because "restart" is a command, not expected state of "the service was restarted and is now running
      - bigstrat2003 2y ago
        Bash scripts are as idempotent as you want to make them, so I think the comparison holds.
- Aeolun 2y ago
  > Where's all the good operations people at nowadays? Still in the same place they were 20 years ago. They’ve just been lost in the swamp of their peers that know nothing.
llama052 2y ago
Kubernetes is actually extremely popular all around the world. Chickfila if I recall correctly deploy it in every single store! A lot of big dinosaur corporations are implementing it actively. Unfortunately VMs or Kubernetes or whatever tooling is still going to suck if you have shitty people using them.
- chris_wot 2y ago
  VMs are going to suck if you use VMWare and they hike the price four fold.
whoknowsidont 2y ago
>The whole ecosystem around it is an example of Conways law This is such an inaccurate take. >we need to stop using containers as bags Containers and container orchestration are a NEEDED and REQUIRED piece of the technology stack in the current reality. That doesn't mean people need to be ignorant of the details that make them work. As someone who's been around for a while we are in a better place than when we had "dedicated operations people" that just gatekept everything. What you hate are teams just parroting what other people are doing with the tech, not the tech itself. But let me tell you, if you're going to have teams of people pretend to know what they're doing wrapping it in a standard "bag" sure does make it a hell of a lot fucking easier to unfuck when things go wrong.
- zer00eyz 2y ago
  > Containers and container orchestration are a NEEDED and REQUIRED piece of the technology stack in the current reality. If you're deploying NODEjs apps to the cloud, they sure are. > if you're going to have teams of people pretend to know what they're doing wrapping it in a standard "bag" sure does make it a hell of a lot fucking easier to unfuck when things go wrong. No one looks in the bag, they just roll back, the bag is opaque, the bag breeds more bags, less trnasparency and higher costs. Step back and read what you wrote, if you described any other relationship in your life in these terms, your friends would be having an intervention. The codependent circle jerk has enabled so much bad behavior. The tooling you're advocating for is an enabler. I want App Store like software roll outs. The infrastructure we have is kind of like that, if you don't look too close. We need to start doing the hard work to make it not suck. We need to stop pretending that this is a good place and put in the work to make it better. Otherwise its never going to get untucked...
  - whoknowsidont 2y ago
    >If you're deploying NODEjs apps to the cloud, they sure are. I don't use nodejs. >No one looks in the bag I sure have. Plenty of times. >if you described any other relationship in your life in these terms, your friends would be having an intervention. For having a standard tool that you know how to use? That doesn't seem correct and it certainly isn't proportional. >I want App Store like software roll outs. Okay it sounds like you want K8's.

weitendorf 2y ago

It's basically a prototype that industry ran away with. It leaks implementation details everywhere and pushes way too many config options up to the developer. Because industry ran away with it before it could be good, it takes projects like Cilium to push it in the right direction, but those take forever to get adopted and are really hard because they don't live in the product itself. You'd need like 10 more Ciliums to make Kubernetes actually good, and they'll never all get adopted. The tech is a dead end that demonstrates some nice tech/patterns but got overhyped. It's like the Hadoop of containers

llama052 2y ago
Can you clarify what you mean by "leaks implementation details everywhere"? I like to think of kubernetes as a big orchestration platform that you can choose to use what you need. If an ingress and pods work then use that, otherwise extend an throw an operator up for what you need (it likely already exists). Cilium for instance is great for that, so is Istio and the like. They aren't hard you just have to understand networking... which is nearly the same energy of running it on another orchestration tool or raw on a network device.
- weitendorf 2y ago
  You shouldn’t have to think about all the implementation details of your deployment target. There shouldn’t be platform engineers or Kubernetes experts. Nobody should be writing YAML, or getting paid to set up Istio. Nobody should have to learn the Kubernetes architecture or know about the kubelet or EBPF. The tools should be simple enough with good defaults that application developers just click a button and have their code run somewhere. Right now platform engineers fill that gap, because the underlying tech doesn’t. IMO you are thinking too much like an engineer if you are saying you “just have to understand networking”. Why? It’s always better when the problem gets solved in a way that allows you to not think of it too much. Right now SREs and the platform team do that for application developers because Kubernetes only does it halfway. Infrastructure/devops/SRE is a pure means to an end of getting actual applications (the ultimate source of all the value in software) to run. It’s an obstacle. Right now the obstacle is bigger than it needs to be

physicles 2y ago

Agreed. If we didn’t use kubernetes, we’d have to reimplement a bunch of its features. Back when we were getting started I tried docker swarm because it was supposed to be simpler, but I had weird issues with its networking. The best part is multiple environments. I can run our full stack on my laptop with k3s and one call to make. We use the same yaml (with kustomize) for all three cloud environments. It feels pretty boring, which is fantastic.

weitendorf 2y ago
The argument against Kubernetes is not that you should go backwards from it, but that there is a huge need for something better than it. Many big tech companies have platforms that are better than it which aren’t properly (if at all) externally productized. And the fact that many medium companies have “platform teams” configuring Kubernetes and gluing together basically the same set of tools (source, build, test, release, ops, obs) together in basically the same way is a huge smell that something better is needed. Basically, doing things the right way is actually a big operational/engineering/monetary burden for most companies that just want to write applications. And K8s is a big part of that
- stackskipton 2y ago
  Big tech companies are willing to invest in those platforms and mold them to work like they want to work. Few other companies do so and Kubernetes becomes the way to get big tech like platform. I do agree it could be better but it's Un opinioned nature also let's these companies shape it in a way that matches where they are. Also, probably biggest issues with it only come up if you are not in managed Kubernetes which many are.
  - physicles 2y ago
    K8s does solve a huge set of problems. But I agree with GP that all the non-application-specific parts on top, especially observability, are a huge time sink. The industry could use a Rails for the cloud. We built ours from the ground up and we’re in a good place now, but we kind of had to reverse engineer things like logging requirements from security questionnaires. I suppose these are things that consultants and saas companies can charge a lot of money for, so there’s no desire to open source it.

udev4096 2y ago

Whoever has worked with bash scripts would know that relying on a bunch of bash scripts for your infrastructure is extremely naive. At the end of the day, Kubernetes is just a tool with complexities in place for companies with larger workloads

28304283409234 2y ago

Many things are 'great pieces of [software] engineering'. That does not mean they are a fit solution to any problem. Many times a script or simple ansible playbook, or docker-compose is just a better fit.