8 ms·

Interesting that the mania for over-investment in devops is beginning to abate. Here on Hacker News I was a steady critic of both Docker and Kubernetes, going t

Interesting that the mania for over-investment in devops is beginning to abate. Here on Hacker News I was a steady critic of both Docker and Kubernetes, going to at least 2017, but most of these posts were unpopular. I have to go back to 2019 to find one that sparked a conversation: https://news.ycombinator.com/item?id=20371961 https://news.ycombinator.com/item?id=20371961 The stuff I posted about Kubernetes did not draw a conversation, but I was simply documenting what I was seeing: vast over-investment in devops even at tiny startups that were just getting going and could have easily dumped everything on a single server, exactly as we used to do things back in 2005.

pclmulqdq 2y ago
The attraction of this stuff is mostly the ability to keep your infrastructure configurations as code. However, I have previously checked in my systemd cofig files for projects and set up a script to pull them on new systems. It's not clear that docker-compose or even kubernetes* is that much more complicated if you are only running 3 things. * if you are an experienced user
- honkycat 2y ago
  Having done both: running a small Kubernetes cluster is simpler than managing a bunch of systemd files.
  - worldsayshi 2y ago
    Yeah this is my impression as well which makes me not understand the k8s hate.
    - pclmulqdq 2y ago
      The complexity of k8s comes the moment you need to hold state of some kind. Now instead of one systemd entry, we have to worry about persistent volume claims and other such nonsense. When you are doing things that are completely stateless, it's simpler than systemd.
      - p_l 2y ago
        If you need to care about state with systemd you still have the "nonsense" of persistent volume claims, they are just something you keep in notes somewhere, in my experience usually in heads of the sysadmins or an excel sheet or a text file that tries to track which server has what data connected how.
        pclmulqdq 2y ago
        Understand that in the hypothetical system we are discussing, there are something like 1-2 servers. In that case the "volume claim" is just "it's a file on the obvious filesystem" and does not actually need to be spelled out they way you need to spell it out in k8s. The file path you give in environment variables is where the most up-to-date version of the volume claim is. And that file is free to expand to hundreds of GB without bothering you.
        p_l 2y ago
        Things get iffier when you start doing things like running multiple instances of something (maybe you're sticking two test environments for your developers), or suddenly you grew a bit or no longer fit on the server and start migrating around. The complexity of PVCs in my experience isn't really that big compared to this, possibly lower, and I did stuff both ways.
OtomotO 2y ago
It's just the hype moving on. Every generation has to make similar mistakes again and again. I am sure if we had the opportunity and the hype was there we would've used k8s in 2005 as well. The same thing is true for e.g. JavaScript on the frontend. I am currently migrating a project from React to HTMX. Suddenly there is no build step anymore. Some people were like: "That's possible?" Yes, yes it is and it turns out for that project it increases stability and makes everything less complex while adding the exact same business value. Does that mean that React is always the wrong choice? Well, yes, React sucks, but solutions like React? No! It depends on what you need, on the project! Just as a carpenter doesn't use a hammer to saw, we as a profession should strive to use the right tool for the right job. (Albeit it's less clear than for the carpenter, granted)
- augbog 2y ago
  It's actually kinda hilarious how RSC (React Server Components) is pretty much going back to what PHP was but yeah proves your point as hype moves on people begin to realize why certain things were good vs not
- ajayvk 2y ago
  Along those lines, I am building https://github.com/claceio/clace https://github.com/claceio/clace for teams to deploy internal tools. It provides a Cloud Run type interface to run containers, including scaling down to zero. It implements an application server than runs containerized apps. Since HTMX was mentioned, Clace also makes it easy to build Hypermedia driven apps.
  - MortyWaves 2y ago
    Would you be open to non Python support as well? This tool seems useful, very useful in fact, but I mainly use .NET (which yes can run very well in containers).
    - ajayvk 2y ago
      Starlark (python like config language) is used to configure Clace. For containerized apps, python frameworks are supported without a Dockerfile being required. All other languages currently require a user provided Dockerfile, the `container` spec can be used. I do plan to add specs for other languages. New specs have to be added here https://github.com/claceio/appspecs https://github.com/claceio/appspecs. New specs can be created locally also in the config, see https://clace.io/docs/develop/#building-apps-from-spec https://clace.io/docs/develop/#building-apps-from-spec
valenterry 2y ago
So, let's say you want to deploy server instances. Let's keep it simple and say you want to have 2 instances running. You want to have zero-downtime-deployment. And you want to have these 2 instances be able to access configuration (that contains secrets). You want load balancing, with the option to integrate an external load balancer. And, last, you want to be able to run this setup both locally and also on at least 2 cloud providers. (EDIT: I meant to be able to run it on 2 cloud providers. Meaning, one at a time, not both at the same time. The idea is that it's easy to migrate if necessary) This is certainly a small subset of what kubernetes offers, but I'm curious, what would be your goto-solution for those requirements?
- caseyohara 2y ago
  I think you are proving the point; there are very, very few applications that need to run on two cloud providers. If you do, sure, use Kubernetes if that makes your job easier. For the other 99% of applications, it’s overkill. Apart from that requirement, all of this is very doable with EC2 instances behind an ALB, each running nginx as a reverse proxy to an application server with hot restarting (e.g. Puma) launched with a systemd unit.
  - osigurdson 2y ago
    To me that sounds harder than just using EKS. Also, other people are more likely to understand how it works, can run it in other environments (e.g. locally), etc.
  - valenterry 2y ago
    Sorry, that was a misunderstanding. I meant that I want to be able to run it on two cloud providers, but one at a time is fine. It just means that it would be easy to migrate/switch over if necessary.
  - globular-toast 2y ago
    Hmm, let's see, so you've got to know: EC2, ALB, Nginx, Puma, Systemd, then presumably something like Terraform and Ansible to deploy those configs, or write a custom set of bash scripts. And all of that and you're tied to one cloud provider. Or, instead of reinventing the same wheels for Nth time, I could just use a set of abstractions that work for 99% of network services out there, on any cloud or bare metal. That set of abstractions is k8s.
honkycat 2y ago
Start-ups that don't need to scale will quickly go away, because how else are you going to make a profit? How have you been going since 2005 and still not understand the economics of software?
- ndriscoll 2y ago
  CPUs are ~300x more powerful and storage offers ~10,000x more IOPS than 2005 hardware. More efficient server code exists today. You can scale very far on one server. If you were bootstrapping a startup, you could probably plan to use a pair of gaming PCs until at least the first 1-10M users.
  - shakiXBT 2y ago
    10 million users on a pair of gaming PCs is ridiculous. What's your product, a website that tells the current time?
    - ndriscoll 2y ago
      How many requests do you expect users actually do? Especially if you're serving a B2B market; not everything is centered around addiction/"engagement". My 8 year old PC can do over 10k page requests/second for a reddit or myspace clone (without getting into caching). A modern high end gaming PC should be around 10x more capable (in terms of both CPU and storage IOPS). The limit in terms of needing to upgrade to "unusual" hardware for a PC would likely be the NIC. Networking is one place where typical consumer gear is stuck in 2005. Webapps might make it hard to tell, but a modern computer (or even an old computer like mine) is mindbogglingly fast.
- arlcode 2y ago
  Just to make it clear: There are a million use cases that don't involve scaling fast. For example B2B businesses where you have very few but extremely high value customers for specialized use cases. Another one is building bully hardware. Your software infrastructure does not need to grow any faster than your shop floor is building it. Whether you want to call that a "startup" is up for debate (and mostly semanticist if you ask me) but at one point they were all a zero employee company and needed to survive their first 5 years. In general you won't find their products on the app store.
sobellian 2y ago
I've worked at a few tiny startups, and I've both manually administered a single server and run small k8s clusters. k8s is way easier. I think I've spent 1, maybe 2 hours on devops this year. It's not a full-time job, it's not a part-time job, it's not even an unpaid internship. Perhaps at a bigger company with more resources and odd requirements...
- nicce 2y ago
  But how much this costs extra? Sounds like you are using cloud-provided k8s.
  - sobellian 2y ago
    EKS is priced at $876 / yr / cluster at current rates. Negligible for me personally, it's much less than either our EC2 or RDS costs.
    - fer 2y ago
      Yeah, using EKS isn't the same thing as "administering k8s", unless I misread you above. Actual administration is already done for you, it's batteries included, turn-key, and integrated with everything AWS. A job ago we had our own k8s cluster in our own DC, and it required a couple of teams to keep running and reasonably integrated with everything else in the rest of the company. It was probably cheaper overall than cloud given the compute capacity we had, but also probably not by much given the amount of people dedicated to it. Even my 3-node k3s at home requires more attention than what you described.
      - sobellian 2y ago
        You did misread me, I never said I administered k8s. The quoted phrase does not exist :)
  - p_l 2y ago
    I currently use k8s to control bunch of servers. The amount of work/cost of using k8s for handling them in comparison to doing it "old style" is probably negative by now.
santoshalper 2y ago
As an industry, we spent so much time sharpening our saw that we nearly forgot to cut down the tree.
rozap 2y ago
ZIRP is over.
harrall 2y ago
People gravely miss-understand containerization and Docker. All it lets you do is put shell commands into a text file and be able to run it self-contained anywhere. What is there to hate? You still use the same local filesystem, the same host networking, still rsync your data dir, still use the same external MySQL server even if you want -- nothing has changed. You do NOT need a load balancer, a control plane, networked storage, Kubernetes or any of that. You ADD ON those things when you want them like you add on optional heated seats to your car.
- skydhash 2y ago
  Why would you want to run it anywhere. People mostly select an OS and just update that. It may be great when distributing applications for others to host, but not when it’s the only strategy. I have to reverse engineer dockerfiles when the developer wouldn’t provide a proper documentation.
  - ftmch 2y ago
    OS upgrades are a pain. Even just package updates could break everything. Having everything in containers makes migrating to another system much easier.
sunshine-o 2y ago
Kubernetes, as an industry standard that a lot of people complain about is just a sitting duck waiting to be disrupted. Anybody who doesn't have the money, time or engineering resources will jump on whatever appear as a decent alternative. My intuition is that alternative already exist but I can't see it... A bit like Spring emerged as an alternative to J2EE or what HTMX is to React & co. Is it k3s or something more radical? Is it on a chinese Github?
- ftmch 2y ago
  I wish Docker Swarm would get more attention. It could be the perfect Kubernetes lightweight alternative. Instead it seems like it could get deprecated any day now.

pclmulqdq 2y ago

The attraction of this stuff is mostly the ability to keep your infrastructure configurations as code. However, I have previously checked in my systemd cofig files for projects and set up a script to pull them on new systems. It's not clear that docker-compose or even kubernetes* is that much more complicated if you are only running 3 things. * if you are an experienced user

honkycat 2y ago
Having done both: running a small Kubernetes cluster is simpler than managing a bunch of systemd files.
- worldsayshi 2y ago
  Yeah this is my impression as well which makes me not understand the k8s hate.
  - pclmulqdq 2y ago
    The complexity of k8s comes the moment you need to hold state of some kind. Now instead of one systemd entry, we have to worry about persistent volume claims and other such nonsense. When you are doing things that are completely stateless, it's simpler than systemd.
    - p_l 2y ago
      If you need to care about state with systemd you still have the "nonsense" of persistent volume claims, they are just something you keep in notes somewhere, in my experience usually in heads of the sysadmins or an excel sheet or a text file that tries to track which server has what data connected how.
      - pclmulqdq 2y ago
        Understand that in the hypothetical system we are discussing, there are something like 1-2 servers. In that case the "volume claim" is just "it's a file on the obvious filesystem" and does not actually need to be spelled out they way you need to spell it out in k8s. The file path you give in environment variables is where the most up-to-date version of the volume claim is. And that file is free to expand to hundreds of GB without bothering you.
        p_l 2y ago
        Things get iffier when you start doing things like running multiple instances of something (maybe you're sticking two test environments for your developers), or suddenly you grew a bit or no longer fit on the server and start migrating around. The complexity of PVCs in my experience isn't really that big compared to this, possibly lower, and I did stuff both ways.

OtomotO 2y ago

It's just the hype moving on. Every generation has to make similar mistakes again and again. I am sure if we had the opportunity and the hype was there we would've used k8s in 2005 as well. The same thing is true for e.g. JavaScript on the frontend. I am currently migrating a project from React to HTMX. Suddenly there is no build step anymore. Some people were like: "That's possible?" Yes, yes it is and it turns out for that project it increases stability and makes everything less complex while adding the exact same business value. Does that mean that React is always the wrong choice? Well, yes, React sucks, but solutions like React? No! It depends on what you need, on the project! Just as a carpenter doesn't use a hammer to saw, we as a profession should strive to use the right tool for the right job. (Albeit it's less clear than for the carpenter, granted)

augbog 2y ago
It's actually kinda hilarious how RSC (React Server Components) is pretty much going back to what PHP was but yeah proves your point as hype moves on people begin to realize why certain things were good vs not
ajayvk 2y ago
Along those lines, I am building https://github.com/claceio/clace https://github.com/claceio/clace for teams to deploy internal tools. It provides a Cloud Run type interface to run containers, including scaling down to zero. It implements an application server than runs containerized apps. Since HTMX was mentioned, Clace also makes it easy to build Hypermedia driven apps.
- MortyWaves 2y ago
  Would you be open to non Python support as well? This tool seems useful, very useful in fact, but I mainly use .NET (which yes can run very well in containers).
  - ajayvk 2y ago
    Starlark (python like config language) is used to configure Clace. For containerized apps, python frameworks are supported without a Dockerfile being required. All other languages currently require a user provided Dockerfile, the `container` spec can be used. I do plan to add specs for other languages. New specs have to be added here https://github.com/claceio/appspecs https://github.com/claceio/appspecs. New specs can be created locally also in the config, see https://clace.io/docs/develop/#building-apps-from-spec https://clace.io/docs/develop/#building-apps-from-spec

valenterry 2y ago

So, let's say you want to deploy server instances. Let's keep it simple and say you want to have 2 instances running. You want to have zero-downtime-deployment. And you want to have these 2 instances be able to access configuration (that contains secrets). You want load balancing, with the option to integrate an external load balancer. And, last, you want to be able to run this setup both locally and also on at least 2 cloud providers. (EDIT: I meant to be able to run it on 2 cloud providers. Meaning, one at a time, not both at the same time. The idea is that it's easy to migrate if necessary) This is certainly a small subset of what kubernetes offers, but I'm curious, what would be your goto-solution for those requirements?

caseyohara 2y ago
I think you are proving the point; there are very, very few applications that need to run on two cloud providers. If you do, sure, use Kubernetes if that makes your job easier. For the other 99% of applications, it’s overkill. Apart from that requirement, all of this is very doable with EC2 instances behind an ALB, each running nginx as a reverse proxy to an application server with hot restarting (e.g. Puma) launched with a systemd unit.
- osigurdson 2y ago
  To me that sounds harder than just using EKS. Also, other people are more likely to understand how it works, can run it in other environments (e.g. locally), etc.
- valenterry 2y ago
  Sorry, that was a misunderstanding. I meant that I want to be able to run it on two cloud providers, but one at a time is fine. It just means that it would be easy to migrate/switch over if necessary.
- globular-toast 2y ago
  Hmm, let's see, so you've got to know: EC2, ALB, Nginx, Puma, Systemd, then presumably something like Terraform and Ansible to deploy those configs, or write a custom set of bash scripts. And all of that and you're tied to one cloud provider. Or, instead of reinventing the same wheels for Nth time, I could just use a set of abstractions that work for 99% of network services out there, on any cloud or bare metal. That set of abstractions is k8s.

honkycat 2y ago

Start-ups that don't need to scale will quickly go away, because how else are you going to make a profit? How have you been going since 2005 and still not understand the economics of software?

ndriscoll 2y ago
CPUs are ~300x more powerful and storage offers ~10,000x more IOPS than 2005 hardware. More efficient server code exists today. You can scale very far on one server. If you were bootstrapping a startup, you could probably plan to use a pair of gaming PCs until at least the first 1-10M users.
- shakiXBT 2y ago
  10 million users on a pair of gaming PCs is ridiculous. What's your product, a website that tells the current time?
  - ndriscoll 2y ago
    How many requests do you expect users actually do? Especially if you're serving a B2B market; not everything is centered around addiction/"engagement". My 8 year old PC can do over 10k page requests/second for a reddit or myspace clone (without getting into caching). A modern high end gaming PC should be around 10x more capable (in terms of both CPU and storage IOPS). The limit in terms of needing to upgrade to "unusual" hardware for a PC would likely be the NIC. Networking is one place where typical consumer gear is stuck in 2005. Webapps might make it hard to tell, but a modern computer (or even an old computer like mine) is mindbogglingly fast.
arlcode 2y ago
Just to make it clear: There are a million use cases that don't involve scaling fast. For example B2B businesses where you have very few but extremely high value customers for specialized use cases. Another one is building bully hardware. Your software infrastructure does not need to grow any faster than your shop floor is building it. Whether you want to call that a "startup" is up for debate (and mostly semanticist if you ask me) but at one point they were all a zero employee company and needed to survive their first 5 years. In general you won't find their products on the app store.

sobellian 2y ago

I've worked at a few tiny startups, and I've both manually administered a single server and run small k8s clusters. k8s is way easier. I think I've spent 1, maybe 2 hours on devops this year. It's not a full-time job, it's not a part-time job, it's not even an unpaid internship. Perhaps at a bigger company with more resources and odd requirements...

nicce 2y ago
But how much this costs extra? Sounds like you are using cloud-provided k8s.
- sobellian 2y ago
  EKS is priced at $876 / yr / cluster at current rates. Negligible for me personally, it's much less than either our EC2 or RDS costs.
  - fer 2y ago
    Yeah, using EKS isn't the same thing as "administering k8s", unless I misread you above. Actual administration is already done for you, it's batteries included, turn-key, and integrated with everything AWS. A job ago we had our own k8s cluster in our own DC, and it required a couple of teams to keep running and reasonably integrated with everything else in the rest of the company. It was probably cheaper overall than cloud given the compute capacity we had, but also probably not by much given the amount of people dedicated to it. Even my 3-node k3s at home requires more attention than what you described.
    - sobellian 2y ago
      You did misread me, I never said I administered k8s. The quoted phrase does not exist :)
- p_l 2y ago
  I currently use k8s to control bunch of servers. The amount of work/cost of using k8s for handling them in comparison to doing it "old style" is probably negative by now.

santoshalper 2y ago

As an industry, we spent so much time sharpening our saw that we nearly forgot to cut down the tree.

rozap 2y ago

ZIRP is over.

harrall 2y ago

People gravely miss-understand containerization and Docker. All it lets you do is put shell commands into a text file and be able to run it self-contained anywhere. What is there to hate? You still use the same local filesystem, the same host networking, still rsync your data dir, still use the same external MySQL server even if you want -- nothing has changed. You do NOT need a load balancer, a control plane, networked storage, Kubernetes or any of that. You ADD ON those things when you want them like you add on optional heated seats to your car.

skydhash 2y ago
Why would you want to run it anywhere. People mostly select an OS and just update that. It may be great when distributing applications for others to host, but not when it’s the only strategy. I have to reverse engineer dockerfiles when the developer wouldn’t provide a proper documentation.
- ftmch 2y ago
  OS upgrades are a pain. Even just package updates could break everything. Having everything in containers makes migrating to another system much easier.

sunshine-o 2y ago

Kubernetes, as an industry standard that a lot of people complain about is just a sitting duck waiting to be disrupted. Anybody who doesn't have the money, time or engineering resources will jump on whatever appear as a decent alternative. My intuition is that alternative already exist but I can't see it... A bit like Spring emerged as an alternative to J2EE or what HTMX is to React & co. Is it k3s or something more radical? Is it on a chinese Github?

ftmch 2y ago
I wish Docker Swarm would get more attention. It could be the perfect Kubernetes lightweight alternative. Instead it seems like it could get deprecated any day now.