13 ms·

The question of whether to use K8s or not is like wondering what kind of saw you should use to cut wood. There's different saws for different purposes. But even

The question of whether to use K8s or not is like wondering what kind of saw you should use to cut wood. There's different saws for different purposes. But even with the right saw, you still have to know how to use it correctly. Better to use a hand saw correctly than a table saw incorrectly. (you can use a table saw incorrectly, but best case the work ends up crap, worst case you lose a finger) After building infrastructure for dozens of teams, I'm quite convinced of the following: - if your people aren't very skilled, they won't build anything well. most software engineers i've seen professionally working in the cloud are handymen trying to build a wood cabinet. - if your people can't build well, it doesn't matter what technology they use. choosing between building a cabinet out of metal or cherry wood doesn't make much difference if they've never built a cabinet before. - if the first two holds: then only use the technology which requires the least skill to use well, and where the amount of maintenance is closest to zero. don't build a wood cabinet from scratch when you can buy flat pack. don't buy flat pack when you can buy an assembled cabinet, get it shipped, and carried into your office. - if using the aforementioned technology requires 'building' or 'assembling', and that is not core to the customer-facing aspect of your product, then you should not be building, you should be buying. if your business doesn't involve assembling flat pack furniture, don't ask your employees to build their own desks and chairs from Home Depot or Ikea parts. buy the premade desk and chairs, use them to make your actual product. - a software engineer knows as much about cloud architecture as a fine woodworker knows about framing. "it's all just wood" until the house takes 10x as long to frame and is 10x as expensive and still doesn't meet code. - people will try to build things they don't fully understand and leave the company before anyone realizes the mess they've made. imagine your retail store is accessible by driving a car over a wooden bridge built by a handyman.

CoastalCoder 4y ago
> Better to use a hand saw correctly than a table saw incorrectly. To be fair, better to use a handsaw even incorrectly than a tablesaw incorrectly.
- Jistern 4y ago
  I concur.
paulryanrogers 4y ago
Can small companies afford the complexities of k8s? I'm not yet convinced. Its pros are compelling yet the learning curve is steep. Which suggests a smaller talent pool.
- throwaway787544 4y ago
  Can small delivery companies afford the complexities of flat-bed trucks? Well: are they driving them? Repairing them? Assembling them? Loading/unloading? You can bet that using a passenger car instead can be more "affordable", because more people know how to drive them, repair them, assemble, load/unload, they're cheaper, etc. However, if what they're delivering can't be hauled by a passenger car, or the logistics wouldn't make financial sense, that again changes the equation. There's no one answer.
  - paulryanrogers 4y ago
    So is everything not K8s a passenger car in this analogy? Solution space is vast. We're all trying and weighing alternatives as we have the time and capacity.
- antonvs 4y ago
  If you use a good managed k8s offering, the complexities are not that great. GKE with Autopilot is a good option. In that case you don't need to know much more than how to write the yaml for a deployment. I've shown developers at all levels how to do that, it's not a barrier.
  - hobofan 4y ago
    +1. GKE Autopilot and sticking to the core ~4 Kubernetes object kinds (Deplyoment, Service, Ingress), was a really easy way for us to get started with K8s (and in many cases can carry you a really long way).
    - amtamt 4y ago
      Though easy to run, if there are multiple workloads of varying resource requirements, there will be a lots of wasted CPU and RAM, just because there are minimum CPU and CPU to RAM requirements.
mrk8s 4y ago
Kubernetes is not a saw. It's a crane.
throwamon 4y ago
> you can use a table saw incorrectly, but best case the work ends up crap, worst case you lose a finger Funny, just yesterday someone mentioned how Rust is compared to a SawStop: https://news.ycombinator.com/item?id=31784253 https://news.ycombinator.com/item?id=31784253 And here's a test showing how unlikely it is that it'll chop off your finger: https://youtu.be/SYLAi4jwXcs https://youtu.be/SYLAi4jwXcs
- throwaway787544 4y ago
  But that won't stop your workpiece from flying into your eyeball/jaw/neck/chest, or you from inhaling fine particles. Safety features are nice, but aren't a replacement for knowing how to use the tool safely. And that's just safety. Knowing how to use the tool properly means using a featherboard, or not using both the miter gauge and fence, using a crosscut sled, and other things which ensure your piece won't be marred or jammed or bind or cut incorrectly. Proper use also leads to increased efficiency and less waste. Same things apply to K8s. Even if you buy some managed product or safety feature, you still need to know how to use it right after that point, or your work will end up shoddy.
  - pcthrowaway 4y ago
    > eyeball The rest of your points stand (though I suspect a saw particle colliding with your jaw is unlikely to cause damage as severe as cutting off your finger), but any responsible person will be wearing safety goggles when operating a table saw.
    - tene 4y ago
      I have never used a table saw before, but from my amateur youtube understanding, safety goggles aren't perfect protection against kickback, right? Like, if I've got a chunk of wood being hurled at my face at high speeds, I'm definitely going to prefer to be wearing safety goggles, but I can still imagine the wood coming in at an angle and knocking them away, or being hit with enough force to break the goggles.
mathattack 4y ago
I’ve experienced a lot of people recommending and learning a trendy technology to get it on their resume, and then leaving a mess behind when it gets them a new job. Leadership can’t fully abdicate the decision.
- aaronbrethorst 4y ago
  Lots of whom are here on HN recommending trendy technologies.
TameAntelope 4y ago
I guess you've never met a competent generalist, then, which is funny because that's exactly who you need as your first employee.
- derfabianpeter 4y ago
  Funny part is that „competent generalists“ (those Brent-style types) are very rare and most technology needs to be designed for incompetent specialists. Also funny is that these generalists typically build as if everyone has their own skillset which renders many systems unusable once Brent has left the building.
  - manigandham 4y ago
    They're not just rare, the bigger problem is that most companies don't know how to deal with them. They either can't pay for them, don't give them enough freedom, weigh them down in a big team, or any number of other issues. One person in a specific role (even if they're not very good at it) is far easier for most organizations to deal with than the 10x generalist that can get things done. It can work well in very early startups, but there's limited reward there for it to be worth it.
  - DeathArrow 4y ago
    >those Brent-style types Which Brent are you talking about?
    - TedDoesntTalk 4y ago
      I get the sense the name is being used in a general way like a “Karen”, but I’ve never heard of a “Brent”.
      - mwarkentin 4y ago
        From The Phoenix Project: https://allengeer.com/how-to-handle-brent-in-the-phoenix-project/ https://allengeer.com/how-to-handle-brent-in-the-phoenix-pro...
manigandham 4y ago
Well said. Your first 3 points summarize the entirety of the problem. Kubernetes is basically best practices (with some quirks) of running containers reliably across a cluster of machines with various deployment types, logging, DNS/discovery, networking, storage, load-balancing, high-availability and other features built-in. If you don't need any of that then don't use it, but if you do then you probably won't build a custom infrastructure that does it better.
bazhova 4y ago
The last company I worked for was essentially finance bros who had a no-code investment solution but wanted to sprinkle ML on top to get clients. Suddenly it needed to be able to run air-gapped on prem. Oh also on Ali cloud in china. Oh also on GCP and AWS hybrid. Business promised the clients it was ready before we even started building. 90% of the team was under 25. We tried our damned hardest. Used K8s to make the whole thing platform agnostic. It worked but it cost a lot. Business people are the worst.
- BossingAround 4y ago
  Sounds like the exact use case for OpenShift.
  - nickstinemates 4y ago
    i dont think there's any use case for openshift
    - throwaway787544 4y ago
      use case: "I would like to use Kubernetes but I also would like a monkey to occasionally come out of a cage and hit my balls with a hammer"
  - systemBuilder 4y ago
    Open shift : an unstable trailing-edge version of Kubernetes with too much security to the point of wasting everybody's time. Or so says my team which is still recovering from that unmitigated disaster ...
- evoxmusic 4y ago
  Sounds like the are looking for Qovery
- dx034 4y ago
  Isn't a single binary the best fit in this case? Telling clients they need to build up a kubernetes cluster (especially in finance) won't always be the easiest decision.
goodpoint 4y ago
> if your people aren't very skilled, they won't build anything well If your people aren't very skilled, making them manage a ton of unnecessary complexity is not a good idea. k8s is not a saw. It's a hovercraft. Expensive to maintain and very few people need one.
danjac 4y ago
The problem I keep seeing in startups is that a lot of startup tech founders and employees don't see a startup as an embryonic business that will grow over the long term into a money-making venture requiring calculated risks and priorities, but rather as a personal playground to do all the fun things they were not allowed to do at their old BigCorp. Use cool framework or language X? Use Kubernetes? Microservices? Lambdas? No-SQL? Two fingers to BigCorp, I'm CTO/tech employee #1 at a new startup and I get to do whatever I want. If and when it goes down in flames, I now have these skills on my resume I can leverage into a better opportunity. Ideally your first tech employee/CTO should be a seasoned hand who has been burned before, who keeps an open mind to new approaches but knows when and how to adopt it. But it's quite rare to have seasoned developers in such startups, they're not attracted by the risk and chaos of these environments as they have better options and startups generally can't afford them.
- dsr_ 4y ago
  Given the failure rate of startups, "I now have these skills on my resume I can leverage into a better opportunity" may be the actual goal of most employees.
  - abirch 4y ago
    I've seen this frequently in large companies as well. That's why you see legacy systems with random tech.
- mooreds 4y ago
  > Use cool framework or language X? Use Kubernetes? Microservices? Lambdas? No-SQL? Two fingers to BigCorp, I'm CTO/tech employee #1 at a new startup and I get to do whatever I want. If and when it goes down in flames, I now have these skills on my resume I can leverage into a better opportunity. I was building a SaaS application (as a tech co-founder) and chose rails. It was newish to me (I'd done 1-2 projects) and I admit, I wanted to get better at rails. But it was also, after assessing the options and my (limited) knowledge of the future, the tech with the best chance of helping the company succeed. I have definitely run into startups where there was overly complicated technology, but tended to be later stage and have raised some money. As a founder/employee #1, I have a hard time believing anyone would pick tech just to burnish their resume; they should be trying to ship so the company succeeds. What am I missing?
pojzon 4y ago
TL;DR: - if you want to build something yourself -> you need experienced ppl to do that - if you dont have the right ppl -> you better pick a SaaS or PaaS, but your business will be at mercy of your provider - there is a huge shortage of experienced skilled ppl
mberning 4y ago
This is so spot on. Developers and “software engineers” seem to have a pathological habit of wanting to bring in gee-whiz technologies that they only have a cursory knowledge of and actually do very little to make the products and projects better.
mmcnl 4y ago
I like what you're saying, but your woodworking metaphor falls a bit flat for me because it teaches me more about woodworking than about building IT infrastructure.
osigurdson 4y ago
As intoxicating as they are, we need to move away from using analogies to problem domains most know little about to attempt to gain further insight into our own.
closeparen 4y ago
> a software engineer knows as much about cloud architecture as a fine woodworker knows about framing A mobile or frontend engineer, perhaps. But the idea that implementing and architecting the same type of object (backend services) are mutually exclusive skillsets strikes me as ridiculous. Since architecture decisions are by definition highly consequential and difficult to back out of, you want your most skilled and experienced software engineers making them, but it is precisely their software engineering experience that makes them qualified to do architecture. I would never trust a so-called "cloud architect" who is not also a first-rate backend engineer.
- throwaway787544 4y ago
  The thing is, Architect is its own role and discipline, and Cloud Architecture is distinct from Software Architecture. For example, you have to know why using PrivateLink is a bad idea, what kinds of backups to what locations/accounts you can do with an encrypted RDS database with a CMK, the 50 different ways you can screw up access to an S3 object, the limits of how many IPs EKS can use per node, why somebody should use the M5dn instance type, when rolling+maintaining your own complicated expensive service might be way better than using a cloud-managed version, what services you cannot use in what regions and zones. You have to be a budget wizard, keep up on the latest solutions, and know how all this dovetails with cloud security. A Software Architect should not have all of that Cloud trivia rattling around in their brain, but it's essential for a great Cloud Architect. Conversely, a Cloud Architect doesn't need to be a great coder to design great Cloud architecture. (Similar to how a construction architect doesn't need to be a great framer to know how the code affects framing, and how a great framer doesn't need to know how to calculate bending moments and shear forces in laminated beams to plan what day they'll need a lift for the beam and how to attach it)

CoastalCoder 4y ago

> Better to use a hand saw correctly than a table saw incorrectly. To be fair, better to use a handsaw even incorrectly than a tablesaw incorrectly.

Jistern 4y ago
I concur.

paulryanrogers 4y ago

Can small companies afford the complexities of k8s? I'm not yet convinced. Its pros are compelling yet the learning curve is steep. Which suggests a smaller talent pool.

throwaway787544 4y ago
Can small delivery companies afford the complexities of flat-bed trucks? Well: are they driving them? Repairing them? Assembling them? Loading/unloading? You can bet that using a passenger car instead can be more "affordable", because more people know how to drive them, repair them, assemble, load/unload, they're cheaper, etc. However, if what they're delivering can't be hauled by a passenger car, or the logistics wouldn't make financial sense, that again changes the equation. There's no one answer.
- paulryanrogers 4y ago
  So is everything not K8s a passenger car in this analogy? Solution space is vast. We're all trying and weighing alternatives as we have the time and capacity.
antonvs 4y ago
If you use a good managed k8s offering, the complexities are not that great. GKE with Autopilot is a good option. In that case you don't need to know much more than how to write the yaml for a deployment. I've shown developers at all levels how to do that, it's not a barrier.
- hobofan 4y ago
  +1. GKE Autopilot and sticking to the core ~4 Kubernetes object kinds (Deplyoment, Service, Ingress), was a really easy way for us to get started with K8s (and in many cases can carry you a really long way).
  - amtamt 4y ago
    Though easy to run, if there are multiple workloads of varying resource requirements, there will be a lots of wasted CPU and RAM, just because there are minimum CPU and CPU to RAM requirements.

mrk8s 4y ago

Kubernetes is not a saw. It's a crane.

throwamon 4y ago

> you can use a table saw incorrectly, but best case the work ends up crap, worst case you lose a finger Funny, just yesterday someone mentioned how Rust is compared to a SawStop: https://news.ycombinator.com/item?id=31784253 https://news.ycombinator.com/item?id=31784253 And here's a test showing how unlikely it is that it'll chop off your finger: https://youtu.be/SYLAi4jwXcs https://youtu.be/SYLAi4jwXcs

throwaway787544 4y ago
But that won't stop your workpiece from flying into your eyeball/jaw/neck/chest, or you from inhaling fine particles. Safety features are nice, but aren't a replacement for knowing how to use the tool safely. And that's just safety. Knowing how to use the tool properly means using a featherboard, or not using both the miter gauge and fence, using a crosscut sled, and other things which ensure your piece won't be marred or jammed or bind or cut incorrectly. Proper use also leads to increased efficiency and less waste. Same things apply to K8s. Even if you buy some managed product or safety feature, you still need to know how to use it right after that point, or your work will end up shoddy.
- pcthrowaway 4y ago
  > eyeball The rest of your points stand (though I suspect a saw particle colliding with your jaw is unlikely to cause damage as severe as cutting off your finger), but any responsible person will be wearing safety goggles when operating a table saw.
  - tene 4y ago
    I have never used a table saw before, but from my amateur youtube understanding, safety goggles aren't perfect protection against kickback, right? Like, if I've got a chunk of wood being hurled at my face at high speeds, I'm definitely going to prefer to be wearing safety goggles, but I can still imagine the wood coming in at an angle and knocking them away, or being hit with enough force to break the goggles.

mathattack 4y ago

I’ve experienced a lot of people recommending and learning a trendy technology to get it on their resume, and then leaving a mess behind when it gets them a new job. Leadership can’t fully abdicate the decision.

aaronbrethorst 4y ago
Lots of whom are here on HN recommending trendy technologies.

TameAntelope 4y ago

I guess you've never met a competent generalist, then, which is funny because that's exactly who you need as your first employee.

derfabianpeter 4y ago
Funny part is that „competent generalists“ (those Brent-style types) are very rare and most technology needs to be designed for incompetent specialists. Also funny is that these generalists typically build as if everyone has their own skillset which renders many systems unusable once Brent has left the building.
- manigandham 4y ago
  They're not just rare, the bigger problem is that most companies don't know how to deal with them. They either can't pay for them, don't give them enough freedom, weigh them down in a big team, or any number of other issues. One person in a specific role (even if they're not very good at it) is far easier for most organizations to deal with than the 10x generalist that can get things done. It can work well in very early startups, but there's limited reward there for it to be worth it.
- DeathArrow 4y ago
  >those Brent-style types Which Brent are you talking about?
  - TedDoesntTalk 4y ago
    I get the sense the name is being used in a general way like a “Karen”, but I’ve never heard of a “Brent”.
    - mwarkentin 4y ago
      From The Phoenix Project: https://allengeer.com/how-to-handle-brent-in-the-phoenix-project/ https://allengeer.com/how-to-handle-brent-in-the-phoenix-pro...

manigandham 4y ago

Well said. Your first 3 points summarize the entirety of the problem. Kubernetes is basically best practices (with some quirks) of running containers reliably across a cluster of machines with various deployment types, logging, DNS/discovery, networking, storage, load-balancing, high-availability and other features built-in. If you don't need any of that then don't use it, but if you do then you probably won't build a custom infrastructure that does it better.

bazhova 4y ago

The last company I worked for was essentially finance bros who had a no-code investment solution but wanted to sprinkle ML on top to get clients. Suddenly it needed to be able to run air-gapped on prem. Oh also on Ali cloud in china. Oh also on GCP and AWS hybrid. Business promised the clients it was ready before we even started building. 90% of the team was under 25. We tried our damned hardest. Used K8s to make the whole thing platform agnostic. It worked but it cost a lot. Business people are the worst.

BossingAround 4y ago
Sounds like the exact use case for OpenShift.
- nickstinemates 4y ago
  i dont think there's any use case for openshift
  - throwaway787544 4y ago
    use case: "I would like to use Kubernetes but I also would like a monkey to occasionally come out of a cage and hit my balls with a hammer"
- systemBuilder 4y ago
  Open shift : an unstable trailing-edge version of Kubernetes with too much security to the point of wasting everybody's time. Or so says my team which is still recovering from that unmitigated disaster ...
evoxmusic 4y ago
Sounds like the are looking for Qovery
dx034 4y ago
Isn't a single binary the best fit in this case? Telling clients they need to build up a kubernetes cluster (especially in finance) won't always be the easiest decision.

goodpoint 4y ago

> if your people aren't very skilled, they won't build anything well If your people aren't very skilled, making them manage a ton of unnecessary complexity is not a good idea. k8s is not a saw. It's a hovercraft. Expensive to maintain and very few people need one.

danjac 4y ago

The problem I keep seeing in startups is that a lot of startup tech founders and employees don't see a startup as an embryonic business that will grow over the long term into a money-making venture requiring calculated risks and priorities, but rather as a personal playground to do all the fun things they were not allowed to do at their old BigCorp. Use cool framework or language X? Use Kubernetes? Microservices? Lambdas? No-SQL? Two fingers to BigCorp, I'm CTO/tech employee #1 at a new startup and I get to do whatever I want. If and when it goes down in flames, I now have these skills on my resume I can leverage into a better opportunity. Ideally your first tech employee/CTO should be a seasoned hand who has been burned before, who keeps an open mind to new approaches but knows when and how to adopt it. But it's quite rare to have seasoned developers in such startups, they're not attracted by the risk and chaos of these environments as they have better options and startups generally can't afford them.

dsr_ 4y ago
Given the failure rate of startups, "I now have these skills on my resume I can leverage into a better opportunity" may be the actual goal of most employees.
- abirch 4y ago
  I've seen this frequently in large companies as well. That's why you see legacy systems with random tech.
mooreds 4y ago
> Use cool framework or language X? Use Kubernetes? Microservices? Lambdas? No-SQL? Two fingers to BigCorp, I'm CTO/tech employee #1 at a new startup and I get to do whatever I want. If and when it goes down in flames, I now have these skills on my resume I can leverage into a better opportunity. I was building a SaaS application (as a tech co-founder) and chose rails. It was newish to me (I'd done 1-2 projects) and I admit, I wanted to get better at rails. But it was also, after assessing the options and my (limited) knowledge of the future, the tech with the best chance of helping the company succeed. I have definitely run into startups where there was overly complicated technology, but tended to be later stage and have raised some money. As a founder/employee #1, I have a hard time believing anyone would pick tech just to burnish their resume; they should be trying to ship so the company succeeds. What am I missing?

pojzon 4y ago

TL;DR: - if you want to build something yourself -> you need experienced ppl to do that - if you dont have the right ppl -> you better pick a SaaS or PaaS, but your business will be at mercy of your provider - there is a huge shortage of experienced skilled ppl

mberning 4y ago

This is so spot on. Developers and “software engineers” seem to have a pathological habit of wanting to bring in gee-whiz technologies that they only have a cursory knowledge of and actually do very little to make the products and projects better.

mmcnl 4y ago

I like what you're saying, but your woodworking metaphor falls a bit flat for me because it teaches me more about woodworking than about building IT infrastructure.

osigurdson 4y ago

As intoxicating as they are, we need to move away from using analogies to problem domains most know little about to attempt to gain further insight into our own.

closeparen 4y ago

> a software engineer knows as much about cloud architecture as a fine woodworker knows about framing A mobile or frontend engineer, perhaps. But the idea that implementing and architecting the same type of object (backend services) are mutually exclusive skillsets strikes me as ridiculous. Since architecture decisions are by definition highly consequential and difficult to back out of, you want your most skilled and experienced software engineers making them, but it is precisely their software engineering experience that makes them qualified to do architecture. I would never trust a so-called "cloud architect" who is not also a first-rate backend engineer.

throwaway787544 4y ago
The thing is, Architect is its own role and discipline, and Cloud Architecture is distinct from Software Architecture. For example, you have to know why using PrivateLink is a bad idea, what kinds of backups to what locations/accounts you can do with an encrypted RDS database with a CMK, the 50 different ways you can screw up access to an S3 object, the limits of how many IPs EKS can use per node, why somebody should use the M5dn instance type, when rolling+maintaining your own complicated expensive service might be way better than using a cloud-managed version, what services you cannot use in what regions and zones. You have to be a budget wizard, keep up on the latest solutions, and know how all this dovetails with cloud security. A Software Architect should not have all of that Cloud trivia rattling around in their brain, but it's essential for a great Cloud Architect. Conversely, a Cloud Architect doesn't need to be a great coder to design great Cloud architecture. (Similar to how a construction architect doesn't need to be a great framer to know how the code affects framing, and how a great framer doesn't need to know how to calculate bending moments and shear forces in laminated beams to plan what day they'll need a lift for the beam and how to attach it)