6 ms·

I had a similar experience when I tried to use kubernetes in my home lab. Crashed it twice and since the technology and tools evolve so fast almost all tutorial

I had a similar experience when I tried to use kubernetes in my home lab. Crashed it twice and since the technology and tools evolve so fast almost all tutorials I found were already so out of date that the commands didn't work and some tools were already deprecated. I moved to docker swarm and love it. It's so much easier, straight forward, automatic ingress network and failover were all working out of the box. I'll stay with swarm for now

GauntletWizard 3y ago
Do you go to the hot new restaurant in town on their grand opening, and then leave a review that says "the food was great, but it was too crowded and the service was slow", and never go there again? That sounds like what you did there.
- globuous 3y ago
  Is that not what you’d do? That sounds like an unusually useful review tbh.
  - starttoaster 3y ago
    The point of their comment, I think, is that a restaurant's grand opening is not always indicative of future experiences. The cooks/chefs, and front of house staff are all fairly new to the environment, menus might change as they find out that certain menu items are throwing off their timing or they don't have a great way to procure certain ingredients, and it will take time to all settle into a routine that works. I think they were saying that is akin to the state of kubernetes, where everything is too new to really judge it today. Let me make clear my bias before I get to my opinion: I'm a big fan of what kubernetes has accomplished with managing the deployment of applications. But I think comparing current day kubernetes to a restaurant's grand opening is naïve. Kubernetes is amazing at what it does, and for how highly flexible it is. But it is highly flexible because of how advanced it has become, which in turn requires more knowledgeable administrators. You can take off with Docker Swarm because... it's very simple. It doesn't aim to be flexible, it just aims to be the docker-compose of distributed systems. Which is fine if you don't need the flexibility that kubernetes offers anyway. And while I do appreciate kubernetes for all of its strengths, going back towards the point of this OP, I do believe it's a bit of a security nightmare. For example, the kubernetes operator pattern often just requires this application to run in your cluster with a ClusterRole granting it read and potentially write levels of access to ConfigMaps and Secrets around your cluster. Often without much in the ways of a method of telling the kubernetes API server what namespaces it is allowed to view those objects in. Worth mentioning though that secrets management is something that Docker Swarm also handles fairly poorly, and honestly isn't much of a security regression from the normal "deploy a VM that has all your decrypted secrets sitting on the system" anyway.
- fatfingerd 3y ago
  The description sounds a lot more like a trendy restaurant where no one can figure out what is actually on the current menu. That's more of a permanent complaint than an opening night one.
  - radicalbyte 3y ago
    Menus replaced by QR codes. No wifi and poor mobile internet.
- Uehreka 3y ago
  Kubernetes is not the hot new restaurant in town, they’ve been around for years. If they’re still in a period of rapid churn that’s concerning.
- rhuru 3y ago
  Personally I think it is more like, I wanted to eat some food so I went to the meat packing factory, hence disappointed I went to a drive through and bought burger.
candiddevmike 3y ago
What did you learn by taking the easy route with Docker Swarm in your home lab? What did you set out to do with Kubernetes to begin with?
- ramraj07 3y ago
  > What did you set out to do with Kubernetes to begin with? Get some actual servers up and running to do real work vs getting certified on how to get k8s running.
LargeDiggerNick 3y ago
I read swarm doesn't have autoscaling but that was a while ago. Did that change?
- dijksterhuis 3y ago
  Nope. no change.
- AlexITC 3y ago
  Which should be good enough for a home lab as well as many prod apps. Many times, simplicity is key and occasional downtime can be accepted.
- dang 3y ago
  You can't use trollish usernames on HN so I've banned the account. You're welcome to email hn@ycombinator.com if you want to use the site as intended in the future. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
KronisLV 3y ago
> I moved to docker swarm and love it. It's so much easier, straight forward, automatic ingress network and failover were all working out of the box. I'll stay with swarm for now. I've had decent luck in the past with the K3s distribution, which is a bit cut down (but certified) Kubernetes: https://k3s.io/ https://k3s.io/ Others might also mention K0s, MicroK8s or others - there's lots of options there. It also integrates nicely with Portainer (aside from occasional Traefik ingress weirdness sometimes), which I already use for Swarm and would suggest to anyone that wants a nice web based UI for either of the technologies: https://www.portainer.io/ https://www.portainer.io/ But even so, I still run Docker Swarm for most of my private stuff as well and it's a breeze. For my needs, it has just the right amount of abstractions: stacks with services and replicas that use networks and can have some storage in the form of volumes or bind mounts. Configuration in the form of environment variables and/or mounted files (or secrets), some deployment constraints and dependencies sometimes, some health checks and restart policies, as well as resource limits. If I need a mail server, then I just have a container that binds to the ports (even low port numbers) that I need and configure it. If I need a web server, then I can just run Apache/Nginx/Caddy and use more or less 1:1 configuration files that I'd use when setting up either outside of containers, but with the added benefit of being able to refer to other apps by their service names (or aliases, if they have underscores in the names, which sometimes isn't liked). At a certain scale, it's dead simple to use - no need for PVs and PVCs, no need for Ingress and Service abstractions, or lots and lots of templating that Helm charts would have (although those are nice in other ways). Unfortunately, Swarm isn't awfully good for job prospects, which I have to admit. Then again, one can sort of say the same about Hashicorp Nomad, yet it's also alive and kicking.
- mulmen 3y ago
  Kubernetes should be called Kafka but the name was already taken. It feels deliberately complicated. I’m honestly curious about Kubernetes and I want to learn more but wow, it really is hard to grok. What does it mean that k3s is “certified”? Should I care? What’s the tradeoff? When you mentioned Traefik my eyes glazed over. Is there a picture of how all this fits together? One not created by a consultant selling their services?
jrochkind1 3y ago
I'm not following what similarity you see between your experience and anything discussed in the article. Did you get hacked?
- fatfingerd 3y ago
  I would hope that people who ended up with unsecured endpoints also didn't feel like they understood it and it was the product for them.

GauntletWizard 3y ago

Do you go to the hot new restaurant in town on their grand opening, and then leave a review that says "the food was great, but it was too crowded and the service was slow", and never go there again? That sounds like what you did there.

globuous 3y ago
Is that not what you’d do? That sounds like an unusually useful review tbh.
- starttoaster 3y ago
  The point of their comment, I think, is that a restaurant's grand opening is not always indicative of future experiences. The cooks/chefs, and front of house staff are all fairly new to the environment, menus might change as they find out that certain menu items are throwing off their timing or they don't have a great way to procure certain ingredients, and it will take time to all settle into a routine that works. I think they were saying that is akin to the state of kubernetes, where everything is too new to really judge it today. Let me make clear my bias before I get to my opinion: I'm a big fan of what kubernetes has accomplished with managing the deployment of applications. But I think comparing current day kubernetes to a restaurant's grand opening is naïve. Kubernetes is amazing at what it does, and for how highly flexible it is. But it is highly flexible because of how advanced it has become, which in turn requires more knowledgeable administrators. You can take off with Docker Swarm because... it's very simple. It doesn't aim to be flexible, it just aims to be the docker-compose of distributed systems. Which is fine if you don't need the flexibility that kubernetes offers anyway. And while I do appreciate kubernetes for all of its strengths, going back towards the point of this OP, I do believe it's a bit of a security nightmare. For example, the kubernetes operator pattern often just requires this application to run in your cluster with a ClusterRole granting it read and potentially write levels of access to ConfigMaps and Secrets around your cluster. Often without much in the ways of a method of telling the kubernetes API server what namespaces it is allowed to view those objects in. Worth mentioning though that secrets management is something that Docker Swarm also handles fairly poorly, and honestly isn't much of a security regression from the normal "deploy a VM that has all your decrypted secrets sitting on the system" anyway.
fatfingerd 3y ago
The description sounds a lot more like a trendy restaurant where no one can figure out what is actually on the current menu. That's more of a permanent complaint than an opening night one.
- radicalbyte 3y ago
  Menus replaced by QR codes. No wifi and poor mobile internet.
Uehreka 3y ago
Kubernetes is not the hot new restaurant in town, they’ve been around for years. If they’re still in a period of rapid churn that’s concerning.
rhuru 3y ago
Personally I think it is more like, I wanted to eat some food so I went to the meat packing factory, hence disappointed I went to a drive through and bought burger.

candiddevmike 3y ago

What did you learn by taking the easy route with Docker Swarm in your home lab? What did you set out to do with Kubernetes to begin with?

ramraj07 3y ago
> What did you set out to do with Kubernetes to begin with? Get some actual servers up and running to do real work vs getting certified on how to get k8s running.

LargeDiggerNick 3y ago

I read swarm doesn't have autoscaling but that was a while ago. Did that change?

dijksterhuis 3y ago
Nope. no change.
AlexITC 3y ago
Which should be good enough for a home lab as well as many prod apps. Many times, simplicity is key and occasional downtime can be accepted.
dang 3y ago
You can't use trollish usernames on HN so I've banned the account. You're welcome to email hn@ycombinator.com if you want to use the site as intended in the future. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html

KronisLV 3y ago

> I moved to docker swarm and love it. It's so much easier, straight forward, automatic ingress network and failover were all working out of the box. I'll stay with swarm for now. I've had decent luck in the past with the K3s distribution, which is a bit cut down (but certified) Kubernetes: https://k3s.io/ https://k3s.io/ Others might also mention K0s, MicroK8s or others - there's lots of options there. It also integrates nicely with Portainer (aside from occasional Traefik ingress weirdness sometimes), which I already use for Swarm and would suggest to anyone that wants a nice web based UI for either of the technologies: https://www.portainer.io/ https://www.portainer.io/ But even so, I still run Docker Swarm for most of my private stuff as well and it's a breeze. For my needs, it has just the right amount of abstractions: stacks with services and replicas that use networks and can have some storage in the form of volumes or bind mounts. Configuration in the form of environment variables and/or mounted files (or secrets), some deployment constraints and dependencies sometimes, some health checks and restart policies, as well as resource limits. If I need a mail server, then I just have a container that binds to the ports (even low port numbers) that I need and configure it. If I need a web server, then I can just run Apache/Nginx/Caddy and use more or less 1:1 configuration files that I'd use when setting up either outside of containers, but with the added benefit of being able to refer to other apps by their service names (or aliases, if they have underscores in the names, which sometimes isn't liked). At a certain scale, it's dead simple to use - no need for PVs and PVCs, no need for Ingress and Service abstractions, or lots and lots of templating that Helm charts would have (although those are nice in other ways). Unfortunately, Swarm isn't awfully good for job prospects, which I have to admit. Then again, one can sort of say the same about Hashicorp Nomad, yet it's also alive and kicking.

mulmen 3y ago
Kubernetes should be called Kafka but the name was already taken. It feels deliberately complicated. I’m honestly curious about Kubernetes and I want to learn more but wow, it really is hard to grok. What does it mean that k3s is “certified”? Should I care? What’s the tradeoff? When you mentioned Traefik my eyes glazed over. Is there a picture of how all this fits together? One not created by a consultant selling their services?

jrochkind1 3y ago

I'm not following what similarity you see between your experience and anything discussed in the article. Did you get hacked?

fatfingerd 3y ago
I would hope that people who ended up with unsecured endpoints also didn't feel like they understood it and it was the product for them.