4 ms·

I feel like Kubernetes is always randomly mentioned in rants like this. Instead of saying your hardened VM has Docker you could have just said it has kubelet on

I feel like Kubernetes is always randomly mentioned in rants like this. Instead of saying your hardened VM has Docker you could have just said it has kubelet on it. Then instead of a bunch of ad hoc "docker services" you could pay pennies for a k8s control plane that gives you control over everything on those VMs. I fail to see how your way is anything but worse. The bad cloud infrastructure is when people try to use every single thing AWS sells and their whole infrastructure is at super high levels of abstraction that they could never migrate to another platform. K8s isn't that at all.

ownagefool 2y ago
In think in either case, if you already have code that's done, using that is going to be less effort than switching. However, I ran kubeadm on a hetzner server and it's just sat chugging along forever basically. I use the cluster to run ephemeral apps where I build and deploy 1 golang service, a couple of node services in about 60 seconds ( with cache, obviously ). As someone old enough and skilled enough to do the same with puppet, why bother when it's simpler easier that even the kids who don't understand TLS can do it with k8s?
zepolen 2y ago
100% best comment in this thread. With k8s you get a way of saying 'WHAT YOU WANT' without 'HOW TO DO IT', and this is applies not only to the actual infra aspect, but the people maintaining it too. Any cloud platform and devops worth their salt can maintain a k8s system. Good luck finding someone to understand what that 'custom Naemon' plugin is doing.
- ghomem 2y ago
  > Good luck finding someone to understand what that 'custom Naemon' plugin is doing. You Kubernetes people get triggered very easily. I was already lucky to have found several juniors that worked in this kind of thing with minimal training. The 'custom Naemon plugin' is 30 lines of bash and you can adapt it to any monitoring system. Of course this is scary and complicated. I might consider switching to 'Kubernetes operators', which sounds simpler :-)
  - globular-toast 2y ago
    I've done all of this and then some. I used to deploy websites by FTPing into the server and copying files. Then it was bash scripts, then Ansible. IMO Kubernetes hits a very good level of abstraction. You can totally deploy 30 lines of bash to every server, you just have to wrap it in a docker container. That's all k8s asks for for a workload. You don't have to use operators. That would be something to explore much later. Honestly I just think you should be more generous and not assume people have created this stuff just for fun. K8s really does address real problems around deployment and it's very well thought out.
    - ghomem 2y ago
      > Honestly I just think you should be more generous I am generous in the context for generosity. Turns out that engineering is not about being generous but rather about choosing the most efficient solution for problems that in the end need to be business driven. This requires evaluating requirements, context and tradeoffs. That takes a cold, rational mind more than generosity. > K8s really does address real problems around deployment and it's very well thought out It's great where it makes sense. It's less than great elsewhere. Not everything is SaaS, not everything needs scaling, not everything needs 99.99% of uptime, not everything needs a CDN, not every company is VC backed operating at high risk / high reward, etc, etc. Context is better than ideology. If you read the article I posted you will see that stated clearly.
      - globular-toast 2y ago
        I completely agree that most people don't need that. This is always what people say when k8s comes up. This is also what people said about git 15 years ago (you're not the kernel etc). But the thing is you don't have to use any of the bits you don't need. At first I listened to the naysayers and was wary of k8s thinking it would create more problems than it solves. That simply hasn't been the case. It's not a no-brainer, there are tradeoffs, but I really think it makes sense especially if you're doing docker anyway. Like I said in another comment, people tend to talk about two different things. There's k8s which can be as little as just a single node k3s server which is basically docker compose with a few extras like automatic rollout etc. Then there's the over the top "cloud native" stuff. One does not imply the other.
karmarepellent 2y ago
Unfortunately in air-gapped systems you cannot simply pay pennies for a managed k8s platform. In these cases you have to bootstrap and manage k8s on your own in your data centers. While I do not think bootstrapping and managing a cluster is difficult at all (especially if you only handle stateless workloads) it may still not fit or integrate well with a companies overall management infrastructure. While I am a happy cloud infrastructure user in private, I have to go through some extra hoops to deploy applications at work, regardless of if k8s is used or not.

ownagefool 2y ago

In think in either case, if you already have code that's done, using that is going to be less effort than switching. However, I ran kubeadm on a hetzner server and it's just sat chugging along forever basically. I use the cluster to run ephemeral apps where I build and deploy 1 golang service, a couple of node services in about 60 seconds ( with cache, obviously ). As someone old enough and skilled enough to do the same with puppet, why bother when it's simpler easier that even the kids who don't understand TLS can do it with k8s?

zepolen 2y ago

100% best comment in this thread. With k8s you get a way of saying 'WHAT YOU WANT' without 'HOW TO DO IT', and this is applies not only to the actual infra aspect, but the people maintaining it too. Any cloud platform and devops worth their salt can maintain a k8s system. Good luck finding someone to understand what that 'custom Naemon' plugin is doing.

ghomem 2y ago
> Good luck finding someone to understand what that 'custom Naemon' plugin is doing. You Kubernetes people get triggered very easily. I was already lucky to have found several juniors that worked in this kind of thing with minimal training. The 'custom Naemon plugin' is 30 lines of bash and you can adapt it to any monitoring system. Of course this is scary and complicated. I might consider switching to 'Kubernetes operators', which sounds simpler :-)
- globular-toast 2y ago
  I've done all of this and then some. I used to deploy websites by FTPing into the server and copying files. Then it was bash scripts, then Ansible. IMO Kubernetes hits a very good level of abstraction. You can totally deploy 30 lines of bash to every server, you just have to wrap it in a docker container. That's all k8s asks for for a workload. You don't have to use operators. That would be something to explore much later. Honestly I just think you should be more generous and not assume people have created this stuff just for fun. K8s really does address real problems around deployment and it's very well thought out.
  - ghomem 2y ago
    > Honestly I just think you should be more generous I am generous in the context for generosity. Turns out that engineering is not about being generous but rather about choosing the most efficient solution for problems that in the end need to be business driven. This requires evaluating requirements, context and tradeoffs. That takes a cold, rational mind more than generosity. > K8s really does address real problems around deployment and it's very well thought out It's great where it makes sense. It's less than great elsewhere. Not everything is SaaS, not everything needs scaling, not everything needs 99.99% of uptime, not everything needs a CDN, not every company is VC backed operating at high risk / high reward, etc, etc. Context is better than ideology. If you read the article I posted you will see that stated clearly.
    - globular-toast 2y ago
      I completely agree that most people don't need that. This is always what people say when k8s comes up. This is also what people said about git 15 years ago (you're not the kernel etc). But the thing is you don't have to use any of the bits you don't need. At first I listened to the naysayers and was wary of k8s thinking it would create more problems than it solves. That simply hasn't been the case. It's not a no-brainer, there are tradeoffs, but I really think it makes sense especially if you're doing docker anyway. Like I said in another comment, people tend to talk about two different things. There's k8s which can be as little as just a single node k3s server which is basically docker compose with a few extras like automatic rollout etc. Then there's the over the top "cloud native" stuff. One does not imply the other.

karmarepellent 2y ago

Unfortunately in air-gapped systems you cannot simply pay pennies for a managed k8s platform. In these cases you have to bootstrap and manage k8s on your own in your data centers. While I do not think bootstrapping and managing a cluster is difficult at all (especially if you only handle stateless workloads) it may still not fit or integrate well with a companies overall management infrastructure. While I am a happy cloud infrastructure user in private, I have to go through some extra hoops to deploy applications at work, regardless of if k8s is used or not.