3 ms·

I think it still bears repeating.. I've seen projects fail or take 10x as long due to choosing to go with k8s day 1 instead of starting with even a basic VM.

I think it still bears repeating.. I've seen projects fail or take 10x as long due to choosing to go with k8s day 1 instead of starting with even a basic VM. It may be easy enough to get k8s itself setup. However, without an in-house expert sat within the dev team, all the access/permissions to resources to/from the cluster were hell. NFS disk Read/Write, S3 Read/Write/List, RDS DB access, Kafka access, corp network access to web server within k8s.. and then all the daemons you need to actually be able to monitor your logs/health of apps/etc. Each of these things in isolation would be no big deal, except that they'd take a different subset of 10 different infra/devops team members 1-2 weeks to sort the first time, in some fragile manner that frequently fell over. All of these for an application that was replacing something basically owned by 1 dude as 10% of his job, running on a 10 year old server under cron, with a big disk attached.

mikkergp 4y ago
It’s not like vms have magical simplicity guardrails. I run all my personal stuff in K8s because it’s what I know and it’s all pretty set it and forget it. If it really is that simple, or your comfortable with Ansible great! but I’ve seen lots of issues on VMs with mountains of custom deploy scripts, or OS’ that can’t be upgraded, or manual configurations people don’t understand.
- rfoo 4y ago
  > OS’ that can’t be upgraded Well, with Kubernetes this became container images versions and operator versions that can't be upgraded. Though I agree with your in general.
  - mikkergp 4y ago
    It's true. And Kubernetes is not the magic sauce that fixes all problems either. It _can_ be a pretty efficient problem generator itself :-)
zinclozenge 4y ago
Don't you have to do all of this stuff even if you're not running k8s? Or are you complaining about a re-write/replacement of something that didn't need to be?
- steveBK123 4y ago
  As a k8s novice, it seems apparent to me that access to physical resources from pods requires more levels of access configuration than bare metal having access to the same physical resources. For example NFS you need to make sure the underlying hardware the k8s is running on has the mount, and then that the pods within have that access. Whatever host level config you need to do on the VM needs to be basically replicated within each of your container images for the pods you want to run, no? Then you have the same update cycle except across N container images instead of 1 VM?
tomtheelder 4y ago
This sounds like projects struggling because they chose to go with k8s when the team lacked k8s expertise. Honestly I feel like the original blog post shouldn't be talking about k8s at all. The real lesson, as far as I can tell, is just "when you're starting out, choose simple tech that you already know how to use." If you already know how to use Kubernetes, then feel free to use it. If not then don't.

mikkergp 4y ago

It’s not like vms have magical simplicity guardrails. I run all my personal stuff in K8s because it’s what I know and it’s all pretty set it and forget it. If it really is that simple, or your comfortable with Ansible great! but I’ve seen lots of issues on VMs with mountains of custom deploy scripts, or OS’ that can’t be upgraded, or manual configurations people don’t understand.

rfoo 4y ago
> OS’ that can’t be upgraded Well, with Kubernetes this became container images versions and operator versions that can't be upgraded. Though I agree with your in general.
- mikkergp 4y ago
  It's true. And Kubernetes is not the magic sauce that fixes all problems either. It _can_ be a pretty efficient problem generator itself :-)

zinclozenge 4y ago

Don't you have to do all of this stuff even if you're not running k8s? Or are you complaining about a re-write/replacement of something that didn't need to be?

steveBK123 4y ago
As a k8s novice, it seems apparent to me that access to physical resources from pods requires more levels of access configuration than bare metal having access to the same physical resources. For example NFS you need to make sure the underlying hardware the k8s is running on has the mount, and then that the pods within have that access. Whatever host level config you need to do on the VM needs to be basically replicated within each of your container images for the pods you want to run, no? Then you have the same update cycle except across N container images instead of 1 VM?

tomtheelder 4y ago

This sounds like projects struggling because they chose to go with k8s when the team lacked k8s expertise. Honestly I feel like the original blog post shouldn't be talking about k8s at all. The real lesson, as far as I can tell, is just "when you're starting out, choose simple tech that you already know how to use." If you already know how to use Kubernetes, then feel free to use it. If not then don't.