5 ms·

I once got an interview from a company in the container space because one of their exec read an article I published talking about the trouble with container sys

I once got an interview from a company in the container space because one of their exec read an article I published talking about the trouble with container systems[1]. (Really good talk/interview, but I ended up not moving forward because I didn't want to move back to the west coast). I've been in smaller shops that wasted a lot of time on K8s stuff and fell behind on their timeline. If you want to run k8s, DC/OS, etc. you need a lot of ramp up time and at least 4 ~ 8 dedicate staff members. I've talked to other startups that preferred running Nomad instead due to setup complexity. I doubt k8s will go the way of Open Stack since it does actually work, but I do think we'll see it limited to big-end enterprise systems and smaller startups will push forward with other, easier to build up clustering technologies. [1]: https://penguindreams.org/blog/my-love-hate-relationship-with-docker-and-container-orchestration-systems/ https://penguindreams.org/blog/my-love-hate-relationship-wit...

segmondy 8y ago
4-8 dedicate staff members to run k8s? seriously, How did you come up with that number? 1) You can run k8s hosted on Google, DigitalOcean with zero effort. 2) I built a k8s cluster in 3 days with zero experience after spending a week playing with minikube, reading the docs are kuberentes.io
- Already__Taken 8y ago
  > I built a k8s cluster in 3 days with zero experience after spending a week playing with minikube, reading the docs are kuberentes.io I'm pretty sure I can do it in an afternoon from scripts on github. But if something goes wrong all bets are off. Just getting something setup is not building a competence around it.
  - vkou 8y ago
    How well would you, with zero experience managing and deploying a stack, do the same in an orthodox LAMP setting?
    - yebyen 8y ago
      Honestly, the answer is only relevant if you actually have zero experience deploying any stack, or orthodox LAMP stack. If you have no experience with (stack Z), then you will have to go out and get some experience before opting to use (stack Z). The problem is, many people hear this and stop there. While there are some barriers to experience and production-readiness, they are not insurmountable, and there may be a pot of gold at the end of the rainbow. There is a cost for everything. Sometimes it's an opportunity cost. (Sometimes the cost can also come from not acting.)
- sk5t 8y ago
  And you feel like you understand the warts and pitfalls, suitable to keeping a mission-critical service running once the circuit breakers start to trip, based on 3-8 days of experimentation?
  - cat199 8y ago
    well, not the op, but i googled 'kindly pls to fix a kubernetes software on the RHEL servers' and I got this in a forum post: $ sudo apt-get install fix-muh-k8s && sudo fix-muh-k8s || reboot so I think i'm good to go, thankyouverymuch.
- bacongobbler 8y ago
  Firing up minikube to toy with Kubernetes and switching your entire software stack to run on top of a production-grade Kubernetes cluster are two very different things.
- cat199 8y ago
  built != run. not sure if I agree with OP on precise figures, but, if you're dealing with: - actually federated k8 infrastructure (distributed etcd, etc) - enough system load that k8 is actually worthwhile and not 'cool' - user / dev requests - whatever background IT projects are going on (updates / new nodes, existing upgrades, testing new configs, etc) and you want: - 24x7 operations (or even stringent 8x5) - people to be able to use the restroom and take vacations and still have 1-2 sets of hands available 1 person doesn't quite cut it, because (s)he will quit eventually, and you will have nooone. dev-centric mentalities of 'it compiled, ship it, i made a new futuristic fancy feature aren't i amazeballs' often don't take into account real operational concerns, which i think is precisely the point in critiquing k8s here
lotyrin 8y ago
I haven't been following OpenStack closely enough to understand the way you feel that it's gone or what I assume is an implication that it failed to work (in which sense?).
monksy 8y ago
Having installed Openstack many times before, often having issues with it running: > t k8s will go the way of Open Stack since it does actually work, _ohh shit__!
parthdesai 8y ago
My friend is trying his luck with his own start up, they have yet to launch a 1.0 of product but the CTO implemented K8s citing scalability. I legit laughed at that statement.
- shaklee3 8y ago
  My friend has a startup and a single person got a k8s cluster running on aws with kops+GPUs in a couple weeks. He loves it. The people who are running it successfully don't come on here to complain.
  - pritambaral 8y ago
    > a single person got a k8s cluster running on aws with kops+GPUs in a couple weeks In the context of your GP's comment: 1. How long would it have taken that one person (or the startup) to get started with AWS+GPUs without k8s? 2. How much effort would it take that person to debug an issue with if/when one springs up? 3. What happens when the single person goes on leave?
    - shaklee3 8y ago
      Probably a lot longer. Do you know of any other orchestrator that supported GPU scheduling natively 6 months ago?
  - parthdesai 8y ago
    The point is they have been coding away for more than a year without having getting out of beta. The CTO spent 3 weeks learning k8s and setting it up, those 3 weeks which could have been spent on features or ironing out bugs in the product and releasing an actual 1.0 of the product. You can always work on k8s once you have paying customers and have actual demand. Are you telling me you need k8s for not even 10 customers in beta?
    - shaklee3 8y ago
      For some use cases (GPU) it's considerably less expensive to spend the time learning kubernetes.

segmondy 8y ago

4-8 dedicate staff members to run k8s? seriously, How did you come up with that number? 1) You can run k8s hosted on Google, DigitalOcean with zero effort. 2) I built a k8s cluster in 3 days with zero experience after spending a week playing with minikube, reading the docs are kuberentes.io

Already__Taken 8y ago
> I built a k8s cluster in 3 days with zero experience after spending a week playing with minikube, reading the docs are kuberentes.io I'm pretty sure I can do it in an afternoon from scripts on github. But if something goes wrong all bets are off. Just getting something setup is not building a competence around it.
- vkou 8y ago
  How well would you, with zero experience managing and deploying a stack, do the same in an orthodox LAMP setting?
  - yebyen 8y ago
    Honestly, the answer is only relevant if you actually have zero experience deploying any stack, or orthodox LAMP stack. If you have no experience with (stack Z), then you will have to go out and get some experience before opting to use (stack Z). The problem is, many people hear this and stop there. While there are some barriers to experience and production-readiness, they are not insurmountable, and there may be a pot of gold at the end of the rainbow. There is a cost for everything. Sometimes it's an opportunity cost. (Sometimes the cost can also come from not acting.)
sk5t 8y ago
And you feel like you understand the warts and pitfalls, suitable to keeping a mission-critical service running once the circuit breakers start to trip, based on 3-8 days of experimentation?
- cat199 8y ago
  well, not the op, but i googled 'kindly pls to fix a kubernetes software on the RHEL servers' and I got this in a forum post: $ sudo apt-get install fix-muh-k8s && sudo fix-muh-k8s || reboot so I think i'm good to go, thankyouverymuch.
bacongobbler 8y ago
Firing up minikube to toy with Kubernetes and switching your entire software stack to run on top of a production-grade Kubernetes cluster are two very different things.
cat199 8y ago
built != run. not sure if I agree with OP on precise figures, but, if you're dealing with: - actually federated k8 infrastructure (distributed etcd, etc) - enough system load that k8 is actually worthwhile and not 'cool' - user / dev requests - whatever background IT projects are going on (updates / new nodes, existing upgrades, testing new configs, etc) and you want: - 24x7 operations (or even stringent 8x5) - people to be able to use the restroom and take vacations and still have 1-2 sets of hands available 1 person doesn't quite cut it, because (s)he will quit eventually, and you will have nooone. dev-centric mentalities of 'it compiled, ship it, i made a new futuristic fancy feature aren't i amazeballs' often don't take into account real operational concerns, which i think is precisely the point in critiquing k8s here

lotyrin 8y ago

I haven't been following OpenStack closely enough to understand the way you feel that it's gone or what I assume is an implication that it failed to work (in which sense?).

monksy 8y ago

Having installed Openstack many times before, often having issues with it running: > t k8s will go the way of Open Stack since it does actually work, _ohh shit__!

parthdesai 8y ago

My friend is trying his luck with his own start up, they have yet to launch a 1.0 of product but the CTO implemented K8s citing scalability. I legit laughed at that statement.

shaklee3 8y ago
My friend has a startup and a single person got a k8s cluster running on aws with kops+GPUs in a couple weeks. He loves it. The people who are running it successfully don't come on here to complain.
- pritambaral 8y ago
  > a single person got a k8s cluster running on aws with kops+GPUs in a couple weeks In the context of your GP's comment: 1. How long would it have taken that one person (or the startup) to get started with AWS+GPUs without k8s? 2. How much effort would it take that person to debug an issue with if/when one springs up? 3. What happens when the single person goes on leave?
  - shaklee3 8y ago
    Probably a lot longer. Do you know of any other orchestrator that supported GPU scheduling natively 6 months ago?
- parthdesai 8y ago
  The point is they have been coding away for more than a year without having getting out of beta. The CTO spent 3 weeks learning k8s and setting it up, those 3 weeks which could have been spent on features or ironing out bugs in the product and releasing an actual 1.0 of the product. You can always work on k8s once you have paying customers and have actual demand. Are you telling me you need k8s for not even 10 customers in beta?
  - shaklee3 8y ago
    For some use cases (GPU) it's considerably less expensive to spend the time learning kubernetes.