5 ms·

"Why piece all of that functionality together yourself into some fragile framework instead of using a industry standard?" Quite recently developed "industry st

"Why piece all of that functionality together yourself into some fragile framework instead of using a industry standard?" Quite recently developed "industry standard". Many tools mentioned have been used for tens of years, they work robustly, are well documented and there is lots of people who can use them. I personally would use the word "industry standard" a little bit differently.

cameronbrown 6y ago
> Quite recently developed "industry standard". Many tools mentioned have been used for tens of years, they work robustly, are well documented and there is lots of people who can use them. k8s is based off of Borg, which has existed for far longer. https://research.google/pubs/pub43438/ https://research.google/pubs/pub43438/
- vertex-four 6y ago
  "What Google does" is not an industry standard.
  - cameronbrown 6y ago
    That's not what I'm saying. Your point was about the maturity of k8s which makes sense if it sprung from nowhere, but k8s encapsulates a lot of "lessons learned" of a very stable and mature product (even if proprietary).
- skj 6y ago
  Really it's just "inspired" by borg. Using both, they're really not very similar.
manigandham 6y ago
You still have to put them all together into some custom solution just for your setup which adds overhead and fragility. New employees will have to learn that instead of using K8S APIs. Deploying new components can’t take advantage of the wide and fast growing ecosystem. There’s really nothing like the full suite that kubernetes provides.
- ethbro 6y ago
  > New employees will have to learn that instead of using K8S APIs You know tech has made it when people offer argumentum ad Java. 'Sure, X might not be the best solution, but you can always buy someone to do it.'
  - dividedbyzero 6y ago
    It's also very empowering for people whose job isn't to run things, but to build the things that are to run. I've worked on a dozen-or-so of bespoke "industry standard" setups, and each and every one had a number of weird quirks, involved learning some new "industry standard" components and either made it very hard and dangerous for non-ops/devops/infra people to run their things themselves, or had homegrown tooling that pretty much replicated what the k8s API can do, just only a small subset and badly. Some YAML and kubectl are well within what a typical data scientist can be expected to understand, more so if it means they can run their things on a dev cluster themselves, and in a pinch, that data scientist can debug prod issues of their things, because it all works the same way. We have a very useful bot that was built and deployed by someone decidedly not ops while waiting for jobs to finish – a simple K8S deploymend YAML is like 50 LOC, with 40 being pretty much standard, et voilà, running bot, without having to build lots and lots of automation in-house or having to take up ops time to deploy this for them or having to grok advanced sysadmin-ing first. Used with appropriate caution and safeguards, it's super powerful.

cameronbrown 6y ago

> Quite recently developed "industry standard". Many tools mentioned have been used for tens of years, they work robustly, are well documented and there is lots of people who can use them. k8s is based off of Borg, which has existed for far longer. https://research.google/pubs/pub43438/ https://research.google/pubs/pub43438/

vertex-four 6y ago
"What Google does" is not an industry standard.
- cameronbrown 6y ago
  That's not what I'm saying. Your point was about the maturity of k8s which makes sense if it sprung from nowhere, but k8s encapsulates a lot of "lessons learned" of a very stable and mature product (even if proprietary).
skj 6y ago
Really it's just "inspired" by borg. Using both, they're really not very similar.

manigandham 6y ago

You still have to put them all together into some custom solution just for your setup which adds overhead and fragility. New employees will have to learn that instead of using K8S APIs. Deploying new components can’t take advantage of the wide and fast growing ecosystem. There’s really nothing like the full suite that kubernetes provides.

ethbro 6y ago
> New employees will have to learn that instead of using K8S APIs You know tech has made it when people offer argumentum ad Java. 'Sure, X might not be the best solution, but you can always buy someone to do it.'
- dividedbyzero 6y ago
  It's also very empowering for people whose job isn't to run things, but to build the things that are to run. I've worked on a dozen-or-so of bespoke "industry standard" setups, and each and every one had a number of weird quirks, involved learning some new "industry standard" components and either made it very hard and dangerous for non-ops/devops/infra people to run their things themselves, or had homegrown tooling that pretty much replicated what the k8s API can do, just only a small subset and badly. Some YAML and kubectl are well within what a typical data scientist can be expected to understand, more so if it means they can run their things on a dev cluster themselves, and in a pinch, that data scientist can debug prod issues of their things, because it all works the same way. We have a very useful bot that was built and deployed by someone decidedly not ops while waiting for jobs to finish – a simple K8S deploymend YAML is like 50 LOC, with 40 being pretty much standard, et voilà, running bot, without having to build lots and lots of automation in-house or having to take up ops time to deploy this for them or having to grok advanced sysadmin-ing first. Used with appropriate caution and safeguards, it's super powerful.