4 ms·

You are correct. > [kubernetes is] a simplified clone of Google’s internal borg system https://medium.com/@steve.yegge/honestly-i-cant-stand-k8s-48c9a600e405

You are correct. > [kubernetes is] a simplified clone of Google’s internal borg system https://medium.com/@steve.yegge/honestly-i-cant-stand-k8s-48c9a600e405 https://medium.com/@steve.yegge/honestly-i-cant-stand-k8s-48...

outside1234 8y ago
That said, Google, to my understanding, does run a completely containerized infrastructure internally, including databases and other stateful things, so it is not wildly off to suggest running a database on Kubernetes.
- kyrra 8y ago
  Packaging was (is?) a bit different, because it existed before Docker existed. Pdf of the presentation: https://www.usenix.org/sites/default/files/conference/protected-files/lisa_2014_talk.pdf https://www.usenix.org/sites/default/files/conference/protec... And a recording of the talk: https://www.usenix.org/conference/lisa14/conference-program/presentation/mcnutt https://www.usenix.org/conference/lisa14/conference-program/...
  - puzzle 8y ago
    Googlers, including the ones working on Kubernetes, like to pick on MPM, but that's probably because they haven't run major services deployed from Docker images.
- puzzle 8y ago
  Before it ran on F1/Spanner, Adwords ran on sharded MySQL on Borg. Not for its entire early life, but for quite a few years. Later in life, Checkout and maybe Wallet ran on MySQL on Borg, too. So did YouTube, which used Vitess, a sharding layer (now ported to Kubernetes and open sourced). Bigtable, Spanner and even Colossus/D run in containers on Borg.
  - tayo42 8y ago
    How is colossus running in a container in borg? That's their network file system that makes running databases in containers possible?
    - puzzle 8y ago
      There's a bootstrapping issue, of course, but that's how it works. And it's even crazier than what you're picturing. What if I told you that Bigtable runs on top of Colossus, but Colossus itself stores metadata about files, including Bigtable's files, in... Bigtable? It's really turtles all the way down, the last of which is, luckily, Chubby. Some of the details here: http://www.pdsw.org/pdsw-discs17/slides/PDSW-DISCS-Google-Keynote.pdf http://www.pdsw.org/pdsw-discs17/slides/PDSW-DISCS-Google-Ke...
daxfohl 8y ago
Wow, I'd never heard of Fargate until this. It seems so somewhat obvious now for things that Lambda doesn't fit. Heroku for containers, not VMs.
- manigandham 8y ago
  There's also Azure Container Instances: https://azure.microsoft.com/en-us/services/container-instances/ https://azure.microsoft.com/en-us/services/container-instanc... And Google has AppEngine, now with Flexible Environment that can run containers: https://cloud.google.com/appengine/docs/flexible/dotnet/quickstart https://cloud.google.com/appengine/docs/flexible/dotnet/quic...
puzzle 8y ago
Kubernetes might be simpler than Borg in so many ways (let me count the ones I care about...), but it also has better features that Borg did not implement (labels and selectors) or that are offered only by some other internal services, which obviously are configured through very different mechanisms (ingress).

outside1234 8y ago

That said, Google, to my understanding, does run a completely containerized infrastructure internally, including databases and other stateful things, so it is not wildly off to suggest running a database on Kubernetes.

kyrra 8y ago
Packaging was (is?) a bit different, because it existed before Docker existed. Pdf of the presentation: https://www.usenix.org/sites/default/files/conference/protected-files/lisa_2014_talk.pdf https://www.usenix.org/sites/default/files/conference/protec... And a recording of the talk: https://www.usenix.org/conference/lisa14/conference-program/presentation/mcnutt https://www.usenix.org/conference/lisa14/conference-program/...
- puzzle 8y ago
  Googlers, including the ones working on Kubernetes, like to pick on MPM, but that's probably because they haven't run major services deployed from Docker images.
puzzle 8y ago
Before it ran on F1/Spanner, Adwords ran on sharded MySQL on Borg. Not for its entire early life, but for quite a few years. Later in life, Checkout and maybe Wallet ran on MySQL on Borg, too. So did YouTube, which used Vitess, a sharding layer (now ported to Kubernetes and open sourced). Bigtable, Spanner and even Colossus/D run in containers on Borg.
- tayo42 8y ago
  How is colossus running in a container in borg? That's their network file system that makes running databases in containers possible?
  - puzzle 8y ago
    There's a bootstrapping issue, of course, but that's how it works. And it's even crazier than what you're picturing. What if I told you that Bigtable runs on top of Colossus, but Colossus itself stores metadata about files, including Bigtable's files, in... Bigtable? It's really turtles all the way down, the last of which is, luckily, Chubby. Some of the details here: http://www.pdsw.org/pdsw-discs17/slides/PDSW-DISCS-Google-Keynote.pdf http://www.pdsw.org/pdsw-discs17/slides/PDSW-DISCS-Google-Ke...

daxfohl 8y ago

Wow, I'd never heard of Fargate until this. It seems so somewhat obvious now for things that Lambda doesn't fit. Heroku for containers, not VMs.

manigandham 8y ago
There's also Azure Container Instances: https://azure.microsoft.com/en-us/services/container-instances/ https://azure.microsoft.com/en-us/services/container-instanc... And Google has AppEngine, now with Flexible Environment that can run containers: https://cloud.google.com/appengine/docs/flexible/dotnet/quickstart https://cloud.google.com/appengine/docs/flexible/dotnet/quic...

puzzle 8y ago

Kubernetes might be simpler than Borg in so many ways (let me count the ones I care about...), but it also has better features that Borg did not implement (labels and selectors) or that are offered only by some other internal services, which obviously are configured through very different mechanisms (ingress).