3 ms·
The borg concepts and interfaces have been the same for ages. The borglet and borgmaster get released and pushed out very frequently (daily or weekly) and it do
by jeffbee 5y ago
The borg concepts and interfaces have been the same for ages. The borglet and borgmaster get released and pushed out very frequently (daily or weekly) and it doesn't break anything among the workloads. There are not maintenance windows for these changes because containers can run while the borglet restarts, and borgmasters are protocol/format-compatible so the new release binary simply joins the quorum. Machines also get rotated out of the borg cell regularly for kernel upgrades, way more often than I've seen outside of Google.
I think an important thing to know about K8s is Omega failed to replace Borg, and then the Omega people created K8s. So K8s does not necessarily descend from Borg, and not all of Borg's desirable attributes made it into K8s.
- dekhn 5y agoThe omega folks were on the borgmaster and borglet teams when they were building omega (I was on the borg team at the same time, but working on a different project). It's fair to say that k8s is an intellectual inheritor of the parts of borg that are required for it to be useful on the outside. Borglet/borgmaster releases definitely break workloads. I recall one where something was rolled out to all the test clusters, passed all the tests (except one of ours) and was about to be promoted to increasing percents of the prod clusters. Whilst debugging why our test (which is not part of the feature rollout) broke we realize that if this had rolled out to prod, it would have broken all Tensorflow jobs, and would have been a major OMG. But yeah, most of the time, the release process for borglet and borgmaster is fairly fast and fairly reliable.
- jeffbee 5y ago> required for it to be useful on the outside. A big part of this is that outside Google, the number of people who have to operate k8s as a fraction of all k8s users is way higher than the fraction of borg users who have to operate borg, so there's a lot of stuff in k8s that is 'end user experience' comforts and affordances for operators.