4 ms·
> Google is still relying on Borg for its internal production infrastructure, but I can tell you it’s not because of the design of Borg interfaces! No matter h
by puzzle 8y ago
> Google is still relying on Borg for its internal production infrastructure, but I can tell you it’s not because of the design of Borg interfaces!
No matter how spartan, the Borg status pages are more helpful than most Kubernetes UIs out there when it comes to debugging a problem in depth, i.e. past CPU and memory graphs. Part of that is made possible by applications exposing debugging endpoints and telling Borg about them.
- GauntletWizard 8y agoYeah, I would kill for the k8s pod contract to have something like the borg status line.
- heavenlyblue 8y agoAs a matter of post-SLA curiousity, what is in the status line?
- jbeda 8y agoIt's something we've talked about on and off. Never seems to make it happen.
- threeseed 8y agoCan't you just use CAdvisor and route metrics to Prometheus/Grafana ?
- puzzle 8y agoThey're sometimes interactive HTTP endpoints, not metrics, although metrics are served by one of them (/varz, described in the SRE book). They're informally called z pages and are the inspiration behind the /debug handlers that Go's net/trace package installs: https://pbs.twimg.com/media/CebGW64W4AAJ_sc?format=jpg https://pbs.twimg.com/media/CebGW64W4AAJ_sc?format=jpg OpenCensus has a bunch, too: https://opencensus.io/zpages/ https://opencensus.io/zpages/
- jamesblonde 8y agoYarn apps typically expose a similar http endpoint for app status reports.
- puzzle 8y agoYeah, but with Borg you can have many of them, all directly accessible with one click from the Borg job status page: e.g. a general status, a view of the work queue if it's something like a crawler, incoming and outgoing RPCs, memory/cpu profiles, etc. You don't want them all on the same page for performance reasons: they would eat up a lot of temporary memory/cycles, hold important locks for long periods of time, etc. And if you need to look at a bunch of containers, you don't want to navigate to subpages for each of them.