I've been running my own Linux servers for about twenty years, so I'm not entirely unexperienced with these things. Borg is/was great for running huge numbers

I've been running my own Linux servers for about twenty years, so I'm not entirely unexperienced with these things. Borg is/was great for running huge numbers of services at truly massive scale, when those services were developed entirely in house and done in the exact way it wanted services to be done. It was a terrible cost the moment you wanted to run anything third party or which wasn't written in that exact Google way, and the costs were especially high if you didn't need to handle huge traffic or data sizes. Borg and Kubernetes don't auto-magically do sysadmin work. A ton of people work in infrastructure at Google. Software updates still need to be applied etc. In Kubernetes that means rebuilding Docker images (which in reality doesn't happen as the tools for this are poor, so you just have a lot of downlevel Ubuntu images floating around). And worse, the whole K8s/Docker paradigm is totally backwards incompatible. At Google this didn't matter because the software stack evolved in parallel with the cluster management. Programs there expected local disk to be volatile, expected to be killed at a moment's notice, expected datacenters to appear and disappear like moles. They were written that way from scratch. But that came with a terrible price: it was basically impossible to use ordinary open source apps. You could import libraries and (carefully!) incorporate them into Borg-ized projects, but that was about it. When this tech was reimplemented and thrown over the wall to the community, the cultural expectations that came with it didn't come with it. So I've seen situations like, "whoops, we deleted our companies private key because it was stored to local disk and then the Docker container was shut down, help!". This was even reported as a bug in the software! No, the bug is that your computer is randomly deleting critical files for no good reason, and normal software does not expect that to happen. How about the way in a Dockerfile you have to write 'apt-get update && apt-get upgrade' if you want a secure OS image when the container is rebuilt? If you put the two commands on separate lines it will appear to be working, right up until you start getting weird errors about missing files from Debian mirrors, because Docker assumes every single command you run is a pure function! And then we get to security. Screwups seem to follow Kubernetes/Docker around like flies. The tech is complex and violates basic expectations programs have about how POSIX works. When it goes wrong, it leads to mistakes in production. Now there have been a few trends over time: 1. Hardware has got a lot more powerful. It has been outpacing growth in the internet and economy. Many more businesses fit in a smaller number of machines than when Borg was designed (an era where 4-core systems were considered high end). 2. Cheap colo providers have been driving the cost of powerful VMs down to the ground. 3. Linux has got easier to administer. These days setting up a bunch of Linux VMs that self-upgrade, run some services via systemd etc isn't difficult, and properly tuned such a setup should be able to serve a monster amount of traffic (watch out though for Azure, which seems to overcommit capacity pretty drastically and their VMs have very unstable performance). Most businesses that are deploying Kubernetes today quite simply do not need a million machines. Even companies that give away complex services like GitLab, as we can see from this thread, they don't really need huge scalability. It's just nice to imagine that the business will experience explosive growth and that growth is now automated, but ironically, the effort to automate business infrastructure scaling takes away from the sort of efforts that actually grow the business. As for the other benefits, you can write a systemd unit that is much simpler than Kubernetes configuration that will give you auto-restart, including if memory limits are hit, you can view the activity of a cluster easily using plain old SSH or something like Cockpit, it will handle log rotation for you out of the box, sandboxing likewise, and so on. And of course the unattended-upgrades package has existed for a while. I agree you need someone to do admin work, whatever path you choose. Having used Kubernetes, and the system it is based on, and plain old Linux, my intuition is that the base cost of Kubernetes is too high for almost all its users. Too many ways to screw it up, too many ways for it to go wrong, too much time spent screwing around, and too expensive. If you become another Google or Facebook then sure, go for it. Otherwise, better avoided.