The reality of it is that a lot of people vastly overestimate the activity of what they build will actually reach. You can easily host a lot of stuff with just

The reality of it is that a lot of people vastly overestimate the activity of what they build will actually reach. You can easily host a lot of stuff with just a "basic" SQL database and a single frontend as long as you have proper rate limiting on your APIs and make sure your frontend is sane. NGINXs original design goal was to handle 10,000 connections at the same time and nowadays it can handle up to ~400,000 without needing reconfiguration. Even a basic cache layer on nginx can often completely circumvent the need of calling the backend, especially for open APIs. Postgres can easily handle dozens of connections at the same time with the right config flags. To give a bit of an estimate on how far basic infrastructure can carry you; the entire RetroArch project[1], which I'd say has about the same userbase as any small tech company is entirely hosted, buildbot (so CI) included, on a single 40$ DigitalOcean instance. Most software is suprisingly efficient/resilient if you just configure it properly. The only place where I've seen horizontal scaling as an actual necessity vs. just properly configuring the tools you're given in most cases is Rails. There's no real efficiency fix for Rails tooling from what I can tell. It's just slow as molasses and you're gonna foot an ever expanding server bill as your userbase grows. [1]: No endorsement.