Sure, if it's simple to horizontally scale your service. But unless it's a simple static site it's usually not, and it's usually a source of a lot of complexit

Sure, if it's simple to horizontally scale your service. But unless it's a simple static site it's usually not, and it's usually a source of a lot of complexity and pain. Which unless you really are doing something twitter scale, is not worth it. By the time you've bought a handful of tiny VPSs and put in the work to horizontally scale your service, you could have put the same money with zero effort into a very reliable dedicated server. Spend the effort elsewhere and save your sanity (and probably improve your reliability). Yes I'm biased against horizontal scaling.