4 ms·
Horizontal scalability carries a lot of overhead. Probably a factor of 10, easily. But the clue is in the name: eventually you'll get to a point where you have
by lmm 7y ago
Horizontal scalability carries a lot of overhead. Probably a factor of 10, easily. But the clue is in the name: eventually you'll get to a point where you have to scale.
Back in 2010 I worked for a company whose system, in Java, ran on a single web server (with one identical machine for failover). We laughed at our nearest rivals, who were using Ruby, and apparently needed 60(!) machines to run their system, which had about 5x the average request latency of ours.
Then traffic doubled, and suddenly we were having to buy six-figure servers and carefully performance-optimize our code, and our rivals with the 120 Ruby servers didn't look so funny any more. And then traffic doubled again.
- steev 7y agoThis sounds like an architecture problem, not a language problem. Can you elaborate?
- lmm 7y agoOf course it's possible to write a horizontally scaled application in Java or C++. But once you have to deal with horizontal scaling anyway, language performance is much less of an advantage: as Carmack says, the difference between 100 servers and 10 is just accounting.
- betaby 7y agoAnd the difference between 1000 servers and 100 is not just accounting. In essence, 1 full rack is easy to reason about while N>1 is not, for myriads of non accounting reasons.
- adrianN 7y agoThe Internet uses on the order of 10% of the world's electricity. A 10x difference is huge.
- jsty 7y agoOut of curiosity - what's the source on that? Most figures I've seen put datacenter usage an order of magnitude lower: e.g. https://www.iea.org/reports/tracking-buildings/data-centres-and-data-transmission-networks https://www.iea.org/reports/tracking-buildings/data-centres-... "Global data centre electricity demand in 2018 was an estimated 198 TWh, or almost 1% of global final demand for electricity (Masanet et al., 2018)."
- adrianN 7y agoThis random site from I found by googling: https://www.insidescandinavianbusiness.com/article.php?id=356 https://www.insidescandinavianbusiness.com/article.php?id=35... I was surprised by that number myself, because I also thought that it was in the single digits.
- icebraining 7y agoIt should be noted that report is talking about the client devices too - desktop, laptops, and even TVs. The datacenters part only accounted for 15% of that.
- RhodesianHunter 7y agoSpoken like an individual contributor far enough down the ladder to not have to worry about the bill.
- lmm 7y agoIn my experience it's the individual contributors who are overly obsessed with being elegant and efficient in their use of machine time. Those who are conscious of the bigger picture tend to have a more accurate sense of the relative costs of machine time versus engineering effort.
- otabdeveloper4 7y agoMultiplying your datacenter bill by 10 because you couldn't be arsed to spend two days thinking about system architecture isn't an "accurate sense of the costs" in any universe, except the one where you're spending venture capital bucks and waiting to get quckly bought by a Google with more free money than they can count.
- kcb 7y agoSometimes in a specialized team, the difference between 100 servers and 10 is profitability. Don't want to get laid off because that cloud bill is $100k a month.
- winrid 7y agoSounds like they had their datastore as part of the application. So only way to scale it would be to write a distributed data store... Or rewrite the app :p
- bcrosby95 7y agoHorizontally scaling a monolith web app has basically zero overhead. Both in app design and production complexity. But it can go decently far.
- lmm 7y agoOnly if you're assuming that the web app is entirely stateless. Which won't be the case unless it was deliberately designed that way.
- Silhouette 7y agoBut the clue is in the name: eventually you'll get to a point where you have to scale. Isn't that the conceit underlying this whole argument, though? Many systems won't ever get to the point where you have to scale in that way, if you build them efficiently in the first place. Perhaps more importantly, for many applications, you'll be able to see the limit coming some way ahead, and if you've reached a size where you do need a fundamental restructuring in order to start scaling horizontally, that's going to be a nice problem to have and you'll also have the resources to do it. I know several online systems that are handling significant traffic volumes perfectly well on a simple, single-server basis. They don't get bogged down in infrastructure and tooling issues, ever. They don't get confused by complicated cloud hosting issues, ever. They are free to spend almost their entire development budget on actually developing useful functionality, which is like a breath of fresh air in today's dev culture. Obviously in the more serious cases they probably also have some redundancy for backup/failover purposes, but even that is simple and if necessary can probably be handled manually when you only have a handful of servers to manage. Here I do slightly disagree with one of Carmack's later tweets, in that I would argue 100 servers instead of 1000 is just accounting, but 100 down to 10 or fewer is also more of a qualitative change (albeit not exactly the same qualitative change as 10 down to 1).
- paulddraper 7y agoWeb servers are usually trivially horizontally scalable. You must have had significant in-memory shared state to encounter that problem. Right? Had you adopted a less stateful model, you'd have looked rather pretty with two Java servers.
- lmm 7y ago> You must have had significant in-memory shared state to encounter that problem. Right? "Significant" is in the eye of the beholder. The core of the system was easy to make shardable. But you'd be surprised how many implicit assumptions creep in, how easy it is for ancillary parts to end up sharing state when it's easy. Also note that just because your state's in a database doesn't mean having two instances of the thing that accesses it will work, you can easily end up with an access pattern that assumes only one reader (for example) even though on paper there's no in memory state. > Had you adopted a less stateful model, you'd have looked rather pretty with two Java servers. Maybe. At the point where we're running 4 or 8 servers we'd have been facing much the same ops problems that they were. Java bought us an extra year or two of not having to deal with that, but also a significant amount of migration work when it just became impossible to stick to the single process model. At the end of the day we still kept the 5x latency advantage, which is definitely not nothing. But there were also definitely features that they brought to market quicker, and I'm pretty sure Ruby played a part in that. Tradeoffs, tradeoffs everywhere. I left before the final outcome of that fight (for all I know it's still ongoing), but I don't think either company was being dumb.
- paulddraper 7y ago> At the point where we're running 4 or 8 servers we'd have been facing much the same ops problems that they were. Yes and no. You'd need some ops work, but you'd need to worry a lot less about managing your infrastructure provisioning to keep costs low, e.g. reserved instances, dynamic scaling, etc. and putting out fires when you inevitable exceed your tight perf margins. You could overprovision 24/7 by 50% and write it off. Your competitors couldn't.
- gerbilly 7y ago