5 ms·

It's sort of strange you have to make these points, but as an industry we seem to have an extremely short memory. Vertical scaling was absolutely the way most

It's sort of strange you have to make these points, but as an industry we seem to have an extremely short memory. Vertical scaling was absolutely the way most big applications were built up until well into the 90s. Companies like Oracle were really built on the fact that getting performance and reliability out of a single highly-contested massive server is hard but important if that's the way you're going. Linux became dominant primarily because horizontal scaling won that argument and it won it pretty much exactly because of: 1)what you said - you hit a hard cap on how big you can make your main server at which point you are really screwed. Scalability pain points become a hard wall. 2) when I say "server" I mean "servers" of course because you'd need an H/A failover, at which point you've eaten the cost of replication, handling failover etc and you may as well distribute 3) cost. Because hardware cost vs capability is exponential, as your requirements become bigger you pretty rapidly hit a point where lots of commodity hardware becomes cheaper for a given performance point than few big servers So there's a reason that distributed systems on commodity hardware became the dominant architectural paradigm. It's not the only way to do it, but it's a reasonable default for many use cases. For a very high-throughput system like twitter it seems a very obvious choice. Clearly there are costs to distribution, so if you can get away with a simpler architecture then as always Occam's razor applies. Also if you can easily distribute later then it probably makes sense to leave that option open and explore it when you need it rather than overcomplicate too early.

Joeri 4y ago
The thing is that hardware scales faster than humanity. When the internet boom happened there was no choice except to scale horizontally to reach a global audience, but as this article points out that assumption might no longer hold true for many services. It might make sense to return to vertically scaled highly reliable servers to achieve software simplicity and a lower overall cost. I’m always reminded of how stackoverflow essentially runs off a single database server. If they can do it, most web properties can do it.
- saagarjha 4y ago
  Stack Overflow happens to go down fair maintenance a fair bit. Now, it’s not really a service that might need reliability like Twitter does*, but it’s important to keep in mind. *Some engineers may disagree
  - brazzledazzle 4y ago
    Also worth pointing out that Stack Overflow's Microsoft-centric architecture may also incentivize them to maximize vertical scaling to save on the licensing cost overhead horizontal scaling would incur for at least part of their stack.
- kaba0 4y ago
  Most, sure. Twitter? No way on Earth.
- jameshart 4y ago
  When the hardware scales, the tricks to wring maximum performance out of it change. When you come back to build the new version of your game on the next gen console sure you can now add all those features but the processing pipelines are different now and the disk performance and memory to cache ratios have changed - getting your hyper optimized code to work in the new platform takes a ton of effort - so you either run it in some kind of emulation mode, sacrificing some of the performance for productivity, or you rewrite it. Same happens with new generations of server hardware. Your clever hack to maximize NUMA locality of data to each core becomes a liability when the next hardware gen comes out and on-die caches are bigger. Decisions about what should use RAM get invalidated by faster SSDs. Maybe you can build a service this way - hardware first. Pick a server platform for a couple of years; build to that capacity; ship; then start designing the next gen service to run on a new set of hardware? To some extent this is how database or virtualization systems software is written. And if I’m not mistaken Twitter actually did develop their own database stack to optimally handle their particular data storage model, and I assume that was done pretty close to the metal.
popcorncowboy 4y ago
It's less "short memory" than the fact that you can be a "senior software engineer" after just 5 years or so experience. There is a significant cohort of (particularly web-tech) developers who were young children in the 90s, and whose professional careers started in the 2010s and have only ever known "the cloud", big-tech and big-tech tech (k8s, etc). It's a similar phenomenon to the observation that tech "innovations" tend to recapitulate research that had its roots in the 50-60-70s. "The industry" doesn't seem to put much stock in generational knowledge transfer.
- britannio 4y ago
  I'm a CS undergrad, do you have any recommendations for effectively combating this?
  - wrigby 4y ago
    No approach is a silver bullet here, but what I've found effective is to seek out friendships with / mentorship from senior and staff-level engineers. They'll have tons of war stories from 10-20 years ago, and may even have some snarky opinions on what "new technologies" are just re-inventions of something that the industry had already solved decades ago. As a current undergrad, you can also look to your professors for this (especially those with industry experience before they went into teaching). After graduation, this may mean working at a company _with_ those older engineers, as opposed to a 5-20 person startup with a homogeneous group of 20-somethings.
    - jameshart 4y ago
      It’s gotta be tough though for young devs to tell the difference between old fogeys just dismissing new tech because it’s new, vs because it’s a recapitulation of an old mistake. And the thing is that what was a bad idea in 2000 might now be an idea whose time has come, because the surrounding context has changed - be it browser technology or the size of machine memory or the capabilities of programming languages. So, like, when I point out that kubernetes is just DCOM all over again I’m not actually dismissing kubernetes (just because we don’t use DCOM any more doesn’t mean it wasn’t a good choice then); less still suggesting that we should go back to using DCOM; I’m just saying ‘maybe there are some lessons we can learn from how people used DCOM back in the day about what cases kubernetes is suited for and what the pitfalls might be’. And, also, maybe raising the possibility that in a few years time we will look back at kubernetes as a bloated outdated approach and be glad to see the back of it - even though right now it might be a great technology to use. But I’m not sure how a new junior dev can possibly pick up all that nuance from just listening to old farts like me talking about how this reminds us of how we used to do things back in the old days.

Joeri 4y ago

The thing is that hardware scales faster than humanity. When the internet boom happened there was no choice except to scale horizontally to reach a global audience, but as this article points out that assumption might no longer hold true for many services. It might make sense to return to vertically scaled highly reliable servers to achieve software simplicity and a lower overall cost. I’m always reminded of how stackoverflow essentially runs off a single database server. If they can do it, most web properties can do it.

saagarjha 4y ago
Stack Overflow happens to go down fair maintenance a fair bit. Now, it’s not really a service that might need reliability like Twitter does*, but it’s important to keep in mind. *Some engineers may disagree
- brazzledazzle 4y ago
  Also worth pointing out that Stack Overflow's Microsoft-centric architecture may also incentivize them to maximize vertical scaling to save on the licensing cost overhead horizontal scaling would incur for at least part of their stack.
kaba0 4y ago
Most, sure. Twitter? No way on Earth.
jameshart 4y ago
When the hardware scales, the tricks to wring maximum performance out of it change. When you come back to build the new version of your game on the next gen console sure you can now add all those features but the processing pipelines are different now and the disk performance and memory to cache ratios have changed - getting your hyper optimized code to work in the new platform takes a ton of effort - so you either run it in some kind of emulation mode, sacrificing some of the performance for productivity, or you rewrite it. Same happens with new generations of server hardware. Your clever hack to maximize NUMA locality of data to each core becomes a liability when the next hardware gen comes out and on-die caches are bigger. Decisions about what should use RAM get invalidated by faster SSDs. Maybe you can build a service this way - hardware first. Pick a server platform for a couple of years; build to that capacity; ship; then start designing the next gen service to run on a new set of hardware? To some extent this is how database or virtualization systems software is written. And if I’m not mistaken Twitter actually did develop their own database stack to optimally handle their particular data storage model, and I assume that was done pretty close to the metal.

popcorncowboy 4y ago

It's less "short memory" than the fact that you can be a "senior software engineer" after just 5 years or so experience. There is a significant cohort of (particularly web-tech) developers who were young children in the 90s, and whose professional careers started in the 2010s and have only ever known "the cloud", big-tech and big-tech tech (k8s, etc). It's a similar phenomenon to the observation that tech "innovations" tend to recapitulate research that had its roots in the 50-60-70s. "The industry" doesn't seem to put much stock in generational knowledge transfer.

britannio 4y ago
I'm a CS undergrad, do you have any recommendations for effectively combating this?
- wrigby 4y ago
  No approach is a silver bullet here, but what I've found effective is to seek out friendships with / mentorship from senior and staff-level engineers. They'll have tons of war stories from 10-20 years ago, and may even have some snarky opinions on what "new technologies" are just re-inventions of something that the industry had already solved decades ago. As a current undergrad, you can also look to your professors for this (especially those with industry experience before they went into teaching). After graduation, this may mean working at a company _with_ those older engineers, as opposed to a 5-20 person startup with a homogeneous group of 20-somethings.
  - jameshart 4y ago
    It’s gotta be tough though for young devs to tell the difference between old fogeys just dismissing new tech because it’s new, vs because it’s a recapitulation of an old mistake. And the thing is that what was a bad idea in 2000 might now be an idea whose time has come, because the surrounding context has changed - be it browser technology or the size of machine memory or the capabilities of programming languages. So, like, when I point out that kubernetes is just DCOM all over again I’m not actually dismissing kubernetes (just because we don’t use DCOM any more doesn’t mean it wasn’t a good choice then); less still suggesting that we should go back to using DCOM; I’m just saying ‘maybe there are some lessons we can learn from how people used DCOM back in the day about what cases kubernetes is suited for and what the pitfalls might be’. And, also, maybe raising the possibility that in a few years time we will look back at kubernetes as a bloated outdated approach and be glad to see the back of it - even though right now it might be a great technology to use. But I’m not sure how a new junior dev can possibly pick up all that nuance from just listening to old farts like me talking about how this reminds us of how we used to do things back in the old days.