4 ms·

Everything Breaks at Scale

75 points by adrianhoward 4y ago

FollowingTheDao 4y ago
bluetomcat 4y ago
TED-style postmodern meme rambling at its finest...
- sremani 4y ago
  It did start good and the presentation format was impressive. The problem ended up being the author lacks self-awareness of her own biases and getting tribal in her own way in the later half. A fine opportunity missed on a topic of importance.
  - hinkley 4y ago
    Well if anything doesn’t scale it’s humans. I’ve been switching up my sales pitch for the productivity work I enjoy doing to frame it as scaling humans vertically instead of horizontally. Messaging overhead grows more slowly that way.
strken 4y ago
Before clicking this, I imagined it was going to be some journey through progressive scaling problems of a massive distributed system, e.g. first PostgreSQL broke, then the Cassandra cluster's defaults settings broke, then their monorepo build tools, then Git itself, then Mesos, then the Linux kernel. I'm a bit disappointed.
jaspax 4y ago
Some good and interesting ideas, but ultimately it falls apart into the same old thing: "the problem with the internet is that we aren't censoring people I disagree with."
globalreset 4y ago
Wth. Clicked for swe wisdom, got rambling about someone's childhood.
jollybean 4y ago
Conspiracy theories also work in ways that are unsettling. 'Kernel of Truth' is that obviously there are policing problems in the USA, much of race related. Surely Kaepernick's 'kneeling' was in good faith initially. But eventually, they made an icon out of him, and he had to keep the 'Faith'. His continued claims of NFL owners being like 'slavers' and specific aspects of the NFL recruiting process like that are basically repulsive, race-bating conspiracy theorizing. Netflix was happy to play along for the views, and the 'controversy' was likely to get them some attention. Which points to the missing item in that very long list of items, and that is just marginal conspiratorialism is useful in all industries - particularly politics and entertainment. Controversy makes headlines, conspiratorial headlines are controversial.
nonethewiser 4y ago
Distributed systems are a necessary evil. They fucking suck. They aren't a good solution. They are just the best solution. I wonder if there will be a paradigm shift that almost completely wipes them out. Such as quantum computing. It stands to reason that something 158 million times faster than current super computers will allow us to just scale things vertically and remove parallelization. https://www.livescience.com/quantum-computing#:~:text=Quantum%20computing%20is%20a%20new%20generation%20of%20technology%20that%20involves,supercomputer%2010%2C000%20years%20to%20accomplish https://www.livescience.com/quantum-computing#:~:text=Quantu.... Of course, as technology increases so do our expectations. So we may still utilize these complicated strategies to solve problems that are utterly unfathomable today. But you might be able to run all your microservices in 1 process.
- abraxas 4y ago
  Most businesses could easily handle their entire enterprise workloads on a single server. Add Raid and backups and for all practical purposes you have a system with the same uptime as the crazy Rube Goldberg contraptions people build in the cloud with things smeared across hundreds of machines doing God knows what.
simne 4y ago
For author, and for people who want to create better community, my advice: read excellent book "Governing the Commons: The Evolution of Institutions for Collective Action." author Elinor Ostrom. In book you will find exact rules, which determine effective community, very clear and very exact. BTW, book considers only offline communities with material borders, but this is one important rule - people offline behavior external to online platform, always influence online, so to make strong online rules, they should affect offline members behavior hard enough.

75 points by adrianhoward 4y ago

FollowingTheDao 4y ago

bluetomcat 4y ago

TED-style postmodern meme rambling at its finest...

sremani 4y ago
It did start good and the presentation format was impressive. The problem ended up being the author lacks self-awareness of her own biases and getting tribal in her own way in the later half. A fine opportunity missed on a topic of importance.
- hinkley 4y ago
  Well if anything doesn’t scale it’s humans. I’ve been switching up my sales pitch for the productivity work I enjoy doing to frame it as scaling humans vertically instead of horizontally. Messaging overhead grows more slowly that way.

strken 4y ago

Before clicking this, I imagined it was going to be some journey through progressive scaling problems of a massive distributed system, e.g. first PostgreSQL broke, then the Cassandra cluster's defaults settings broke, then their monorepo build tools, then Git itself, then Mesos, then the Linux kernel. I'm a bit disappointed.

jaspax 4y ago

Some good and interesting ideas, but ultimately it falls apart into the same old thing: "the problem with the internet is that we aren't censoring people I disagree with."

globalreset 4y ago

Wth. Clicked for swe wisdom, got rambling about someone's childhood.

jollybean 4y ago

Conspiracy theories also work in ways that are unsettling. 'Kernel of Truth' is that obviously there are policing problems in the USA, much of race related. Surely Kaepernick's 'kneeling' was in good faith initially. But eventually, they made an icon out of him, and he had to keep the 'Faith'. His continued claims of NFL owners being like 'slavers' and specific aspects of the NFL recruiting process like that are basically repulsive, race-bating conspiracy theorizing. Netflix was happy to play along for the views, and the 'controversy' was likely to get them some attention. Which points to the missing item in that very long list of items, and that is just marginal conspiratorialism is useful in all industries - particularly politics and entertainment. Controversy makes headlines, conspiratorial headlines are controversial.

nonethewiser 4y ago

Distributed systems are a necessary evil. They fucking suck. They aren't a good solution. They are just the best solution. I wonder if there will be a paradigm shift that almost completely wipes them out. Such as quantum computing. It stands to reason that something 158 million times faster than current super computers will allow us to just scale things vertically and remove parallelization. https://www.livescience.com/quantum-computing#:~:text=Quantum%20computing%20is%20a%20new%20generation%20of%20technology%20that%20involves,supercomputer%2010%2C000%20years%20to%20accomplish https://www.livescience.com/quantum-computing#:~:text=Quantu.... Of course, as technology increases so do our expectations. So we may still utilize these complicated strategies to solve problems that are utterly unfathomable today. But you might be able to run all your microservices in 1 process.

abraxas 4y ago
Most businesses could easily handle their entire enterprise workloads on a single server. Add Raid and backups and for all practical purposes you have a system with the same uptime as the crazy Rube Goldberg contraptions people build in the cloud with things smeared across hundreds of machines doing God knows what.

simne 4y ago

For author, and for people who want to create better community, my advice: read excellent book "Governing the Commons: The Evolution of Institutions for Collective Action." author Elinor Ostrom. In book you will find exact rules, which determine effective community, very clear and very exact. BTW, book considers only offline communities with material borders, but this is one important rule - people offline behavior external to online platform, always influence online, so to make strong online rules, they should affect offline members behavior hard enough.