4 ms·

>Design for your current workload Please be my friend. The bulk of my job these days is sitting in design reviews trying to convince people that just making u

>Design for your current workload Please be my friend. The bulk of my job these days is sitting in design reviews trying to convince people that just making up scenarios where you need to scale to 100x isn't actually engineering. It's exhausting self-indulgence. Nothing is easier than inventing scenarios where you'll need to "scale". It's borderline impossible to get software folks to just live in the world that exists in front of their eyes. "Scale" should go back to its original meaning: change relative to some starting point. Slap a trend line on what you know. Do some estimation for what you don't. Design accordingly.

jpgvm 3y ago
Yeah unless you are designing the replacement for a system that has already reached it's scalability limits you shouldn't be worrying too much other than not doing very silly things architecturally. When you are designing replacements though you need to have an idea of what your scalability runway is before the next re-design will be possible. Sometimes that is just 2x, often times it's 10x, occasionally it's 100x but it's all situational.
ShroudedNight 3y ago
> just making up scenarios where you need to scale to 100x isn't actually engineering. Even for "peak" Amazon's concern seemed limited to about ~5x daily TPS maximums unless one had extraordinary evidence. The counter-balance to limiting resource balooning to 5x scale is introducing Erlang-B modelling. Depending on how many 9s you require, you may need way more service availability than expected. The 100x calculations are probably doubly wrong (both too large and too small), providing negative value false confidence.
Cthulhu_ 3y ago
This has been a problem in a few places I've worked at that decided to build "microservices" (read: simple apps that moved the complexity to a higher architectural level, i.e. by having every service talk to every other service over a REST API or event bus), not because it solved a problem they had, but because it MIGHT solve a problem they'd LIKE to have. Cargo cult, in a nutshell. My current employer is going all-in on serverless because it solved a problem they had with performance. The problem wasn't solved by serverless, but by moving away from an old fashioned and unscalable solution. The real problem or bottleneck - a centralized SAP database - has not been solved yet. They would have achieved the same results if they rebuilt their API in a generic Java monolith. Here's my prediction: when the crack team of consultants that powered through building a serverless architecture leaves (because they will, one because they're consultants and two because they get bored when the problem has been solved), they won't be able to find competent developers to take over and the whole thing will be rebuilt again in something they CAN find developers for. I mean it's just NodeJS, but the architecture is distributed and much harder to manage than in a simple monolithic app.
iamleppert 3y ago
With few exceptions, the motivation for scale that isn't driven by a real need in the real world is because someone wants to experiment with distributed systems and is looking for an excuse. I think it comes from a good place -- wanting to learn something new, or add a new skill to your toolbox. However, what happens is most of the time the product or company will never even reach that scale so that effort will be wasted. If you're not able to close the loop and actually support a running system at scale, with a real workload (not simulated or ab stats) it actually does the opposite: you don't learn what really works and what doesn't. It's easy to convince yourself you have built a system that can handle such scale based off back of the envelope calculations, synthetic testing, or cargo-culting. Most of what actually happens with scale in the real world has to do with old & boring things like appropriate choice of data structures, cache design, load balancing, locality, etc. and has nothing to do with distributed computing, serverless, etc. which are just tools that have specific characteristics that might make them good choices or not.
kamray23 3y ago
Believe me, our startup offering a coffee marketplace for fans of TANO*C HARDCORE music who want to contribute to Azeri poverty elimination efforts is going to need to scale from hundreds of users to hundreds of millions in a matter of months and you need to design ahead for that.

jpgvm 3y ago

Yeah unless you are designing the replacement for a system that has already reached it's scalability limits you shouldn't be worrying too much other than not doing very silly things architecturally. When you are designing replacements though you need to have an idea of what your scalability runway is before the next re-design will be possible. Sometimes that is just 2x, often times it's 10x, occasionally it's 100x but it's all situational.

ShroudedNight 3y ago

> just making up scenarios where you need to scale to 100x isn't actually engineering. Even for "peak" Amazon's concern seemed limited to about ~5x daily TPS maximums unless one had extraordinary evidence. The counter-balance to limiting resource balooning to 5x scale is introducing Erlang-B modelling. Depending on how many 9s you require, you may need way more service availability than expected. The 100x calculations are probably doubly wrong (both too large and too small), providing negative value false confidence.

Cthulhu_ 3y ago

This has been a problem in a few places I've worked at that decided to build "microservices" (read: simple apps that moved the complexity to a higher architectural level, i.e. by having every service talk to every other service over a REST API or event bus), not because it solved a problem they had, but because it MIGHT solve a problem they'd LIKE to have. Cargo cult, in a nutshell. My current employer is going all-in on serverless because it solved a problem they had with performance. The problem wasn't solved by serverless, but by moving away from an old fashioned and unscalable solution. The real problem or bottleneck - a centralized SAP database - has not been solved yet. They would have achieved the same results if they rebuilt their API in a generic Java monolith. Here's my prediction: when the crack team of consultants that powered through building a serverless architecture leaves (because they will, one because they're consultants and two because they get bored when the problem has been solved), they won't be able to find competent developers to take over and the whole thing will be rebuilt again in something they CAN find developers for. I mean it's just NodeJS, but the architecture is distributed and much harder to manage than in a simple monolithic app.

iamleppert 3y ago

With few exceptions, the motivation for scale that isn't driven by a real need in the real world is because someone wants to experiment with distributed systems and is looking for an excuse. I think it comes from a good place -- wanting to learn something new, or add a new skill to your toolbox. However, what happens is most of the time the product or company will never even reach that scale so that effort will be wasted. If you're not able to close the loop and actually support a running system at scale, with a real workload (not simulated or ab stats) it actually does the opposite: you don't learn what really works and what doesn't. It's easy to convince yourself you have built a system that can handle such scale based off back of the envelope calculations, synthetic testing, or cargo-culting. Most of what actually happens with scale in the real world has to do with old & boring things like appropriate choice of data structures, cache design, load balancing, locality, etc. and has nothing to do with distributed computing, serverless, etc. which are just tools that have specific characteristics that might make them good choices or not.

kamray23 3y ago

Believe me, our startup offering a coffee marketplace for fans of TANO*C HARDCORE music who want to contribute to Azeri poverty elimination efforts is going to need to scale from hundreds of users to hundreds of millions in a matter of months and you need to design ahead for that.