5 ms·

While this is true in some cases, more frequently I saw apps designed and able to handle millions of users and billions of transactions that ended being used by

While this is true in some cases, more frequently I saw apps designed and able to handle millions of users and billions of transactions that ended being used by tens of users and hundreds of transactions. All the effort spent on testing and optimizations for scaling purpose was a waste of time and resources, that could be better spent elsewhere. I'm not telling one should not care, or code sloppy, but there is a balance where code is just good enough for the purpose. There's a lot of truth in this "don't do premature optimization".

heavenlyblue 3y ago
Yeah and these apps probably would never work with that many of users in practice because they missed a few things here and there and the only way to fix them would have been to have that amount of traffic in the first place.
jasfi 3y ago
I think the GP was saying that it should scale without breaking. It can get slow, fine, that's a different challenge. But it shouldn't segfault (as an example).
- yebyen 3y ago
  The gist of it is this: many load tests don't even consider the actual potential volume of traffic. But that's fine, if you're using a load tester, you don't have to estimate the traffic well - even to within an order of magnitude. You can just add a couple extra zeroes, and see where it breaks. Failing to do this simple thing will usually lead to objectively worse software, and there's a chance that some day you'll need to handle that much traffic. But that chance isn't the sole reason why you're doing that load test. The reason is to improve the software. You're identifying defects by stressing the limits. When you're doing a load test (or any test really) the possible outcomes are basically three: (1) it works! (2) it broke. (3) huh, that's interesting. If your tests are always coming up (1) then you're not obtaining any benefit from them. Don't you want to know where the limiting factors are in your app? If you're able to remove those limits, but not for production (at least not right now), wouldn't it be great to know what will break next month (or next year) when you do? Think of the person who writes unit tests for every piece of code, but not as TDD. There's a school of thought that you should write the test first, then write the simplest code that passes, and that's fine but not what I'm talking about. Imagine a person who writes perfect code and perfect tests. Every code works, every test passes confirming that it worked. What is even the value of writing the test? That's what load testing under only the expected conditions is like. We already know the software works under those conditions (likely) because it's already in production, handling that amount of load. So while there is value in a load test that runs prior to deployment, in order to check that nothing of the change is likely to induce a break under the expected/existing load, it's a different kind of testing and produces different value than a stress test that is designed to hopefully induce a failure and show you where there is a defect in code. Where it segfaults, for example. And just because you've identified a limiting factor outside the bounds of what expected activity is likely to go through the system soon, doesn't mean you need to fix it now. Having one less "known unknown" on the table is a thing of value. Now that stress won't be able to surprise you later, when that parameter has drifted into the danger zone because of organic development, and now it's becoming a thing in the way.
- wernercd 3y ago
  "GP was saying that it should scale without breaking" and the responder was saying that making that a priority means that your wasting time that probably won't be needed. The time you spend making it work for millions of users that won't be needed is time not spent making value to customers that do need it.
pixl97 3y ago
The question comes.. when your app is on the growth curve, do you start to test this? I work in enterprise software and one of the big problems I see is companies suck at software growth when it's obvious the software is in the upward curve. Large companies will throw huge amounts of data at your app once you sell it to them.
Cthulhu_ 3y ago
It's a difficult one; if you don't know yet if you will have billions of transactions, you should focus on clarity and flexibility - that is, you should be able to rewrite and re-architect your application and its runtime IF it turns out to be successful. A parent comment mentioned SQL databases for example; those are great because they can scale both horizontally and vertically these days, sometimes with the click of a button in AWS. Other good practices are things like stateless back-end services so they can scale horizontally, thoroughly documenting (and maintaining documentation) on business processes handled by the software, monitoring, etc. Disclaimer: I'm an armchair expert, I've never had to deal with back-end scaling.
- coffeebeqn 3y ago
  Something has gone horribly wrong if you don’t know if the requirements are 10 request per second or a billion per second. We build some services from the ground up for very high traffic and the hoops you have to jump through and the tradeoffs just don’t make sense for a basic CRUD thing which can run on a boring ole machine and a little SQL instance
wintogreen74 3y ago
also all projects, git or anything else, have limited resourcing. I'd rather it's spent on the prioritized features & needs than exhaustive testing for edge cases.
jandrewrogers 3y ago
You still need to do enough to buy time if you do need more scalability, since scalability tends to be architectural. Waiting until you hit a wall in production is usually months too late to start working on it. As a moving target, I often try to test at 10x the current workload, which is usually enough to deal with load spikes and surfaces scalability issues early enough that customers don't see them.
thrashh 3y ago
This is where experience comes in. Someone experienced will know how much work a certain approach will take and its capacity. Sometimes there are quick wins to give like 100x capacity to a system just by doing things slightly differently, but only with experience will you know that.
klysm 3y ago
I think you can write good code that intentionally omits performance optimizations you know could be made, but don’t want to make right now because it trades off complexity for performance. I usually leave myself a note of how to improve it if it does in fact become the bottleneck or starts to hurt latency

heavenlyblue 3y ago

Yeah and these apps probably would never work with that many of users in practice because they missed a few things here and there and the only way to fix them would have been to have that amount of traffic in the first place.

jasfi 3y ago

I think the GP was saying that it should scale without breaking. It can get slow, fine, that's a different challenge. But it shouldn't segfault (as an example).

yebyen 3y ago
The gist of it is this: many load tests don't even consider the actual potential volume of traffic. But that's fine, if you're using a load tester, you don't have to estimate the traffic well - even to within an order of magnitude. You can just add a couple extra zeroes, and see where it breaks. Failing to do this simple thing will usually lead to objectively worse software, and there's a chance that some day you'll need to handle that much traffic. But that chance isn't the sole reason why you're doing that load test. The reason is to improve the software. You're identifying defects by stressing the limits. When you're doing a load test (or any test really) the possible outcomes are basically three: (1) it works! (2) it broke. (3) huh, that's interesting. If your tests are always coming up (1) then you're not obtaining any benefit from them. Don't you want to know where the limiting factors are in your app? If you're able to remove those limits, but not for production (at least not right now), wouldn't it be great to know what will break next month (or next year) when you do? Think of the person who writes unit tests for every piece of code, but not as TDD. There's a school of thought that you should write the test first, then write the simplest code that passes, and that's fine but not what I'm talking about. Imagine a person who writes perfect code and perfect tests. Every code works, every test passes confirming that it worked. What is even the value of writing the test? That's what load testing under only the expected conditions is like. We already know the software works under those conditions (likely) because it's already in production, handling that amount of load. So while there is value in a load test that runs prior to deployment, in order to check that nothing of the change is likely to induce a break under the expected/existing load, it's a different kind of testing and produces different value than a stress test that is designed to hopefully induce a failure and show you where there is a defect in code. Where it segfaults, for example. And just because you've identified a limiting factor outside the bounds of what expected activity is likely to go through the system soon, doesn't mean you need to fix it now. Having one less "known unknown" on the table is a thing of value. Now that stress won't be able to surprise you later, when that parameter has drifted into the danger zone because of organic development, and now it's becoming a thing in the way.
wernercd 3y ago
"GP was saying that it should scale without breaking" and the responder was saying that making that a priority means that your wasting time that probably won't be needed. The time you spend making it work for millions of users that won't be needed is time not spent making value to customers that do need it.

pixl97 3y ago

The question comes.. when your app is on the growth curve, do you start to test this? I work in enterprise software and one of the big problems I see is companies suck at software growth when it's obvious the software is in the upward curve. Large companies will throw huge amounts of data at your app once you sell it to them.

Cthulhu_ 3y ago

It's a difficult one; if you don't know yet if you will have billions of transactions, you should focus on clarity and flexibility - that is, you should be able to rewrite and re-architect your application and its runtime IF it turns out to be successful. A parent comment mentioned SQL databases for example; those are great because they can scale both horizontally and vertically these days, sometimes with the click of a button in AWS. Other good practices are things like stateless back-end services so they can scale horizontally, thoroughly documenting (and maintaining documentation) on business processes handled by the software, monitoring, etc. Disclaimer: I'm an armchair expert, I've never had to deal with back-end scaling.

coffeebeqn 3y ago
Something has gone horribly wrong if you don’t know if the requirements are 10 request per second or a billion per second. We build some services from the ground up for very high traffic and the hoops you have to jump through and the tradeoffs just don’t make sense for a basic CRUD thing which can run on a boring ole machine and a little SQL instance

wintogreen74 3y ago

also all projects, git or anything else, have limited resourcing. I'd rather it's spent on the prioritized features & needs than exhaustive testing for edge cases.

jandrewrogers 3y ago

You still need to do enough to buy time if you do need more scalability, since scalability tends to be architectural. Waiting until you hit a wall in production is usually months too late to start working on it. As a moving target, I often try to test at 10x the current workload, which is usually enough to deal with load spikes and surfaces scalability issues early enough that customers don't see them.

thrashh 3y ago

This is where experience comes in. Someone experienced will know how much work a certain approach will take and its capacity. Sometimes there are quick wins to give like 100x capacity to a system just by doing things slightly differently, but only with experience will you know that.

klysm 3y ago

I think you can write good code that intentionally omits performance optimizations you know could be made, but don’t want to make right now because it trades off complexity for performance. I usually leave myself a note of how to improve it if it does in fact become the bottleneck or starts to hurt latency