3 ms·

Yes, that's true but beware of the fact it only works this way if your code is 0%-serial, see https://en.wikipedia.org/wiki/Amdahl%27s_law https://en.wikipedia.

Yes, that's true but beware of the fact it only works this way if your code is 0%-serial, see https://en.wikipedia.org/wiki/Amdahl%27s_law https://en.wikipedia.org/wiki/Amdahl%27s_law for the math. By the way, this law is why supercomputers like Fugaku https://www.youtube.com/watch?v=WVsFFojdq3c https://www.youtube.com/watch?v=WVsFFojdq3c simply have no other way than optimizing their code. 0.1% serial code means that a code running on 160000 CPUs will be just 1000x faster than the code running on a single CPU. Further, https://en.wikipedia.org/wiki/Gustafson%27s_law https://en.wikipedia.org/wiki/Gustafson%27s_law may be more appropriate here as it is common for the app requirements grow with the performance increase of the compute hardware. Edit: bottom line is that the conclusions of the article are dangerous. Reducing the controller method exec time by 10x is not likely to allow your system to process 10x RPS by itself, the cost and product backlog delay caused by a perf optimisation project may not be acceptable etc. But beware that slow code in the critical path of your distributed/parallel system such as kernel code, event loop code, transaction commit code, distributed consensus protocol implementation can make your system unscalable beyond N factor (whether it is cores or servers). You can surely add 100xN nodes but it won't help.

CyberDildonics 6y ago
This is already a gross misunderstanding of Amdahl's Law and what it means pragmatically, but you have it things completely reversed. Optimizing something to run 100x faster avoids needing more servers, thus avoiding the synchronization from splitting things up to them. That synchronization by the way is not actually done by the servers, it would be done by dedicated networking hardware. Even on a single computer, for web server programs the synchronization is not really happening in the user space programs, it is happening in the in kernel for the IO to and from disk and to/from the network.
- smarx007 6y ago
  We are actually talking about orthogonal aspects. You are right that being able to run the system on a smaller number of nodes reduces inefficiencies and reduces the "hit" of diminishing returns stemming from Amdahl's law. However, to do that, you need to speed up your code and I argue that unless you speed up the non-parallelizable part together with the parallelizable part, you won't be able to achieve that system speedup that allows you to remove a number of servers from your cluster in the first place. However, I think you are right that on small number of servers speeding up parallel part can have a more immediate effect. I did a small calculation just to check the numbers. Assume you have a server that can do 100 RPS. You can either speed up the parallel code 2x (let's be realistic here and stop this 100x nonsense) and bring the RPS almost to 200 or you can take that serial code that is just a fraction and reduce it by half, so that f=0.995 and not 0.99 (and adding a marginal RPS increase to 102 RPS). Here is the plot: https://imgur.com/a/7w1hms9 https://imgur.com/a/7w1hms9 Edit: read http://www.frankmcsherry.org/assets/COST.pdf http://www.frankmcsherry.org/assets/COST.pdf as recommended in the upvoted comment above!
  - CyberDildonics 6y ago
    I think you are building on misunderstanding with assumptions that aren't backed up by anything. > non-parallelizable part Which part is that? Synchronization does not have to be expensive. The original paper had a theoretical 'serial' part of a program, but synchronization is different in that it can be very fast and doesn't tie up other resources. For a web server the synchronization is in IO, which is being handled by queues. It doesn't stall cores, processes or threads can put their data in a queue and another core can handle it. > You can either speed up the parallel code 2x (let's be realistic here and stop this 100x nonsense) Far from it. If python is translated to C++ directly you would already have a massive speedup (I would guess at least 20x, maybe more). If memory allocations are minimized you get another huge (7x) speedup on top of that. If larger chunks are dealt with at one time you get another huge speedup. I think 100x would be common. Amdahl's Law is really about diminishing returns when there are significant places that need to be serial. Any emulator can be a good example. There aren't nearly as many scenarios that have to be serial as most people think.
ajnin 6y ago
That's not how it works, Amdahl's law is about the execution speedup you get by splitting a task across multiple cores, it is not about how many tasks you can run simultaneously on those cores, usually when we're talking about horizontal scaling what is meant is multiple servers executing tasks independently, not parallelizing the execution of individual tasks, which is a much harder problem with the issues you describe.
- smarx007 6y ago
  So what you are saying is that because Fugaku supercomputer has 158,976 nodes, it's "horizontally distributed" and Amdahl's law does not apply to it but only to the 48 cores on each node? A task that we have (if we want to formalize it for analysis via Amdahl's law) is to process 1MM requests on 1000 web servers and a single SQL server behind them. What we are after is how much faster can this setup process 1MM requests compared to a setup with 1 web server and 1 SQL server (1 SQL server is used here to exemplify the sequential part of the request handling). Indirectly from this calculation you can derive the increase in overall system RPS after you run a benchmark actually pushing 1MM requests through 1000 servers vs 1 server. If you have a line of code `sqlUpdateTransaction.commitBlockingWait()`, and it takes 1% of your request handler execution time, then even if you add 1000 web servers but have only 1 SQL server machine behind them (a little bit oversimplifying here as contention is likely to degrade performance further, assume we are able to scale the SQL vertically just so that the call above always takes 1% of the request handler time for the sake of simplicity), your system will not process 1MM requests more than 91x faster than a setup with a single web server and a single SQL server. See https://www.wolframalpha.com/input/?i=1%2F%28%281-0.99%29%2B0.99%2F1000%29 https://www.wolframalpha.com/input/?i=1%2F%28%281-0.99%29%2B.... And no amount of green threads will allow you to outsmart the Amdahl's law. The only thing you can do is to return 200 OK before the transaction is committed, which would bring sequential part of the handler code from 1% to (nearly) 0% (and arguably be an unacceptable method of performance optimisation).
  - CyberDildonics 6y ago
    > And no amount of green threads will allow you to outsmart the Amdahl's law. The only thing you can do is to return 200 OK before the transaction is committed, Ignoring all the wild assumptions and made up numbers here, this again has nothing to do with Amdahl's law because the "serial" part is unnecessary synchronization and waiting, not unavoidable serial computation. If you have a thousand web servers, having all of them wait to sync would be the exact thing anyone spending millions on servers would work to avoid. To start at the obvious bare basics, if only one computer can sync at one time, you might as well just use one computer. Computers are queues and buffers all the way down and all the way up to the network level. A single computer or core merging data is not Amdahl's law. There is no reason computation resources have to wait on some other resource synchronizing, and this is not serial computation, which is what Amdahl's law is about.

CyberDildonics 6y ago

This is already a gross misunderstanding of Amdahl's Law and what it means pragmatically, but you have it things completely reversed. Optimizing something to run 100x faster avoids needing more servers, thus avoiding the synchronization from splitting things up to them. That synchronization by the way is not actually done by the servers, it would be done by dedicated networking hardware. Even on a single computer, for web server programs the synchronization is not really happening in the user space programs, it is happening in the in kernel for the IO to and from disk and to/from the network.

smarx007 6y ago
We are actually talking about orthogonal aspects. You are right that being able to run the system on a smaller number of nodes reduces inefficiencies and reduces the "hit" of diminishing returns stemming from Amdahl's law. However, to do that, you need to speed up your code and I argue that unless you speed up the non-parallelizable part together with the parallelizable part, you won't be able to achieve that system speedup that allows you to remove a number of servers from your cluster in the first place. However, I think you are right that on small number of servers speeding up parallel part can have a more immediate effect. I did a small calculation just to check the numbers. Assume you have a server that can do 100 RPS. You can either speed up the parallel code 2x (let's be realistic here and stop this 100x nonsense) and bring the RPS almost to 200 or you can take that serial code that is just a fraction and reduce it by half, so that f=0.995 and not 0.99 (and adding a marginal RPS increase to 102 RPS). Here is the plot: https://imgur.com/a/7w1hms9 https://imgur.com/a/7w1hms9 Edit: read http://www.frankmcsherry.org/assets/COST.pdf http://www.frankmcsherry.org/assets/COST.pdf as recommended in the upvoted comment above!
- CyberDildonics 6y ago
  I think you are building on misunderstanding with assumptions that aren't backed up by anything. > non-parallelizable part Which part is that? Synchronization does not have to be expensive. The original paper had a theoretical 'serial' part of a program, but synchronization is different in that it can be very fast and doesn't tie up other resources. For a web server the synchronization is in IO, which is being handled by queues. It doesn't stall cores, processes or threads can put their data in a queue and another core can handle it. > You can either speed up the parallel code 2x (let's be realistic here and stop this 100x nonsense) Far from it. If python is translated to C++ directly you would already have a massive speedup (I would guess at least 20x, maybe more). If memory allocations are minimized you get another huge (7x) speedup on top of that. If larger chunks are dealt with at one time you get another huge speedup. I think 100x would be common. Amdahl's Law is really about diminishing returns when there are significant places that need to be serial. Any emulator can be a good example. There aren't nearly as many scenarios that have to be serial as most people think.

ajnin 6y ago

That's not how it works, Amdahl's law is about the execution speedup you get by splitting a task across multiple cores, it is not about how many tasks you can run simultaneously on those cores, usually when we're talking about horizontal scaling what is meant is multiple servers executing tasks independently, not parallelizing the execution of individual tasks, which is a much harder problem with the issues you describe.

smarx007 6y ago
So what you are saying is that because Fugaku supercomputer has 158,976 nodes, it's "horizontally distributed" and Amdahl's law does not apply to it but only to the 48 cores on each node? A task that we have (if we want to formalize it for analysis via Amdahl's law) is to process 1MM requests on 1000 web servers and a single SQL server behind them. What we are after is how much faster can this setup process 1MM requests compared to a setup with 1 web server and 1 SQL server (1 SQL server is used here to exemplify the sequential part of the request handling). Indirectly from this calculation you can derive the increase in overall system RPS after you run a benchmark actually pushing 1MM requests through 1000 servers vs 1 server. If you have a line of code `sqlUpdateTransaction.commitBlockingWait()`, and it takes 1% of your request handler execution time, then even if you add 1000 web servers but have only 1 SQL server machine behind them (a little bit oversimplifying here as contention is likely to degrade performance further, assume we are able to scale the SQL vertically just so that the call above always takes 1% of the request handler time for the sake of simplicity), your system will not process 1MM requests more than 91x faster than a setup with a single web server and a single SQL server. See https://www.wolframalpha.com/input/?i=1%2F%28%281-0.99%29%2B0.99%2F1000%29 https://www.wolframalpha.com/input/?i=1%2F%28%281-0.99%29%2B.... And no amount of green threads will allow you to outsmart the Amdahl's law. The only thing you can do is to return 200 OK before the transaction is committed, which would bring sequential part of the handler code from 1% to (nearly) 0% (and arguably be an unacceptable method of performance optimisation).
- CyberDildonics 6y ago
  > And no amount of green threads will allow you to outsmart the Amdahl's law. The only thing you can do is to return 200 OK before the transaction is committed, Ignoring all the wild assumptions and made up numbers here, this again has nothing to do with Amdahl's law because the "serial" part is unnecessary synchronization and waiting, not unavoidable serial computation. If you have a thousand web servers, having all of them wait to sync would be the exact thing anyone spending millions on servers would work to avoid. To start at the obvious bare basics, if only one computer can sync at one time, you might as well just use one computer. Computers are queues and buffers all the way down and all the way up to the network level. A single computer or core merging data is not Amdahl's law. There is no reason computation resources have to wait on some other resource synchronizing, and this is not serial computation, which is what Amdahl's law is about.