4 ms·

During their black friday / cyber monday load peak, Shopify averaged between ~0.85 and ~1.94 back-to-back RPS per CPU core. Take from that what you will. Refer

During their black friday / cyber monday load peak, Shopify averaged between ~0.85 and ~1.94 back-to-back RPS per CPU core. Take from that what you will. Reference: https://x.com/ShopifyEng/status/1863953413559472291 https://x.com/ShopifyEng/status/1863953413559472291

kstrauser 2y ago
Ouch. I had no idea it was that much of a resource hog.
cristianbica 2y ago
Obviously you meant for the whole infrastructure: ruby / rails workers, Mysql, Kafka, whatever other stuff their app needs (redis, memcache, etc), loadbalancers, infrastructure monitoring, etc.
- neonsunset 2y ago
  This is correct! I thought this was clear but I guess not...
  - ksec 2y ago
    It is not because this is the first time I heard about back to back RPS. Which when come to think of it isn't too bad of a metric from a business POV. We can also infer that into how much saving YJIT provides. At this point Shopify is likely already getting a return of investment from YJIT.
PhilippGille 2y ago
You seem to imply that everything they run is Ruby, but they're talking about 2.4 million CPU cores on their K8s cluster, where maybe other stuff runs as well, like their Kafka clusters [1] and Airflow [2]? [1] https://shopify.engineering/running-apache-kafka-on-kubernetes-at-shopify https://shopify.engineering/running-apache-kafka-on-kubernet... [2] https://shopify.engineering/lessons-learned-apache-airflow-scale https://shopify.engineering/lessons-learned-apache-airflow-s...
phoronixrly 2y ago
Just to reiterate stuff said in the other comments because your comment is maybe deliberately misrepresenting what was said in the thread. Their entire cluster was 2.4 million CPU cores (without more info on what the cores were). This includes not only Ruby web applications that handle requests, but also other infrastructure. Asynchronous processing, database servers, message queue processing, data workflows etc, etc, etc. You cannot run a back of the envelope calculation and say 0.85 requests per second per core and that is why they're optimising Ruby. While that might be the end result and a commentary on contemporary software architecture as a whole, it does not tell you much about the performance of the Ruby part of the equation in isolation. They had bursts of 280 million rpm (4.6 million rps) with average of 2.8 million rps.
- neonsunset 2y ago
  > It does not tell you much about the performance of the Ruby part of the equation in isolation. Indeed, it doesn't. However, it would be a fairly safe bet to assume it was the slowest part of their architecture. I keep wondering how the numbers would change if Ruby were to be replaced with something else.
  - phoronixrly 2y ago
    Since we're insinuating, I bet you that Ruby is not their chief bottleneck. You won't get much more RPS if you wait on an SQL query or RPC/HTTP API call. In my experience when you have a bottleneck in the actual Ruby code (not speaking about n+1s or heavy SQL queries or other IO), the code itself is written in such a way that it would be slow in whichever language. Again, in my experience this involves lots of (oft unnecessary) allocations and slow data transformations. Usually this is preceded by a slow heavy SQL query. You fix the query and get a speed-up of 0.8 rps to 40 rps, add a TODO entry "the following code needs to be refactored" but you already ran out of estimation and mark the issue as resolved. Couple of months later the optimization allowed the resultset to grow and the new bottleneck is memory use and the speed of the naive algorithm and lack of appropriate data structures in the data transformation step... Again in the same code you diligently TODOed... Tell me how this is Ruby's fault. Another example is one of the 'Oh we'll just introduce Redis-backed cache to finally make use of shared caching and alleviate the DB bottleneck'. Implementation and validation took weeks. Finally all tests are green. The test suite runs for half an hour longer. Issue was traced to latency to the Redis server and starvation due to locking between parallel workers. The task was quietly shelved afterwards without ever hitting production or being mentioned again in a prime example of learned helplessness. If only we had used an actual real programming language and not Ruby, we would not be hitting this issue (/s) I wish most performance problems would be solved by just using a """fast language"""...

kstrauser 2y ago

Ouch. I had no idea it was that much of a resource hog.

cristianbica 2y ago

Obviously you meant for the whole infrastructure: ruby / rails workers, Mysql, Kafka, whatever other stuff their app needs (redis, memcache, etc), loadbalancers, infrastructure monitoring, etc.

neonsunset 2y ago
This is correct! I thought this was clear but I guess not...
- ksec 2y ago
  It is not because this is the first time I heard about back to back RPS. Which when come to think of it isn't too bad of a metric from a business POV. We can also infer that into how much saving YJIT provides. At this point Shopify is likely already getting a return of investment from YJIT.

PhilippGille 2y ago

You seem to imply that everything they run is Ruby, but they're talking about 2.4 million CPU cores on their K8s cluster, where maybe other stuff runs as well, like their Kafka clusters [1] and Airflow [2]? [1] https://shopify.engineering/running-apache-kafka-on-kubernetes-at-shopify https://shopify.engineering/running-apache-kafka-on-kubernet... [2] https://shopify.engineering/lessons-learned-apache-airflow-scale https://shopify.engineering/lessons-learned-apache-airflow-s...

phoronixrly 2y ago

Just to reiterate stuff said in the other comments because your comment is maybe deliberately misrepresenting what was said in the thread. Their entire cluster was 2.4 million CPU cores (without more info on what the cores were). This includes not only Ruby web applications that handle requests, but also other infrastructure. Asynchronous processing, database servers, message queue processing, data workflows etc, etc, etc. You cannot run a back of the envelope calculation and say 0.85 requests per second per core and that is why they're optimising Ruby. While that might be the end result and a commentary on contemporary software architecture as a whole, it does not tell you much about the performance of the Ruby part of the equation in isolation. They had bursts of 280 million rpm (4.6 million rps) with average of 2.8 million rps.

neonsunset 2y ago
> It does not tell you much about the performance of the Ruby part of the equation in isolation. Indeed, it doesn't. However, it would be a fairly safe bet to assume it was the slowest part of their architecture. I keep wondering how the numbers would change if Ruby were to be replaced with something else.
- phoronixrly 2y ago
  Since we're insinuating, I bet you that Ruby is not their chief bottleneck. You won't get much more RPS if you wait on an SQL query or RPC/HTTP API call. In my experience when you have a bottleneck in the actual Ruby code (not speaking about n+1s or heavy SQL queries or other IO), the code itself is written in such a way that it would be slow in whichever language. Again, in my experience this involves lots of (oft unnecessary) allocations and slow data transformations. Usually this is preceded by a slow heavy SQL query. You fix the query and get a speed-up of 0.8 rps to 40 rps, add a TODO entry "the following code needs to be refactored" but you already ran out of estimation and mark the issue as resolved. Couple of months later the optimization allowed the resultset to grow and the new bottleneck is memory use and the speed of the naive algorithm and lack of appropriate data structures in the data transformation step... Again in the same code you diligently TODOed... Tell me how this is Ruby's fault. Another example is one of the 'Oh we'll just introduce Redis-backed cache to finally make use of shared caching and alleviate the DB bottleneck'. Implementation and validation took weeks. Finally all tests are green. The test suite runs for half an hour longer. Issue was traced to latency to the Redis server and starvation due to locking between parallel workers. The task was quietly shelved afterwards without ever hitting production or being mentioned again in a prime example of learned helplessness. If only we had used an actual real programming language and not Ruby, we would not be hitting this issue (/s) I wish most performance problems would be solved by just using a """fast language"""...