3 ms·

But, but... Rails doesn't scale!!!!!

hschne 3y ago
You beat me to it. In all seriousness though, I'd love to know more about Shopify's infrastructure. Pretty marvelous how they handle such an enormous influx of load.
ndriscoll 3y ago
The dashboard shows 25k orders per minute right now (~400/s). A modern server shouldn't even be close to 1% utilization to handle that.
- qiller 3y ago
  While 400 TPS might not be huge, there is still a lot of processing that goes into placing each order, and it's easy to underestimate the necessary capacity. Latencies add up (a call to a tax service, a call to a shipping service, etc), state adds up, and suddenly it's way over 1%.
  - ndriscoll 3y ago
    Latency doesn't affect throughput. As soon as you fire off a call to an external vendor, your server can work on processing the next request. It doesn't matter if that call takes 10 ms or 5 seconds. Even if it took 30 seconds, that means you need to be able to handle 12k concurrent requests (almost all of which are asleep), which is nothing.
- ZephyrBlu 3y ago
  Creating an order isn't a stateless process.
- onion2k 3y ago
  Are you assuming that 1 order is one inbound request, one database transaction, and one outbound response? Because it isn't that. You need to integrate with address verification services, fraud detection, payment services, stock management, communications like email and alerting, telemetry, and more. It's all manageable and a lot of it can be queued, but it's still hard.
  - ndriscoll 3y ago
    I'm not, otherwise something like 0.1% would be a reasonable upper bound. Even if you figure each order involves 20 async steps, that would be ~8k steps per second, which is nothing for a modern server. My old i5 can do 70k simple requests/second (the first type you describe) on the JVM with the app and database on the same computer (so contending for CPU time). A modern server CPU is ~15x more capable than mine without getting into multi-socket. Put another way, a dual socket Xeon platinum can give you 120 cores. To do 400 orders/second on such a machine, you need to do less than 300 ms of core-time per order, which is an insanely large number. The things you mentioned spend almost all of their time waiting.
- Scarbutt 3y ago
  I can build that in a weekend!
  - maronato 3y ago
    Why even have a whole system for it? Just expose an FTP server and let users edit an “orders.txt” file
- quadrature 3y ago
  1) Orders are complex and require many steps to process correctly. 2) Orders are only the writes, we have to serve many magnitudes more read traffic and not all of it is cached (invalidations, inventory counts).
  - ndriscoll 3y ago
    1. Yes, of course. 2. How many is "many" magnitudes? The word suggests at least 4, but if so, it'd make more sense to brag about that. And anything is cacheable if it doesn't affect transactional logic. If the user is looking at a page for 10 seconds, are they going to care that the inventory count is 3 seconds old? At your many orders of magnitude, caching for a couple seconds would give you hits almost all of the time for popular pages. Also, aren't inventory counts that are displayed to users typically completely made up? Why would you make something like that dynamic and accurate in a high load system? The only reason to have it is to create FOMO or for b2b bulk ordering, and the latter is a whole different thing.
charcircuit 3y ago
This page doesn't answer that question because it doesn't include how many servers are being used to handle it.
Scarbutt 3y ago
It also could be that something like Java/Go/Rust is doing the hard work.
- burke 3y ago
  I’m not sure what the globe is written but I’m pretty sure the comment is referring to the shopify core platform scaling to that many dollars of throughput, and while, yes, certainly there are peripheral bits in a variety of languages, the core is still very much a rails project.

You beat me to it. In all seriousness though, I'd love to know more about Shopify's infrastructure. Pretty marvelous how they handle such an enormous influx of load.

ndriscoll 3y ago

The dashboard shows 25k orders per minute right now (~400/s). A modern server shouldn't even be close to 1% utilization to handle that.

qiller 3y ago
While 400 TPS might not be huge, there is still a lot of processing that goes into placing each order, and it's easy to underestimate the necessary capacity. Latencies add up (a call to a tax service, a call to a shipping service, etc), state adds up, and suddenly it's way over 1%.
- ndriscoll 3y ago
  Latency doesn't affect throughput. As soon as you fire off a call to an external vendor, your server can work on processing the next request. It doesn't matter if that call takes 10 ms or 5 seconds. Even if it took 30 seconds, that means you need to be able to handle 12k concurrent requests (almost all of which are asleep), which is nothing.
ZephyrBlu 3y ago
Creating an order isn't a stateless process.
onion2k 3y ago
Are you assuming that 1 order is one inbound request, one database transaction, and one outbound response? Because it isn't that. You need to integrate with address verification services, fraud detection, payment services, stock management, communications like email and alerting, telemetry, and more. It's all manageable and a lot of it can be queued, but it's still hard.
- ndriscoll 3y ago
  I'm not, otherwise something like 0.1% would be a reasonable upper bound. Even if you figure each order involves 20 async steps, that would be ~8k steps per second, which is nothing for a modern server. My old i5 can do 70k simple requests/second (the first type you describe) on the JVM with the app and database on the same computer (so contending for CPU time). A modern server CPU is ~15x more capable than mine without getting into multi-socket. Put another way, a dual socket Xeon platinum can give you 120 cores. To do 400 orders/second on such a machine, you need to do less than 300 ms of core-time per order, which is an insanely large number. The things you mentioned spend almost all of their time waiting.
Scarbutt 3y ago
I can build that in a weekend!
- maronato 3y ago
  Why even have a whole system for it? Just expose an FTP server and let users edit an “orders.txt” file
quadrature 3y ago
1) Orders are complex and require many steps to process correctly. 2) Orders are only the writes, we have to serve many magnitudes more read traffic and not all of it is cached (invalidations, inventory counts).
- ndriscoll 3y ago
  1. Yes, of course. 2. How many is "many" magnitudes? The word suggests at least 4, but if so, it'd make more sense to brag about that. And anything is cacheable if it doesn't affect transactional logic. If the user is looking at a page for 10 seconds, are they going to care that the inventory count is 3 seconds old? At your many orders of magnitude, caching for a couple seconds would give you hits almost all of the time for popular pages. Also, aren't inventory counts that are displayed to users typically completely made up? Why would you make something like that dynamic and accurate in a high load system? The only reason to have it is to create FOMO or for b2b bulk ordering, and the latter is a whole different thing.

charcircuit 3y ago

This page doesn't answer that question because it doesn't include how many servers are being used to handle it.

Scarbutt 3y ago

It also could be that something like Java/Go/Rust is doing the hard work.

burke 3y ago
I’m not sure what the globe is written but I’m pretty sure the comment is referring to the shopify core platform scaling to that many dollars of throughput, and while, yes, certainly there are peripheral bits in a variety of languages, the core is still very much a rails project.