3 ms·

I agree, it does seem awfully complicated and there are quite a few pieces missing for this to be a complete solution. I'm a bit surprised about the scalabilit

I agree, it does seem awfully complicated and there are quite a few pieces missing for this to be a complete solution. I'm a bit surprised about the scalability case against a simpler solution. This is not about Shopify's scale. We're talking about contention for a specific SKU of a specific seller at a specific warehouse location. How many shopping carts are competing for a single SKU at the payment stage at peak hours? Can this really be too much lock contention for a single database row? I realise Shopify engineers are neither stupid nor inexperienced. Hence my surprise. I would have liked to hear more about that specific problem.

kevincox 2mo ago
Flash sales are a huge scaling issue for Shopify. There are celebrities who want to sell thousands of items in a few minutes window at the end of an advertised countdown. Basically this is an incredibly rare case but a feature that they want to support.
- fauigerzigerk 2mo ago
  Makes sense.
- cowsandmilk 2mo ago
  There were woot offs 20 years ago. Flash sales are not some new scaling issue.
  - risyachka 2mo ago
    Its also not a solved one considering basically no platforms support them.
    - malfist 2mo ago
      It's not? Shopify does it, woot does it, amazon does it, yes style does it, royal Caribbean does it. Seems like a lot of places can do it for it not to be a solved problem.
misiek08 2mo ago
We observe 200+ (can’t say closer number) purchases per second for single SKU with good marketing and price. The other thing that bothers me - why not real stable, but maybe „too old, medieval” solution with Redis as the main source of through - without any sync with SQL at all in terms of stock… it worked in my previous job with much higher traffic (1000s/s). Yup, we ended up with app-side sharding, but it was stupid-simple.
- theptip 2mo ago
  You need a sql DB for the actual purchase transaction, you can’t keep financial records in Redis. As they say in TFA they used Redis for the cart reservation but then you need to sync the two stores.
sgarland 2mo ago
> Can this really be too much lock contention for a single database row? When you consider how many other (often poorly-crafted) DB queries and service calls are being performed while holding the row locked, yes, it can rapidly add up. If the entire pipeline takes 500 msec, congratulations, you can’t sell more than 2 units per second of that SKU. If the item is popular and being actively hyped, then yes, that can be a problem.

kevincox 2mo ago

Flash sales are a huge scaling issue for Shopify. There are celebrities who want to sell thousands of items in a few minutes window at the end of an advertised countdown. Basically this is an incredibly rare case but a feature that they want to support.

fauigerzigerk 2mo ago
Makes sense.
cowsandmilk 2mo ago
There were woot offs 20 years ago. Flash sales are not some new scaling issue.
- risyachka 2mo ago
  Its also not a solved one considering basically no platforms support them.
  - malfist 2mo ago
    It's not? Shopify does it, woot does it, amazon does it, yes style does it, royal Caribbean does it. Seems like a lot of places can do it for it not to be a solved problem.

misiek08 2mo ago

We observe 200+ (can’t say closer number) purchases per second for single SKU with good marketing and price. The other thing that bothers me - why not real stable, but maybe „too old, medieval” solution with Redis as the main source of through - without any sync with SQL at all in terms of stock… it worked in my previous job with much higher traffic (1000s/s). Yup, we ended up with app-side sharding, but it was stupid-simple.

theptip 2mo ago
You need a sql DB for the actual purchase transaction, you can’t keep financial records in Redis. As they say in TFA they used Redis for the cart reservation but then you need to sync the two stores.

sgarland 2mo ago

> Can this really be too much lock contention for a single database row? When you consider how many other (often poorly-crafted) DB queries and service calls are being performed while holding the row locked, yes, it can rapidly add up. If the entire pipeline takes 500 msec, congratulations, you can’t sell more than 2 units per second of that SKU. If the item is popular and being actively hyped, then yes, that can be a problem.