3 ms·

> If it's the real-deal, and not like people saying "Bun.js can serve 65k req/s+ (cough cough to localhost,)" that's impressive. Not all req/s are made the sam

> If it's the real-deal, and not like people saying "Bun.js can serve 65k req/s+ (cough cough to localhost,)" that's impressive. Not all req/s are made the same. Amazon search is made of 100s of services, and Amazon's search page loads 20 products per page, that means 80k search req/s translates to 1.6 MM product API req/s for example. FWIW a search request at Amazon hits roughly 100 unique search clusters (think of this as ElasticSearch clusters - but its not ES), with different product groupings in each cluster. Each cluster is made up of 1000s of nodes running Lucene (think similar to ES shards). This is just for the "match set", i.e. which products to return. Then there are services to re-sort those matched products based on popularity, likelihood of purchase, etc. Think giant ML models. Then there are product lookups. Before all of this, there is Query analysis to simplify/improve the query (think giant ML models) to classify "Apple" into electronics vs groceries based on the other keywords and your current context. Meanwhile, Bun.js is talking about 65k "hello world" type req/s. The compute per req is magnitudes different.

Tomis02 3y ago
> Not all req/s are made the same. > Amazon search is made of 100s of services, and Amazon's search page loads 20 products per page, that means 80k search req/s translates to 1.6 MM product API req/s for example. That's their problem, no? Nobody's forcing them to have an architecture where a request propagates to hundreds of services.
- dmoy 3y ago
  > That's their problem, no? Nobody's forcing them to have an architecture where a request propagates to hundreds of services. Well, Jeff Bezos circa 2002 or so did: https://news.ycombinator.com/item?id=3102800 https://news.ycombinator.com/item?id=3102800 It's hard to remove that ethos now
  - hughesjj 3y ago
    And thank God he did, amazon would have direct without the SOA enforcement. Scale issues are emergent issues. It's a phase transition.
- txcwpalpha 3y ago
  Why is it anyone's "problem"? Nobody said they're being forced to do it this way - just that they are. And I guarantee that there are thousands of other companies out there that have an API-fanout model as well, and might be interested in how Amazon does it. I don't get the hostility around this article. Nobody is forcing you to read it or to do it this way. If your system is architected in a different way where you can run your whole system on a single instance, then good for you! But Amazon presumably doesn't have that luxury, and others may not either.
- yazaddaruvala 3y ago
  > That's their problem, no? Nobody's forcing them to have an architecture where a request propagates to hundreds of services. I mean, physics kinda is. Just the volume of data that needs to be hosted, needs multiple nodes. ElasticSearch has some good general documentation of search engines if you want to learn more. In Amazon's case, a general query will hit a fanout of about 10,000x nodes - I wasn't even counting that fanout because it is all technically 1 service. Meanwhile, Bun.js (no offense to Bun) is built for IO-bounded workloads and 65k req/s is great for that. However, executing the required Natural Language Processing (i.e. multiple ML models) would result in an exhausted CPU (even if Bun could distribute the compute across cores or compute the ML inference on a GPU). I'd be willing to bet even on a great CPU it gets throttled at 10 req/s (at most 100 req/s). This really isn't some "They should have just done it all in Postgress, Bun.js can just use a nice ORM and be IO-bounded" type situation - which is a philosophy I very much agree with for 99.9999% of usecases.
- NBJack 3y ago
  I suspect it grew into that. Amazon has been around for almost 30 years. That's a long time to incrementally change a service ecosystem.
- yathaid 3y ago
  My favorite kind of HN answer. 100% snark, 100% certain, 100% wrong. So, instead of that, what would you have? A single, binary that makes multiple DB calls? Hmmm, let's see, some people have tried that and written extensively about the problems they faced doing that. Wait, one of them is actually a small e-commerce firm named after a larger river. Wonder what the issues were ...

Tomis02 3y ago

> Not all req/s are made the same. > Amazon search is made of 100s of services, and Amazon's search page loads 20 products per page, that means 80k search req/s translates to 1.6 MM product API req/s for example. That's their problem, no? Nobody's forcing them to have an architecture where a request propagates to hundreds of services.

dmoy 3y ago
> That's their problem, no? Nobody's forcing them to have an architecture where a request propagates to hundreds of services. Well, Jeff Bezos circa 2002 or so did: https://news.ycombinator.com/item?id=3102800 https://news.ycombinator.com/item?id=3102800 It's hard to remove that ethos now
- hughesjj 3y ago
  And thank God he did, amazon would have direct without the SOA enforcement. Scale issues are emergent issues. It's a phase transition.
txcwpalpha 3y ago
Why is it anyone's "problem"? Nobody said they're being forced to do it this way - just that they are. And I guarantee that there are thousands of other companies out there that have an API-fanout model as well, and might be interested in how Amazon does it. I don't get the hostility around this article. Nobody is forcing you to read it or to do it this way. If your system is architected in a different way where you can run your whole system on a single instance, then good for you! But Amazon presumably doesn't have that luxury, and others may not either.
yazaddaruvala 3y ago
> That's their problem, no? Nobody's forcing them to have an architecture where a request propagates to hundreds of services. I mean, physics kinda is. Just the volume of data that needs to be hosted, needs multiple nodes. ElasticSearch has some good general documentation of search engines if you want to learn more. In Amazon's case, a general query will hit a fanout of about 10,000x nodes - I wasn't even counting that fanout because it is all technically 1 service. Meanwhile, Bun.js (no offense to Bun) is built for IO-bounded workloads and 65k req/s is great for that. However, executing the required Natural Language Processing (i.e. multiple ML models) would result in an exhausted CPU (even if Bun could distribute the compute across cores or compute the ML inference on a GPU). I'd be willing to bet even on a great CPU it gets throttled at 10 req/s (at most 100 req/s). This really isn't some "They should have just done it all in Postgress, Bun.js can just use a nice ORM and be IO-bounded" type situation - which is a philosophy I very much agree with for 99.9999% of usecases.
NBJack 3y ago
I suspect it grew into that. Amazon has been around for almost 30 years. That's a long time to incrementally change a service ecosystem.
yathaid 3y ago
My favorite kind of HN answer. 100% snark, 100% certain, 100% wrong. So, instead of that, what would you have? A single, binary that makes multiple DB calls? Hmmm, let's see, some people have tried that and written extensively about the problems they faced doing that. Wait, one of them is actually a small e-commerce firm named after a larger river. Wonder what the issues were ...