4 ms·

Good luck generating Prime Day load artificially.

sitkack 8y ago
Please, generating load is always easier than servicing it.
HankB99 8y ago
Could you do something like that from another cloud service? I suppose the difference would be whether 10K requests from one IP address would be the same as 10K requests coming from 10K IP addresses. The further you move from the actual production load, the higher risk that the test doesn't test everything. For example if the network could only handle connections from 5k hosts then the former could pass while the latter failed miserably.
pageald 8y ago
Simulating millions of users should be well within the capabilities of a company as large as Amazon. Off the shelf load testing tools like Locust can create thousands of fake users with one worker.
jjirsa 8y ago
(◔_◔)
greenleafjacob 8y ago
At Uber we invested in synthetic load tools that create fake riders and drivers, match them, etc and test our entire dispatch system end to end to arbitrary amounts of online driver and riders. I don’t see why they couldn’t do the same with carts, adding products to carts, etc
- wiredone 8y ago
  They do. But I'm guessing you don't scale up 1M+ servers for your Uber canary traffic tests - these are some of the scales Amazon undergoes in these events. The scale is unlike almost any other web property around.
  - greenleafjacob 8y ago
    I’m not sure how it’s relevant. If you have the infrastructure to send 1000 concurrent users you can probably send 1M concurrent users. We only test small integer multiples of our peak traffic, and if your absolute number of servers to service that is in the millions then it would make absolute sense to be routinely running that capacity test. If that means “scale up 1M+ servers” then that is what you have to do, otherwise how can you be sure?
  - BinaryIdiot 8y ago
    > But I'm guessing you don't scale up 1M+ servers Do they really need 1 million servers? Many of my friends who work at other tech companies need such few servers in comparison even with significantly high traffic that just screams massive inefficiencies...which seems wrong. But I've never worked at Amazon so I wouldn't know.
    - deegles 8y ago
      Judging by all the error pages after launch, they needed more!
    - convivialdingo 8y ago
      At one of my previous companies we managed about 100 servers for peaks of about 30 million users. But we didn’t handle free-form text search like Amazon. I can imagine that would necessitate a huge scaling up of compute and data.
jtchang 8y ago
It's actually pretty trivial. Getting the right parameters for the load will be hard and making sure you are loading it properly. Think about DDOS attacks. Generating the load is rarely the issue.
yeukhon 8y ago
Probably can find a hotspot and load test on the hospot.

Please, generating load is always easier than servicing it.

Could you do something like that from another cloud service? I suppose the difference would be whether 10K requests from one IP address would be the same as 10K requests coming from 10K IP addresses. The further you move from the actual production load, the higher risk that the test doesn't test everything. For example if the network could only handle connections from 5k hosts then the former could pass while the latter failed miserably.

pageald 8y ago

Simulating millions of users should be well within the capabilities of a company as large as Amazon. Off the shelf load testing tools like Locust can create thousands of fake users with one worker.

jjirsa 8y ago

(◔_◔)

greenleafjacob 8y ago

At Uber we invested in synthetic load tools that create fake riders and drivers, match them, etc and test our entire dispatch system end to end to arbitrary amounts of online driver and riders. I don’t see why they couldn’t do the same with carts, adding products to carts, etc

wiredone 8y ago
They do. But I'm guessing you don't scale up 1M+ servers for your Uber canary traffic tests - these are some of the scales Amazon undergoes in these events. The scale is unlike almost any other web property around.
- greenleafjacob 8y ago
  I’m not sure how it’s relevant. If you have the infrastructure to send 1000 concurrent users you can probably send 1M concurrent users. We only test small integer multiples of our peak traffic, and if your absolute number of servers to service that is in the millions then it would make absolute sense to be routinely running that capacity test. If that means “scale up 1M+ servers” then that is what you have to do, otherwise how can you be sure?
- BinaryIdiot 8y ago
  > But I'm guessing you don't scale up 1M+ servers Do they really need 1 million servers? Many of my friends who work at other tech companies need such few servers in comparison even with significantly high traffic that just screams massive inefficiencies...which seems wrong. But I've never worked at Amazon so I wouldn't know.
  - deegles 8y ago
    Judging by all the error pages after launch, they needed more!
  - convivialdingo 8y ago
    At one of my previous companies we managed about 100 servers for peaks of about 30 million users. But we didn’t handle free-form text search like Amazon. I can imagine that would necessitate a huge scaling up of compute and data.

jtchang 8y ago

It's actually pretty trivial. Getting the right parameters for the load will be hard and making sure you are loading it properly. Think about DDOS attacks. Generating the load is rarely the issue.

yeukhon 8y ago

Probably can find a hotspot and load test on the hospot.