3 ms·
Because of the nature of Heroku (and thus EC2) there will always be unknown and random latency associated with shared nodes and hidden network utilization. Wha
by evanphx 13y ago
Because of the nature of Heroku (and thus EC2) there will always be unknown and random latency associated with shared nodes and hidden network utilization.
What's important is that you did all the tests on heroku and gave enough of a window to subject all tests to the random latency.
This allows users to say "if I'm going to be deploying on Heroku, these measurements are valid." If they deploy anywhere else, those variances will be different and change the equation (though it's unlikely they'd completely flip the results).
Thanks a ton for the great analysis!
- jrochkind1 13y agoThanks! And thanks for puma. I think I actually probably DIDN'T give quite enough time to give enough window to even out the env -- I generally only ran each scenario test for around 30-60 seconds (you can see how long each test ran for in the results, it's there in the captured output). Still, by running so many tests and getting _largely_ consistent results, I think it prob averages out into giving us a basic picture, and would be surprised if results weren't basically consistent even if you ran longer. But if someone wanted to re-run it giving dozens of minutes or more to each scenario, hey, you've got my code, heh. It would probably take a day or more to run everything, but such a person wouldn't have to spend the time I took setting it all up if they used my code.