2 ms·
> I wonder if they use exponential backoff? More importantly, did they use randomised exponential backoff? Having all the retires hitting at the same time can
by bbrazil 11y ago
> I wonder if they use exponential backoff?
More importantly, did they use randomised exponential backoff?
Having all the retires hitting at the same time can lead to a pulses of outages until things settle down.
- jmason23 11y agoI have no doubt they would have been using randomised exponential backoff -- these issues are well recognised inside Amazon and best practices are well known. For example, here's a blog post from March this year from Marc Brooker on the topic: http://www.awsarchitectureblog.com/2015/03/backoff.html http://www.awsarchitectureblog.com/2015/03/backoff.html . It may not have been correctly tuned for this scenario however. I'd say the Dynamo team are well aware of what they should have been doing, and kicking themselves for not foreseeing this cascading-failure case. ouch!