3 ms·
In the AWS builders library they suggest setting the timeout to be at the p99 for the expected latency of the operation (or choose a different percentile if you
by hderms 6y ago
In the AWS builders library they suggest setting the timeout to be at the p99 for the expected latency of the operation (or choose a different percentile if you want to be more or less tolerant of false positives). That methodology seems pretty solid, provided it's something that's continually re-evaluated and tested under load.
Also important to consider what the client is advised to do in the case of a timeout. Retries, for instance should likely have backoff and jitter attached, or a retry budget.
- user5994461 6y agoThat number sounds like really bad advise to me. Should be more like 99.99% in my experience. Internal services have extremely low response time during normal operation (p99 around a second) but then the database will start a snapshot or a large analytics query hits on the week end (high IO) and the latency is through the roof for a short while. Too bad if services have short timeouts, they're all failing all requests now for no reason. p99 is normal operation. Services shouldn't be configured to systematically fail for 1% of operations.
- hderms 6y agofair enough, that's why they call out that you need to load test it and actually determine that the value you set meets expectations. Agreed that blindly setting a value is problematic