3 ms·

I'm shocked at how few HTTP libraries on GitHub properly handle exponential backoff, let alone retries. Do the Internet a favor, and file an issue with your fa

I'm shocked at how few HTTP libraries on GitHub properly handle exponential backoff, let alone retries. Do the Internet a favor, and file an issue with your favorite HTTP library asking them to implement exponential backoff. I haven't found anything in JS that does this properly though. Do people really just write apps that crap out upon the first HTTP request failure? The best library I have come across is actually SquareUp's OkHttp (the payment processing companies seem to be the only ones getting this right).

programd 10y ago
There's a decent Go library for various backoff algorithms. Useful if you're creating services which call webhooks: https://github.com/cenkalti/backoff https://github.com/cenkalti/backoff
jdc 10y ago
It seems to me that OkHttp doesn't in fact support a retry back off. https://github.com/square/okhttp/issues/2489 https://github.com/square/okhttp/issues/2489
daurnimator 10y ago
As the author of a http library (lua-http https://github.com/daurnimator/lua-http https://github.com/daurnimator/lua-http) that doesn't, I'm interested in how you'd want retries (not to mention exponential backoffs) to work: - Should they be the default? - What requests should be retried? (as much as we wish GETs were idempotent... they're not) see https://lists.w3.org/Archives/Public/ietf-http-wg/2017JanMar/0194.html https://lists.w3.org/Archives/Public/ietf-http-wg/2017JanMar... for an intro to the complexities here - What should get reused? (e.g. do new dns lookup? reuse connection? reuse proxy connection?) - How to handle non-replayable pieces? (e.g. request bodys coming from a fifo) - Usually a request has a timeout/deadline. should it be restarted for the retry? (probably not) I haven't implemented retries yet, as the above questions seem hard to answer without knowing application/server specific qualities (and hence should be left to the user of the http library). Please feel free to file an issue :)
- nebiyu 10y ago
  That list of questions is why I prefer retry and back-off to be separate from the internals of the http library. As a library user I need to work around the peculiarities of the particular service endpoints I'm integrating with. Backoff and retry aren't specific to the application/transport protocol through which a service is consumed. For example, (in the java world) the approach in libraries like hysterix, guava-retrying, and failsafe https://github.com/jhalterman/failsafe#retries https://github.com/jhalterman/failsafe#retries
  - Hydraulix989 10y ago
    What do you recommend for JS?
    - nebiyu 10y ago
      Libraries that provide similar abstractions in js do exist (https://github.com/tim-kos/node-retry https://github.com/tim-kos/node-retry and https://github.com/MathieuTurcotte/node-backoff https://github.com/MathieuTurcotte/node-backoff), but I'm not qualified to recommend anything. Most of the colleagues I've had have rolled their own one off solutions when needed.

programd 10y ago

There's a decent Go library for various backoff algorithms. Useful if you're creating services which call webhooks: https://github.com/cenkalti/backoff https://github.com/cenkalti/backoff

jdc 10y ago

It seems to me that OkHttp doesn't in fact support a retry back off. https://github.com/square/okhttp/issues/2489 https://github.com/square/okhttp/issues/2489

daurnimator 10y ago

As the author of a http library (lua-http https://github.com/daurnimator/lua-http https://github.com/daurnimator/lua-http) that doesn't, I'm interested in how you'd want retries (not to mention exponential backoffs) to work: - Should they be the default? - What requests should be retried? (as much as we wish GETs were idempotent... they're not) see https://lists.w3.org/Archives/Public/ietf-http-wg/2017JanMar/0194.html https://lists.w3.org/Archives/Public/ietf-http-wg/2017JanMar... for an intro to the complexities here - What should get reused? (e.g. do new dns lookup? reuse connection? reuse proxy connection?) - How to handle non-replayable pieces? (e.g. request bodys coming from a fifo) - Usually a request has a timeout/deadline. should it be restarted for the retry? (probably not) I haven't implemented retries yet, as the above questions seem hard to answer without knowing application/server specific qualities (and hence should be left to the user of the http library). Please feel free to file an issue :)

nebiyu 10y ago
That list of questions is why I prefer retry and back-off to be separate from the internals of the http library. As a library user I need to work around the peculiarities of the particular service endpoints I'm integrating with. Backoff and retry aren't specific to the application/transport protocol through which a service is consumed. For example, (in the java world) the approach in libraries like hysterix, guava-retrying, and failsafe https://github.com/jhalterman/failsafe#retries https://github.com/jhalterman/failsafe#retries
- Hydraulix989 10y ago
  What do you recommend for JS?
  - nebiyu 10y ago
    Libraries that provide similar abstractions in js do exist (https://github.com/tim-kos/node-retry https://github.com/tim-kos/node-retry and https://github.com/MathieuTurcotte/node-backoff https://github.com/MathieuTurcotte/node-backoff), but I'm not qualified to recommend anything. Most of the colleagues I've had have rolled their own one off solutions when needed.