5 ms·

does anybody know some http client libraries that have scalable, performant asynchronous support (including transparent handling of various issues, like too man

does anybody know some http client libraries that have scalable, performant asynchronous support (including transparent handling of various issues, like too many requests pending etc.). A big plus would be if it would implement the CompletableFuture-interface (idk what it was called). Ideally with an easy, fluent api. I started with using unirest (http://unirest.io/java.html http://unirest.io/java.html), but it behaved poorly under load (also no CompletableFuture-interface implementation).

charleslmunger 10y ago
What's the interest in CompleteableFuture? It's probably my least favorite JDK addition...
- LeanderK 10y ago
  yeah, i feel like they messed up the api. easy integration into ratpack[1] non-blocking system. Propagation of the async-functionality through various classes without hardcoding an custom library-solution. [1] https://ratpack.io https://ratpack.io Also clumsy api, but performant for the required workloads.
  - charleslmunger 10y ago
    My biggest gripe isn't the massive API, it's the existence of the obtrude* methods - allowing a Future to change its completion result after it completes undermines the whole thing.
etaty 10y ago
http://doc.akka.io/docs/akka-http/10.0.0/java/http/client-side/index.html http://doc.akka.io/docs/akka-http/10.0.0/java/http/client-si...
shadowmatter 10y ago
Retrofit has support for RxJava, both of which are mentioned on the list. If Retrofit deserializes your JSON response payload as type T, then upon making a request it will return an Observable<T> that will either emit the result (if the request succeeds) or emit an Exception (if it fails). Most Android applications that rely on both Retrofit and RxJava use them together in this manner. As for too many requests pending, I'm not sure if OkHttp (which Retrofit uses for making requests and is again on that list) is configurable in that regard. If you want to handle it at the application level, you could maybe model the request queue to the network layer as an Observable<Request> and use RxJava's backpressure operators. For more information on functional reactive programming (which I would call promises or CompletableFuture done better), see http://reactivex.io/ http://reactivex.io/ And for more information on RxJava, see https://github.com/ReactiveX/RxJava https://github.com/ReactiveX/RxJava
carey 10y ago
Take it with a grain of salt because it's years since I used it, but the Jetty HTTP Client has a fluent API and can be used completely asynchronously. It doesn't use CompletableFuture, but it works with Java 8 lambdas and wouldn't be hard to adapt.
djKianoosh 10y ago
what did you notice with unirest under load? I've been using it a lot, and other than one or two little quirks I havent noticed perf issues. async calls can be a bit involved but I figured thats just java ;)
tbrowbdidnso 10y ago
Look at Netty. You need to know a good bit about the protocol to use it but there's nothing faster save compiled C for network access
tomfitz 10y ago
When I load tested async HTTP clients at the BBC (for 1000rps), both https://hc.apache.org/httpcomponents-asyncclient-dev/ https://hc.apache.org/httpcomponents-asyncclient-dev/ and https://github.com/AsyncHttpClient/async-http-client/ https://github.com/AsyncHttpClient/async-http-client/ performed well. We used Apache's since it exposes connection pool statistics. Apache's client doesn't have CompletableFuture, but it was easy to add one: https://gist.github.com/tomfitzhenry/4bb032f6a9f56d95f6fb54454e617fc1 https://gist.github.com/tomfitzhenry/4bb032f6a9f56d95f6fb544... async-http-client is the HTTP client used by Gatling (same developer, in fact).

charleslmunger 10y ago

What's the interest in CompleteableFuture? It's probably my least favorite JDK addition...

LeanderK 10y ago
yeah, i feel like they messed up the api. easy integration into ratpack[1] non-blocking system. Propagation of the async-functionality through various classes without hardcoding an custom library-solution. [1] https://ratpack.io https://ratpack.io Also clumsy api, but performant for the required workloads.
- charleslmunger 10y ago
  My biggest gripe isn't the massive API, it's the existence of the obtrude* methods - allowing a Future to change its completion result after it completes undermines the whole thing.

etaty 10y ago

http://doc.akka.io/docs/akka-http/10.0.0/java/http/client-side/index.html http://doc.akka.io/docs/akka-http/10.0.0/java/http/client-si...

shadowmatter 10y ago

Retrofit has support for RxJava, both of which are mentioned on the list. If Retrofit deserializes your JSON response payload as type T, then upon making a request it will return an Observable<T> that will either emit the result (if the request succeeds) or emit an Exception (if it fails). Most Android applications that rely on both Retrofit and RxJava use them together in this manner. As for too many requests pending, I'm not sure if OkHttp (which Retrofit uses for making requests and is again on that list) is configurable in that regard. If you want to handle it at the application level, you could maybe model the request queue to the network layer as an Observable<Request> and use RxJava's backpressure operators. For more information on functional reactive programming (which I would call promises or CompletableFuture done better), see http://reactivex.io/ http://reactivex.io/ And for more information on RxJava, see https://github.com/ReactiveX/RxJava https://github.com/ReactiveX/RxJava

carey 10y ago

Take it with a grain of salt because it's years since I used it, but the Jetty HTTP Client has a fluent API and can be used completely asynchronously. It doesn't use CompletableFuture, but it works with Java 8 lambdas and wouldn't be hard to adapt.

djKianoosh 10y ago

what did you notice with unirest under load? I've been using it a lot, and other than one or two little quirks I havent noticed perf issues. async calls can be a bit involved but I figured thats just java ;)

tbrowbdidnso 10y ago

Look at Netty. You need to know a good bit about the protocol to use it but there's nothing faster save compiled C for network access

tomfitz 10y ago

When I load tested async HTTP clients at the BBC (for 1000rps), both https://hc.apache.org/httpcomponents-asyncclient-dev/ https://hc.apache.org/httpcomponents-asyncclient-dev/ and https://github.com/AsyncHttpClient/async-http-client/ https://github.com/AsyncHttpClient/async-http-client/ performed well. We used Apache's since it exposes connection pool statistics. Apache's client doesn't have CompletableFuture, but it was easy to add one: https://gist.github.com/tomfitzhenry/4bb032f6a9f56d95f6fb54454e617fc1 https://gist.github.com/tomfitzhenry/4bb032f6a9f56d95f6fb544... async-http-client is the HTTP client used by Gatling (same developer, in fact).