3 ms·

I would love to learn a little more here. So for 30 second tail latency cases, doesn't Ruby HTTP client support a timeout with cancellation? The only useful cas

I would love to learn a little more here. So for 30 second tail latency cases, doesn't Ruby HTTP client support a timeout with cancellation? The only useful case where I believe Async Ruby can help is the parallel requests cases. Disclaimer: I am not a full time Ruby person, so if it sounds naive please feel free to point it out.

kayodelycaon 5y ago
Ruby’s built in timeout library creates another thread with a sleep call. If something is blocking on a socket in the C code, the timeout doesn’t work.
- insensible 5y ago
  Right, don’t use that, but there are well-implemented timeouts in network libraries, etc.
- ioquatix 5y ago
  The fiber scheduler also hooks into the `Timeout` module to provide safe(r) timeouts using the event loop.
- jrochkind1 5y ago
  Is it true that a timeout wont' work on something blocking on a socket in C code? As I understand it, normally, a thread blocking on IO -- even in C code -- does not prevent context switching to another thread -- such as the timeout thread. But there are other reasons to avoid the stdlib timeout library, it's true; and also other ways to timeout on a network call, network call API's should offer their own timeout arguments normally.
BMorearty 5y ago
Yes, Ruby’s HTTP client supports timeout. As I mentioned, we weren’t using Ruby’s HTTP client, we were using Airbnb’s HTTP client with some additional capabilities. But it also supports timeout. The issue here was that I don’t think it would help to time out with cancellation and then try again (if that’s what you’re suggesting). Genesys API calls weren’t failing, they were just returning very slowly. A retry wasn’t necessarily likely to return any faster since the slow responses tended to happen in bursts, presumably based on something weird happening on their side. Meanwhile, phone calls continued coming in to Airbnb’s support center (this gem handles incoming calls) and we needed to continue routing those to Genesys as well. You mentioned “The only useful case where I believe Async Ruby can help is the parallel requests cases.” That’s what was happening here: because of the tail latency, we sometimes needed to have a lot more parallel requests. Does that clear it up?
- boundlessdreamz 5y ago
  How does falcon come into the picture here? Is this structured like a microservice between rest of airbnb and genesys, so that whatever needs to interact with genesys, interacts with this service instead and get a consistent latency?
  - BMorearty 5y ago
    The Airbnb service’s API is implemented over HTTPS. Falcon is built on top of Async. As its README says, “Each request is executed within a lightweight fiber and can block on up-stream requests without stalling the entire server process.” So whenever an API request comes into Airbnb’s service, Falcon creates a fiber to handle it in-process. If that fiber makes an API call to Genesys, control returns immediately to Falcon in the main fiber’s event loop to be able to handle more requests while the request fiber is awaiting Genesys’ response. I don’t know the details of where this service sits in the service diagram. I think it may actually be something Genesys calls, and it in turn sometimes calls back to Genesys. No, the point of this service is not to get consistent latency on Genesys API calls.

kayodelycaon 5y ago

Ruby’s built in timeout library creates another thread with a sleep call. If something is blocking on a socket in the C code, the timeout doesn’t work.

insensible 5y ago
Right, don’t use that, but there are well-implemented timeouts in network libraries, etc.
ioquatix 5y ago
The fiber scheduler also hooks into the `Timeout` module to provide safe(r) timeouts using the event loop.
jrochkind1 5y ago
Is it true that a timeout wont' work on something blocking on a socket in C code? As I understand it, normally, a thread blocking on IO -- even in C code -- does not prevent context switching to another thread -- such as the timeout thread. But there are other reasons to avoid the stdlib timeout library, it's true; and also other ways to timeout on a network call, network call API's should offer their own timeout arguments normally.

BMorearty 5y ago

Yes, Ruby’s HTTP client supports timeout. As I mentioned, we weren’t using Ruby’s HTTP client, we were using Airbnb’s HTTP client with some additional capabilities. But it also supports timeout. The issue here was that I don’t think it would help to time out with cancellation and then try again (if that’s what you’re suggesting). Genesys API calls weren’t failing, they were just returning very slowly. A retry wasn’t necessarily likely to return any faster since the slow responses tended to happen in bursts, presumably based on something weird happening on their side. Meanwhile, phone calls continued coming in to Airbnb’s support center (this gem handles incoming calls) and we needed to continue routing those to Genesys as well. You mentioned “The only useful case where I believe Async Ruby can help is the parallel requests cases.” That’s what was happening here: because of the tail latency, we sometimes needed to have a lot more parallel requests. Does that clear it up?

boundlessdreamz 5y ago
How does falcon come into the picture here? Is this structured like a microservice between rest of airbnb and genesys, so that whatever needs to interact with genesys, interacts with this service instead and get a consistent latency?
- BMorearty 5y ago
  The Airbnb service’s API is implemented over HTTPS. Falcon is built on top of Async. As its README says, “Each request is executed within a lightweight fiber and can block on up-stream requests without stalling the entire server process.” So whenever an API request comes into Airbnb’s service, Falcon creates a fiber to handle it in-process. If that fiber makes an API call to Genesys, control returns immediately to Falcon in the main fiber’s event loop to be able to handle more requests while the request fiber is awaiting Genesys’ response. I don’t know the details of where this service sits in the service diagram. I think it may actually be something Genesys calls, and it in turn sometimes calls back to Genesys. No, the point of this service is not to get consistent latency on Genesys API calls.