6 ms·

This is going to sound bizarre, but why do webhooks and not just an AMQP queue? I get that receiving HTTP POSTs is easier, but it just seems better to setup a p

This is going to sound bizarre, but why do webhooks and not just an AMQP queue? I get that receiving HTTP POSTs is easier, but it just seems better to setup a publisher/subscriber relationship. That way, if a subscriber goes down, they can always catch up. And publishers can allow messages to sit in the queue with a TTL and max_size. It seems like a win-win for everyone.

elmigranto 10y ago
Though this is an approach worth considering for a bunch of local services, at least a subset of them, I don't see it working with 3rd party APIs. Consider something like Stripe — it is probably far more straightforward to invoke some HTTP endpoints, than to set up a huge infrastructure with millions persistent client connections.
- giuliano84 10y ago
  Agreed. I believe that AMQP connections are best suited for wide and articulated local environments. Moreover REST HTTP endpoints are today's esperanto :)
jbert 10y ago
One difference is in "dead connection detection". How do you know that your AMQP connection is down? At some level you're polling, whether that be TCP keepalive, application keepalive or something else. If you're doing polling, you're actually back at the same pre-webhook place - polling their server on some timescale which is a compromise between latency and load. Yes, a TCP keepalive is generally cheaper than an HTTP long poll request, but only by a constant factor.
jon-wood 10y ago
It's not AMQP (sadly) but something I've done previously is to have the actual webhook endpoint be as dumb as possible, doing nothing but accepting the payload (maybe with some very high level validation that the request was expected) and pushing it into a real queueing system. This means you can handle all sorts of failure modes, not just the backend going down, but also bugs in the consumer that would otherwise result in losing the request. I've not tried it, but I imagine this is a pretty good usecase for AWS Lambda as it's a small bit of glue code.
- fiatjaf 10y ago
  Shameless plug: https://requesthub.xyz https://requesthub.xyz is ideal for these cases.
  - jon-wood 10y ago
    That's pretty awesome, thanks for the tip.
- stephenr 10y ago
  I've used the same basic concept for accepting payment-received notifications (from a http redirect based payment gateway): Read the transaction ID from the request body, and store it (with a date/time) in a table. A periodic process later checks them, and uses the payment service's API to validate the payment is valid and take appropriate action.
niftich 10y ago
Because anyone can throw another route onto port 443 if you already host a website. A non-HTTP protocol running on a dedicated port, despite often being a superior solution, requires extra effort to set up, if the hosting environment even provides that option.
morgante 10y ago
It's a whole lot simpler to do secure cross-organization HTTP requests than it is to figure out how to have multiple AMQP subscribers from untrusted companies.

elmigranto 10y ago

Though this is an approach worth considering for a bunch of local services, at least a subset of them, I don't see it working with 3rd party APIs. Consider something like Stripe — it is probably far more straightforward to invoke some HTTP endpoints, than to set up a huge infrastructure with millions persistent client connections.

giuliano84 10y ago
Agreed. I believe that AMQP connections are best suited for wide and articulated local environments. Moreover REST HTTP endpoints are today's esperanto :)

jbert 10y ago

One difference is in "dead connection detection". How do you know that your AMQP connection is down? At some level you're polling, whether that be TCP keepalive, application keepalive or something else. If you're doing polling, you're actually back at the same pre-webhook place - polling their server on some timescale which is a compromise between latency and load. Yes, a TCP keepalive is generally cheaper than an HTTP long poll request, but only by a constant factor.

jon-wood 10y ago

It's not AMQP (sadly) but something I've done previously is to have the actual webhook endpoint be as dumb as possible, doing nothing but accepting the payload (maybe with some very high level validation that the request was expected) and pushing it into a real queueing system. This means you can handle all sorts of failure modes, not just the backend going down, but also bugs in the consumer that would otherwise result in losing the request. I've not tried it, but I imagine this is a pretty good usecase for AWS Lambda as it's a small bit of glue code.

fiatjaf 10y ago
Shameless plug: https://requesthub.xyz https://requesthub.xyz is ideal for these cases.
- jon-wood 10y ago
  That's pretty awesome, thanks for the tip.
stephenr 10y ago
I've used the same basic concept for accepting payment-received notifications (from a http redirect based payment gateway): Read the transaction ID from the request body, and store it (with a date/time) in a table. A periodic process later checks them, and uses the payment service's API to validate the payment is valid and take appropriate action.

niftich 10y ago

Because anyone can throw another route onto port 443 if you already host a website. A non-HTTP protocol running on a dedicated port, despite often being a superior solution, requires extra effort to set up, if the hosting environment even provides that option.

morgante 10y ago

It's a whole lot simpler to do secure cross-organization HTTP requests than it is to figure out how to have multiple AMQP subscribers from untrusted companies.