8 ms·

Webhooks Are Magic

103 points by paddyforan 13y ago

duaneb 13y ago
Ok, maybe it's just the terminology that's confusing, but how do webhooks differ from callback requests? Is it that the semantics are standardized, or is there something that I'm missing? The Author is quite excited about them.
- paddyforan 13y ago
  I'm incredibly excited about them. As far as I'm concerned, webhooks and callback requests are the same. If we were going to get pedantic, I suppose we could say that for a request to be a callback, it would have to be a response to an earlier request (think asynchronous processing) whereas a webhook is an evented request (it wasn't prompted by a previous request). But being pedantic is lame. Let's just be excited about both! Relevant: https://twitter.com/paddyforan/statuses/254375310554968064 https://twitter.com/paddyforan/statuses/254375310554968064
adelevie 13y ago
Great post, Paddy. One thing I'd love to see in the HUD is a GitHub OAuth + repo webhook config. That is, you log in with GitHub credentials, you see a list of repos and can select which workers will get POSTed to after specific GitHub repo events[1]. Examples: - The "adelevie/MyJekyllBlog" repo will POST to the worker "BuildAndDeploy" after event "push." - The "adelevie/MyRailsApp" repo will POST to the worker "TestMyRailsApp" after events "push" and "pull_request." [1] http://developer.github.com/v3/repos/hooks/ http://developer.github.com/v3/repos/hooks/
- paddyforan 13y ago
  I love this idea. I've been wanting to do something similar to this, making webhooks even easier to hook into the Iron.io platform. I don't know if it will make it into HUD, but I'm definitely going to be trying to setup some push-button webhook configurations for popular services that support configuring webhooks from an API.
  - adelevie 13y ago
    Will/can there be an Iron.io OAuth API?
    - paddyforan 13y ago
      "We have nothing to announce at this time." I'd like there to be is all I've got for you at the moment. :( Priorities, and all that.
carimura 13y ago
The next coming of SOA!
zrail 13y ago
Stripe's webhooks are really nice. Everything that happens with your account, be it charges, disputes, transfers, or people changing subscription levels, all generate events that you can receive and do something with. In my book and the app that sells it[1] I use them to trigger emails to myself when important things happen. There's also an iPhone app named Paypad[2] that registers a webhook on your account that sends you push notifications when things happen. [1]: https://www.petekeen.net/mastering-modern-payments https://www.petekeen.net/mastering-modern-payments [2]: https://www.pay-pad.com https://www.pay-pad.com
- baudehlo 13y ago
  Just don't disable SSL on your server. http://baudehlo.wordpress.com/2013/09/06/stripe-webhooks-dont-work-with-tls/ http://baudehlo.wordpress.com/2013/09/06/stripe-webhooks-don...
uptown 13y ago
I'm new to webhooks - I understand them conceptually, but is there a standard for authenticating the incoming notification? Take the Stripe example - how do you know that the HTTP POST is actually coming from Stripe? Do you use a shared-secret, or some other mechanism to verify the HTTP request is authentic?
- paddyforan 13y ago
  Great question! For Iron.io, all our requests are authenticated using an OAuth token. So only people that know your OAuth token (and thus could use our API anyways) can use your webhook endpoints. But there are other ways, too. The most common I've seen is to provide an API endpoint to verify events with (Stripe does this). If you use HTTPS to receive the webhook, and verify it with a request over HTTPS to the API you expect it to be coming from, you're ensuring the request is authentic. For some webhook styles, it doesn't actually matter. Some people use webhooks to just say "Something happened", without actually saying what. In this style, the API is still responsible for the data and authentication, the webhook just says "Hey, wake up, the API has something new for you."
  - uptown 13y ago
    Got it. Thanks for the initial write-up, and your explanation.
- zrail 13y ago
  With Stripe, the process is: 1) Receive a webhook and record the ID in your database. If it's a duplicate, stop. 2) Ask Stripe for the event matching the ID they gave you. If they reject it, stop. 3) Process the event. It requires a roundtrip, but it's the only sane way to validate that the event you got is real and that you haven't seen it before without trying to validate a signature (lots of other APIs make you do that, though)
  - rryan 13y ago
    Yes! This is the only part of Stripe's API that I raised an eyebrow at. I think it's a little irresponsible to send all the data via web-hook without providing a way to verify it came from Stripe. At the very least they should only provide the event ID over webhook otherwise people will take the lazy route and trust trivially forgeable messages.
mwetzler 13y ago
Love the post... and stripe webhooks! Integrated them with Keen IO in about 60 seconds and even made a (perhaps overly enthusiastic) blog post about it. https://keen.io/blog/48628875285/integrate-stripe-and-keen-io-in-60-seconds https://keen.io/blog/48628875285/integrate-stripe-and-keen-i...
- paddyforan 13y ago
  Awesome!
aytekin 13y ago
If you have an API make sure webhooks can be created using the API. It is win-win for everyone. It is easier to implement and apps don't have to poll you constantly.
- zrail 13y ago
  There was a somewhat-confusing attempt at standardizing that pattern a few weeks ago: http://resthooks.org http://resthooks.org
huxley 13y ago
I've started adding web hooks to one of my Django web apps with Zapier's django-rest-hooks which I'm liking a lot: https://github.com/zapier/django-rest-hooks https://github.com/zapier/django-rest-hooks
richardlblair 13y ago
Webhooks are great, until you don't get one... Then they suddenly become a nightmare.
- paddyforan 13y ago
  That's why logs from the webhook provider are important. :)
  - jessaustin 13y ago
    Yuck. Who wants to deal with that? b^)
  - philsnow 13y ago
    That sounds even worse than: > "I watch people poll APIs or create convoluted connections, and I cry a little on the inside." If the action being sent by the webhook is really important, there seems to be no way around doing it somewhat like this: Write a polling (or hanging-get) system, but use webhooks as an unreliable but lower-latency push mechanism. In this setup, webhooks give you low-latency updates (when they work), and polling ensures completeness.
    - drivebyacct2 13y ago
      Sounds like a reinvention of Pubsubhubbub anyway.
    - roarroar 13y ago
      Depending on your requirements the remote end can be responsible for ensuring that you receive and acknowledge the wake up. Once you hit your own timeout you might do any number of things, not necessarily repoll the other end.
- philsnow 13y ago
  I came across http://blog.iron.io/2013/01/queue-webhook-events-with-ironmq.html http://blog.iron.io/2013/01/queue-webhook-events-with-ironmq... This is pretty sweet, but it seems like it might be more useful for the baked-in reliability to be on the remote end. That is, the side sending the POSTs should notice that it didn't get a 200 reply and retry some number of times with backoff. Also, the POSTs need to be idempotent (include a sequence number or nonce or something) so that if the "200 OK" gets lost and the sending side retries, you can discard the duplicates on the receiving side. That's of course a lot harder since you have to convince everybody whose service you want to use that this is worth the time/effort :)
samograd 13y ago
> It makes the web capable of pushing information to its users, instead of waiting for users to ask for information. http://www.wired.com/wired/archive/5.03/ff_push.html http://www.wired.com/wired/archive/5.03/ff_push.html [1997] It seems I'm not the only one to remember the promises of this prophecy: http://www.boston.com/business/technology/articles/2004/01/05/push_technology_gets_a_nudge/ http://www.boston.com/business/technology/articles/2004/01/0...
mikeknoop 13y ago
Webhooks are awesome but please don't make your users set them up: https://resthooks.org https://resthooks.org
- adventured 13y ago
  Firefox is throwing a "This Connection is Untrusted" alert on attempting to load resthooks.org via ssl. Someone should probably look into that.
  - e12e 13y ago
    https://resthooks.herokuapp.com/ https://resthooks.herokuapp.com/ appears to be the meat of it. But that is really just a pointer to: https://github.com/net-engine/resthooks/blob/master/README.md https://github.com/net-engine/resthooks/blob/master/README.m...
jkarneges 13y ago
http://webhookinbox.com/ http://webhookinbox.com/ gives you throwaway URLs for receiving webhooks. Good for testing.
- paddyforan 13y ago
  John Sheehan, of Runscope.com (which, by the way, is a super rad tool for API developers) did a great roundup on receiving webhooks for testing: http://john-sheehan.com/blog/a-survey-of-the-localhost-proxying-landscape http://john-sheehan.com/blog/a-survey-of-the-localhost-proxy...
tootie 13y ago
This is basically pub/sub messaging, but over the internet?
- dragonwriter 13y ago
  > This is basically pub/sub messaging, but over the internet? Its basically hubless pub/sub messaging over HTTP ("...over the internet", sure, but there is lots of much older versions of that); there's also the with-a-hub version, PubSubHubbub.
dwb 13y ago
Great an' all, but what happens when a request fails? There's no (specified) retry mechanism — and POSTs aren't meant to be idempotent. I guess many applications don't need strong reliability and so this is cool, but I can't help but feel HTTP's design is really being stretched with ideas like this. I totally get that it's really easy to get stuff going if everything just speaks HTTP, but is it really for the best?
- baudehlo 13y ago
  Most senders seem to implement retries if they don't get a 200 response.
- paddyforan 13y ago
  I can't speak for other providers, but our practice is to let users set the number of retries they'd like and the space between retries. We also are close to rolling out error queues, so when a push notification fails, the message is automatically placed into a queue, to be processed at the user's convenience. I agree, the need for safe retries (as this is a process without user feedback) calls for idempotency, but that's at odds with the need for a webhook to be possible to be triggered multiple times. One way around this is to use "tickle" webhooks--webhooks that tell the destination that data has changed, but require the destination to pull in that information themselves. Then the webhook call is still idempotent in practice (at worst, you're refreshing data more than you need to, which is still less than polling), but multiple webhook calls can be made.
- dragonwriter 13y ago
  > There's no (specified) retry mechanism — and POSTs aren't meant to be idempotent. The semantics of the data exchanged by POST can be such that the request is effectively idempotent even though that's not an HTTP-level expectation (it's not wrong to have a layered-over-HTTP protocol in which methods are safer than HTTP requires, it would be wrong for them to be less safe than HTTP requires -- e.g., non-idempotent GET.) If you are sending updated notifications over POST with the actual data then needing to be retrieved by a request the other directions, the first POST is effectively idempotent. If you are sending the content in the POST without a key to prevent duplication, then there is a problem with retries.
Sir_Cmpwn 13y ago
MediaCrush updates itself through GitHub hooks [1]. When we get the POST, it git pulls and restarts the site automatically. Best thing we ever did. Not worrying about deployment makes our lives a lot easier. [1] https://github.com/MediaCrush/MediaCrush/blob/master/mediacrush/views/hook.py https://github.com/MediaCrush/MediaCrush/blob/master/mediacr...
e12e 13y ago
Thanks for summing up, concisely, why webhooks really aren't magic at all, in the fourth paragraph: "All [webhooks] are is a promise by an application or API: “when this thing happens, I’ll send this HTTP request with this data.”" It's nice to see that all the hype really just boils down to generalized pingbacks[1]. Now, having a standard is great -- I'm not entirely sure we have a standard yet, though. It's more like a fancy name for pingbacks that aren't just for blog posts. I'm also not entirely sure if I'd prefer webhooks for eg. processing emails. Emails are naturally "push" -- they get pushed via SMTP. I know that people are scared of email/SMTP -- but it seems quite a lot easier to make sure you don't loose any "events" with SMTP than with HTTP(S). What if your site is down for a few minutes, just as some service out there tries to "ping" you with a webhook? Email handles that. I think what most people seem to mix up is that handling "real" email may be hard -- but looking for a well formed subject that exactly matches some regex, then parsing that, isn't that hard. Everything else you can either bounce (a little dangerous) or just drop (not quite as dangerous). Mailing-lists work. What you need isn't much more complicated. Maybe allow for a mime-part with a json or xml, or some other well-formed body. Use gpg or smime to encrypt data, if you need something more than a short hash/token. [1] http://en.wikipedia.org/wiki/Pingback http://en.wikipedia.org/wiki/Pingback
- daigoba66 13y ago
  The interesting thing about SMTP is that a mail transfer agent is typically built using queues for both outbound and inbound transmission. Those queues help ensure durability of a message. You could implement webhooks with a transmission queue, retrying for a bit, maybe giving up depending on the nature of webhook. The awesome part about SMTP is that there is so much existing infrastructure in place, although some agents aren't as reliable as others at delivery.
- baudehlo 13y ago
  Part of the problem has traditionally been that email servers haven't been as easy to "hack" as web servers have. That was a failing on the part of the developers of those servers. Thankfully now you have things like Qpsmtpd, James, Lamson and Haraka which can do those things for you. But not many people install those servers. I'd love to see that change (for obvious reasons).

103 points by paddyforan 13y ago

Ok, maybe it's just the terminology that's confusing, but how do webhooks differ from callback requests? Is it that the semantics are standardized, or is there something that I'm missing? The Author is quite excited about them.

paddyforan 13y ago
I'm incredibly excited about them. As far as I'm concerned, webhooks and callback requests are the same. If we were going to get pedantic, I suppose we could say that for a request to be a callback, it would have to be a response to an earlier request (think asynchronous processing) whereas a webhook is an evented request (it wasn't prompted by a previous request). But being pedantic is lame. Let's just be excited about both! Relevant: https://twitter.com/paddyforan/statuses/254375310554968064 https://twitter.com/paddyforan/statuses/254375310554968064

adelevie 13y ago

Great post, Paddy. One thing I'd love to see in the HUD is a GitHub OAuth + repo webhook config. That is, you log in with GitHub credentials, you see a list of repos and can select which workers will get POSTed to after specific GitHub repo events[1]. Examples: - The "adelevie/MyJekyllBlog" repo will POST to the worker "BuildAndDeploy" after event "push." - The "adelevie/MyRailsApp" repo will POST to the worker "TestMyRailsApp" after events "push" and "pull_request." [1] http://developer.github.com/v3/repos/hooks/ http://developer.github.com/v3/repos/hooks/

paddyforan 13y ago
I love this idea. I've been wanting to do something similar to this, making webhooks even easier to hook into the Iron.io platform. I don't know if it will make it into HUD, but I'm definitely going to be trying to setup some push-button webhook configurations for popular services that support configuring webhooks from an API.
- adelevie 13y ago
  Will/can there be an Iron.io OAuth API?
  - paddyforan 13y ago
    "We have nothing to announce at this time." I'd like there to be is all I've got for you at the moment. :( Priorities, and all that.

carimura 13y ago

The next coming of SOA!

zrail 13y ago

Stripe's webhooks are really nice. Everything that happens with your account, be it charges, disputes, transfers, or people changing subscription levels, all generate events that you can receive and do something with. In my book and the app that sells it[1] I use them to trigger emails to myself when important things happen. There's also an iPhone app named Paypad[2] that registers a webhook on your account that sends you push notifications when things happen. [1]: https://www.petekeen.net/mastering-modern-payments https://www.petekeen.net/mastering-modern-payments [2]: https://www.pay-pad.com https://www.pay-pad.com

baudehlo 13y ago
Just don't disable SSL on your server. http://baudehlo.wordpress.com/2013/09/06/stripe-webhooks-dont-work-with-tls/ http://baudehlo.wordpress.com/2013/09/06/stripe-webhooks-don...

uptown 13y ago

I'm new to webhooks - I understand them conceptually, but is there a standard for authenticating the incoming notification? Take the Stripe example - how do you know that the HTTP POST is actually coming from Stripe? Do you use a shared-secret, or some other mechanism to verify the HTTP request is authentic?

paddyforan 13y ago
Great question! For Iron.io, all our requests are authenticated using an OAuth token. So only people that know your OAuth token (and thus could use our API anyways) can use your webhook endpoints. But there are other ways, too. The most common I've seen is to provide an API endpoint to verify events with (Stripe does this). If you use HTTPS to receive the webhook, and verify it with a request over HTTPS to the API you expect it to be coming from, you're ensuring the request is authentic. For some webhook styles, it doesn't actually matter. Some people use webhooks to just say "Something happened", without actually saying what. In this style, the API is still responsible for the data and authentication, the webhook just says "Hey, wake up, the API has something new for you."
- uptown 13y ago
  Got it. Thanks for the initial write-up, and your explanation.
zrail 13y ago
With Stripe, the process is: 1) Receive a webhook and record the ID in your database. If it's a duplicate, stop. 2) Ask Stripe for the event matching the ID they gave you. If they reject it, stop. 3) Process the event. It requires a roundtrip, but it's the only sane way to validate that the event you got is real and that you haven't seen it before without trying to validate a signature (lots of other APIs make you do that, though)
- rryan 13y ago
  Yes! This is the only part of Stripe's API that I raised an eyebrow at. I think it's a little irresponsible to send all the data via web-hook without providing a way to verify it came from Stripe. At the very least they should only provide the event ID over webhook otherwise people will take the lazy route and trust trivially forgeable messages.

mwetzler 13y ago

Love the post... and stripe webhooks! Integrated them with Keen IO in about 60 seconds and even made a (perhaps overly enthusiastic) blog post about it. https://keen.io/blog/48628875285/integrate-stripe-and-keen-io-in-60-seconds https://keen.io/blog/48628875285/integrate-stripe-and-keen-i...

paddyforan 13y ago
Awesome!

aytekin 13y ago

If you have an API make sure webhooks can be created using the API. It is win-win for everyone. It is easier to implement and apps don't have to poll you constantly.

zrail 13y ago
There was a somewhat-confusing attempt at standardizing that pattern a few weeks ago: http://resthooks.org http://resthooks.org

huxley 13y ago

I've started adding web hooks to one of my Django web apps with Zapier's django-rest-hooks which I'm liking a lot: https://github.com/zapier/django-rest-hooks https://github.com/zapier/django-rest-hooks

richardlblair 13y ago

Webhooks are great, until you don't get one... Then they suddenly become a nightmare.

paddyforan 13y ago
That's why logs from the webhook provider are important. :)
- jessaustin 13y ago
  Yuck. Who wants to deal with that? b^)
- philsnow 13y ago
  That sounds even worse than: > "I watch people poll APIs or create convoluted connections, and I cry a little on the inside." If the action being sent by the webhook is really important, there seems to be no way around doing it somewhat like this: Write a polling (or hanging-get) system, but use webhooks as an unreliable but lower-latency push mechanism. In this setup, webhooks give you low-latency updates (when they work), and polling ensures completeness.
  - drivebyacct2 13y ago
    Sounds like a reinvention of Pubsubhubbub anyway.
  - roarroar 13y ago
    Depending on your requirements the remote end can be responsible for ensuring that you receive and acknowledge the wake up. Once you hit your own timeout you might do any number of things, not necessarily repoll the other end.
philsnow 13y ago
I came across http://blog.iron.io/2013/01/queue-webhook-events-with-ironmq.html http://blog.iron.io/2013/01/queue-webhook-events-with-ironmq... This is pretty sweet, but it seems like it might be more useful for the baked-in reliability to be on the remote end. That is, the side sending the POSTs should notice that it didn't get a 200 reply and retry some number of times with backoff. Also, the POSTs need to be idempotent (include a sequence number or nonce or something) so that if the "200 OK" gets lost and the sending side retries, you can discard the duplicates on the receiving side. That's of course a lot harder since you have to convince everybody whose service you want to use that this is worth the time/effort :)

samograd 13y ago

> It makes the web capable of pushing information to its users, instead of waiting for users to ask for information. http://www.wired.com/wired/archive/5.03/ff_push.html http://www.wired.com/wired/archive/5.03/ff_push.html [1997] It seems I'm not the only one to remember the promises of this prophecy: http://www.boston.com/business/technology/articles/2004/01/05/push_technology_gets_a_nudge/ http://www.boston.com/business/technology/articles/2004/01/0...

mikeknoop 13y ago

Webhooks are awesome but please don't make your users set them up: https://resthooks.org https://resthooks.org

adventured 13y ago
Firefox is throwing a "This Connection is Untrusted" alert on attempting to load resthooks.org via ssl. Someone should probably look into that.
- e12e 13y ago
  https://resthooks.herokuapp.com/ https://resthooks.herokuapp.com/ appears to be the meat of it. But that is really just a pointer to: https://github.com/net-engine/resthooks/blob/master/README.md https://github.com/net-engine/resthooks/blob/master/README.m...

jkarneges 13y ago

http://webhookinbox.com/ http://webhookinbox.com/ gives you throwaway URLs for receiving webhooks. Good for testing.

paddyforan 13y ago
John Sheehan, of Runscope.com (which, by the way, is a super rad tool for API developers) did a great roundup on receiving webhooks for testing: http://john-sheehan.com/blog/a-survey-of-the-localhost-proxying-landscape http://john-sheehan.com/blog/a-survey-of-the-localhost-proxy...

tootie 13y ago

This is basically pub/sub messaging, but over the internet?

dragonwriter 13y ago
> This is basically pub/sub messaging, but over the internet? Its basically hubless pub/sub messaging over HTTP ("...over the internet", sure, but there is lots of much older versions of that); there's also the with-a-hub version, PubSubHubbub.

dwb 13y ago

Great an' all, but what happens when a request fails? There's no (specified) retry mechanism — and POSTs aren't meant to be idempotent. I guess many applications don't need strong reliability and so this is cool, but I can't help but feel HTTP's design is really being stretched with ideas like this. I totally get that it's really easy to get stuff going if everything just speaks HTTP, but is it really for the best?

baudehlo 13y ago
Most senders seem to implement retries if they don't get a 200 response.
paddyforan 13y ago
I can't speak for other providers, but our practice is to let users set the number of retries they'd like and the space between retries. We also are close to rolling out error queues, so when a push notification fails, the message is automatically placed into a queue, to be processed at the user's convenience. I agree, the need for safe retries (as this is a process without user feedback) calls for idempotency, but that's at odds with the need for a webhook to be possible to be triggered multiple times. One way around this is to use "tickle" webhooks--webhooks that tell the destination that data has changed, but require the destination to pull in that information themselves. Then the webhook call is still idempotent in practice (at worst, you're refreshing data more than you need to, which is still less than polling), but multiple webhook calls can be made.
dragonwriter 13y ago
> There's no (specified) retry mechanism — and POSTs aren't meant to be idempotent. The semantics of the data exchanged by POST can be such that the request is effectively idempotent even though that's not an HTTP-level expectation (it's not wrong to have a layered-over-HTTP protocol in which methods are safer than HTTP requires, it would be wrong for them to be less safe than HTTP requires -- e.g., non-idempotent GET.) If you are sending updated notifications over POST with the actual data then needing to be retrieved by a request the other directions, the first POST is effectively idempotent. If you are sending the content in the POST without a key to prevent duplication, then there is a problem with retries.

Sir_Cmpwn 13y ago

MediaCrush updates itself through GitHub hooks [1]. When we get the POST, it git pulls and restarts the site automatically. Best thing we ever did. Not worrying about deployment makes our lives a lot easier. [1] https://github.com/MediaCrush/MediaCrush/blob/master/mediacrush/views/hook.py https://github.com/MediaCrush/MediaCrush/blob/master/mediacr...

e12e 13y ago

Thanks for summing up, concisely, why webhooks really aren't magic at all, in the fourth paragraph: "All [webhooks] are is a promise by an application or API: “when this thing happens, I’ll send this HTTP request with this data.”" It's nice to see that all the hype really just boils down to generalized pingbacks[1]. Now, having a standard is great -- I'm not entirely sure we have a standard yet, though. It's more like a fancy name for pingbacks that aren't just for blog posts. I'm also not entirely sure if I'd prefer webhooks for eg. processing emails. Emails are naturally "push" -- they get pushed via SMTP. I know that people are scared of email/SMTP -- but it seems quite a lot easier to make sure you don't loose any "events" with SMTP than with HTTP(S). What if your site is down for a few minutes, just as some service out there tries to "ping" you with a webhook? Email handles that. I think what most people seem to mix up is that handling "real" email may be hard -- but looking for a well formed subject that exactly matches some regex, then parsing that, isn't that hard. Everything else you can either bounce (a little dangerous) or just drop (not quite as dangerous). Mailing-lists work. What you need isn't much more complicated. Maybe allow for a mime-part with a json or xml, or some other well-formed body. Use gpg or smime to encrypt data, if you need something more than a short hash/token. [1] http://en.wikipedia.org/wiki/Pingback http://en.wikipedia.org/wiki/Pingback

daigoba66 13y ago
The interesting thing about SMTP is that a mail transfer agent is typically built using queues for both outbound and inbound transmission. Those queues help ensure durability of a message. You could implement webhooks with a transmission queue, retrying for a bit, maybe giving up depending on the nature of webhook. The awesome part about SMTP is that there is so much existing infrastructure in place, although some agents aren't as reliable as others at delivery.
baudehlo 13y ago
Part of the problem has traditionally been that email servers haven't been as easy to "hack" as web servers have. That was a failing on the part of the developers of those servers. Thankfully now you have things like Qpsmtpd, James, Lamson and Haraka which can do those things for you. But not many people install those servers. I'd love to see that change (for obvious reasons).