6 ms·

Webhooks Level Up

128 points by Zikes 13y ago

jds375 13y ago
The direct configuration will be pretty neat to try out. For those unfamiliar with webhooks: http://developer.github.com/webhooks/ http://developer.github.com/webhooks/
sync 13y ago
How are people integrating their own webhooks into their apps? e.g. What is the backend for github or stripe's webhooks? I'm surprised there's not a SaaS that handles the distribution, security, retries, and introspection for you...
- rpearl 13y ago
  And then you have to inform the service when there are relevant changes which requires secure communications to the service, retry attempts in the event of failure, and debugging...?
- jkarneges 13y ago
  FWIW, http://fanout.io http://fanout.io is a SaaS that can do webhooks distribution. No introspection yet though.
- tdumitrescu 13y ago
  What kinds of consumers, you mean? I've found them most useful for integration with new hosted services that haven't yet made it onto the external services list provided out-of-the-box by Github. For instance, when you want to try out (or develop) some new project tracker or hosted CI solution. I'm a little surprised they still don't offer any auth configuration for webhooks the way they do for most external services, so you're stuck with basic auth on the post-receive URL.
  - delroth 13y ago
    There's some HMAC-SHA1 available but it's hidden in their REST API. I really don't understand why the option is not available via their web interface, especially since it is already supported in their backend.
- pearkes 13y ago
  Yea, it's definitely true in the Stripe world. I end up re-writing boiler plate for each project. (i.e, send a receipt from this webhook, etc.) I ended up writing an independent service for it[1], but it'd be cool to let someone else deal. Also, to make receipts nice, and so on. [1]: https://github.com/pearkes/stripe-hooks https://github.com/pearkes/stripe-hooks
- johns 13y ago
  We can help with the introspection: http://blog.runscope.com/posts/api-providers-build-a-twilio-like-request-inspector-with-runscope http://blog.runscope.com/posts/api-providers-build-a-twilio-...
- rs 13y ago
  xp-dev.com has had webhooks since 2010-ish: https://xp-dev.com/docs/d/user-guide/repositories/webhooks https://xp-dev.com/docs/d/user-guide/repositories/webhooks
jackcarter 13y ago
Can I use this to automatically push to (e.g.) Heroku upon pushing to Github?
- deleted 13y ago
  [deleted]
- michaelmior 13y ago
  You could, but you'd have to have some intermediate service to handle that process. One great choice is Travis CI[1] which is really a continuous integration service, but would let you do what you want. You can also just attach multiple URLs to a single git remote so you push to both GitHub and Heroku all the time. However, I think running a test suite before deploying is a much safer option as it stops you from accidentally deploying something you haven't properly tested. [1] http://docs.travis-ci.com/user/deployment/heroku/ http://docs.travis-ci.com/user/deployment/heroku/
  - atmos 13y ago
    We have a preview API in place for these types of deployment tasks. Lots of folks, travis/etc, seem to have piggy backed on top of successful builds too. http://developer.github.com/v3/repos/deployments/ http://developer.github.com/v3/repos/deployments/
    - jon-wood 13y ago
      Seeing that in the list of events to listen to was probably the thing I was most excited to see, I'm looking forward to seeing where that API goes!
  - nthj 13y ago
    CircleCI is another option: https://circleci.com https://circleci.com
jnwng 13y ago
digging through some of the events we can hook on to here: http://developer.github.com/webhooks/#events http://developer.github.com/webhooks/#events, i noticed the `gollum` event, which is triggered whenever a wiki page gets updated. i'm curious why its named that?
- chrisguitarguy 13y ago
  https://github.com/gollum/gollum https://github.com/gollum/gollum Named after the software that powers the wiki.
  - jnwng 13y ago
    ah, got it. thanks!
skywhopper 13y ago
Here's an example of the kinds of features Github's staff is working on that takes a significant amount of time and coordination, and it's why we don't see 100 new "who of your friends also starred this repo?" features each week.
- fudged71 13y ago
  "who of your friends also starred this repo?" I hope you know that feature came out this week...
  - skywhopper 13y ago
    Yep. I was indirectly responding to a complaint from that thread in which the commenter wondered why Github wasn't announcing new UI features every day since the friend-starring feature was just "an afternoon's worth of work".
    - deleted 13y ago
      [deleted]
akerl_ 13y ago
I'm stoked to see the webhooks get an overhaul. I was just about to use them for a project, and this'll be a great help. I wish they'd do similarly for the services. Currently, editing multiple services across repos is a bit of a pain, due to the length of the list and how it jumps you to the top on any interactions. (Originally, I thought this announcement meant they'd revamped that as well) Out of curiosity, is there a generally-accepted best practice for rate-limiting / authenticating webhook pushes? I'm planning to run a daemon that listens for them, and while the effect of somebody finding the URL and hammering it isn't likely to be catastrophic, I'd still like to rate limit it if I can. Alternately, are GitHub's webhook-sending IPs static enough to put in an IP set and add to my firewall?
- atmos 13y ago
  You can always get an updated list from http://developer.github.com/v3/meta/ http://developer.github.com/v3/meta/ but they're pretty static right now.
- jon-wood 13y ago
  If I were building something to handle webhooks I'd probably have a really minimal API which just takes the message as it comes in and then queues it for later processing - that way you can either decide to let the queue back up, or add some more workers. Having said that, I'm pretty sure GitHub publish a list of their public IPs, so you could also limit what is allowed to call your API.
joeshaw 13y ago
I wrote a tool to integrate GitHub pull requests with Jenkins builds called Leeroy: https://github.com/litl/leeroy https://github.com/litl/leeroy This new feature will make things so much easier to set up and debug, I'm thrilled. Before, you could not create a pull request hook without using the API, and this caused a lot of people confusion. We set up the hook automatically in Leeroy, but this will allow people to set it up manually if they want, and debug when it doesn't work.
- hack_edu 13y ago
  Just to share an awesome GitHub feature that I learned about when doing a similar integration as you; your repository also tracks all Pull Request related commits. Really helpful when you're trying to build these sorts of tools in the future. If you set refspec to the following then you'll get automated builds for any commit activity related to all Pulls. Its saved me tons of time writing middleware to handle these things. +refs/pull/*:refs/remotes/origin/pr/* The major Jenkins plugins have their respective build-on-new-commits, so you can automate these builds and just treat these like feature branches.'

128 points by Zikes 13y ago

The direct configuration will be pretty neat to try out. For those unfamiliar with webhooks: http://developer.github.com/webhooks/ http://developer.github.com/webhooks/

sync 13y ago

How are people integrating their own webhooks into their apps? e.g. What is the backend for github or stripe's webhooks? I'm surprised there's not a SaaS that handles the distribution, security, retries, and introspection for you...

rpearl 13y ago
And then you have to inform the service when there are relevant changes which requires secure communications to the service, retry attempts in the event of failure, and debugging...?
jkarneges 13y ago
FWIW, http://fanout.io http://fanout.io is a SaaS that can do webhooks distribution. No introspection yet though.
tdumitrescu 13y ago
What kinds of consumers, you mean? I've found them most useful for integration with new hosted services that haven't yet made it onto the external services list provided out-of-the-box by Github. For instance, when you want to try out (or develop) some new project tracker or hosted CI solution. I'm a little surprised they still don't offer any auth configuration for webhooks the way they do for most external services, so you're stuck with basic auth on the post-receive URL.
- delroth 13y ago
  There's some HMAC-SHA1 available but it's hidden in their REST API. I really don't understand why the option is not available via their web interface, especially since it is already supported in their backend.
pearkes 13y ago
Yea, it's definitely true in the Stripe world. I end up re-writing boiler plate for each project. (i.e, send a receipt from this webhook, etc.) I ended up writing an independent service for it[1], but it'd be cool to let someone else deal. Also, to make receipts nice, and so on. [1]: https://github.com/pearkes/stripe-hooks https://github.com/pearkes/stripe-hooks
johns 13y ago
We can help with the introspection: http://blog.runscope.com/posts/api-providers-build-a-twilio-like-request-inspector-with-runscope http://blog.runscope.com/posts/api-providers-build-a-twilio-...
rs 13y ago
xp-dev.com has had webhooks since 2010-ish: https://xp-dev.com/docs/d/user-guide/repositories/webhooks https://xp-dev.com/docs/d/user-guide/repositories/webhooks

jackcarter 13y ago

Can I use this to automatically push to (e.g.) Heroku upon pushing to Github?

deleted 13y ago
[deleted]
michaelmior 13y ago
You could, but you'd have to have some intermediate service to handle that process. One great choice is Travis CI[1] which is really a continuous integration service, but would let you do what you want. You can also just attach multiple URLs to a single git remote so you push to both GitHub and Heroku all the time. However, I think running a test suite before deploying is a much safer option as it stops you from accidentally deploying something you haven't properly tested. [1] http://docs.travis-ci.com/user/deployment/heroku/ http://docs.travis-ci.com/user/deployment/heroku/
- atmos 13y ago
  We have a preview API in place for these types of deployment tasks. Lots of folks, travis/etc, seem to have piggy backed on top of successful builds too. http://developer.github.com/v3/repos/deployments/ http://developer.github.com/v3/repos/deployments/
  - jon-wood 13y ago
    Seeing that in the list of events to listen to was probably the thing I was most excited to see, I'm looking forward to seeing where that API goes!
- nthj 13y ago
  CircleCI is another option: https://circleci.com https://circleci.com

jnwng 13y ago

digging through some of the events we can hook on to here: http://developer.github.com/webhooks/#events http://developer.github.com/webhooks/#events, i noticed the `gollum` event, which is triggered whenever a wiki page gets updated. i'm curious why its named that?

chrisguitarguy 13y ago
https://github.com/gollum/gollum https://github.com/gollum/gollum Named after the software that powers the wiki.
- jnwng 13y ago
  ah, got it. thanks!

skywhopper 13y ago

Here's an example of the kinds of features Github's staff is working on that takes a significant amount of time and coordination, and it's why we don't see 100 new "who of your friends also starred this repo?" features each week.

fudged71 13y ago
"who of your friends also starred this repo?" I hope you know that feature came out this week...
- skywhopper 13y ago
  Yep. I was indirectly responding to a complaint from that thread in which the commenter wondered why Github wasn't announcing new UI features every day since the friend-starring feature was just "an afternoon's worth of work".
  - deleted 13y ago
    [deleted]

akerl_ 13y ago

I'm stoked to see the webhooks get an overhaul. I was just about to use them for a project, and this'll be a great help. I wish they'd do similarly for the services. Currently, editing multiple services across repos is a bit of a pain, due to the length of the list and how it jumps you to the top on any interactions. (Originally, I thought this announcement meant they'd revamped that as well) Out of curiosity, is there a generally-accepted best practice for rate-limiting / authenticating webhook pushes? I'm planning to run a daemon that listens for them, and while the effect of somebody finding the URL and hammering it isn't likely to be catastrophic, I'd still like to rate limit it if I can. Alternately, are GitHub's webhook-sending IPs static enough to put in an IP set and add to my firewall?

atmos 13y ago
You can always get an updated list from http://developer.github.com/v3/meta/ http://developer.github.com/v3/meta/ but they're pretty static right now.
jon-wood 13y ago
If I were building something to handle webhooks I'd probably have a really minimal API which just takes the message as it comes in and then queues it for later processing - that way you can either decide to let the queue back up, or add some more workers. Having said that, I'm pretty sure GitHub publish a list of their public IPs, so you could also limit what is allowed to call your API.

joeshaw 13y ago

I wrote a tool to integrate GitHub pull requests with Jenkins builds called Leeroy: https://github.com/litl/leeroy https://github.com/litl/leeroy This new feature will make things so much easier to set up and debug, I'm thrilled. Before, you could not create a pull request hook without using the API, and this caused a lot of people confusion. We set up the hook automatically in Leeroy, but this will allow people to set it up manually if they want, and debug when it doesn't work.

hack_edu 13y ago
Just to share an awesome GitHub feature that I learned about when doing a similar integration as you; your repository also tracks all Pull Request related commits. Really helpful when you're trying to build these sorts of tools in the future. If you set refspec to the following then you'll get automated builds for any commit activity related to all Pulls. Its saved me tons of time writing middleware to handle these things. +refs/pull/*:refs/remotes/origin/pr/* The major Jenkins plugins have their respective build-on-new-commits, so you can automate these builds and just treat these like feature branches.'