6 ms·

PubSubHubbub

44 points by stanleydrew 16y ago

simias 16y ago
That looks quite complicated and the only benefit I can see is not having to wait 5 minutes for your RSS client to poll the source. Could anybody explain what are the true benefits over our current RSS/Atom polling systems? EDIT: I also fail to see how this would reduce the load. It just transfers the load to the "hub" node. If a big website runs its own hub node to serve its content, then there's no gain (you just had a layer of indirection and complexity). There could be big external hub, but what would be the point for them? Man-in-the-middle-ing the streams and inserting ads? Doesn't sound really great. And if a bazillion micro hubs emerge, then the overall load on the providers will pretty much remain the same.
- stanleydrew 16y ago
  Many many fewer HTTP requests.
  - simias 16y ago
    I suppose webapps such as google reader query only once the source for all the users subscribed to a certain feed, effectively reducing the number of HTTP requests on the upstream provider. Isn't it a simpler and well tested solution to this problem? It basically makes the google server the Hub and your browser the subscriber to use the website's terminology.
    - jonasvp 16y ago
      You might say that Pubsubhubhub is the logical extension of the Google Reader concept. It allows you to "read" those feeds programmatically and enables the content site to notify Google Reader of updates so they get there quicker. Also, anyone can choose to be a hub, not just Google Reader. There's your simple and well tested solution wrapped in a standard for anyone to implement.
    - dpritchett 16y ago
      It's converting a two-tier one-to-many pull system with 5 minute polling to a three-tier one-to-many-to-many push (err.. PuSH) system. More efficient, allows for quicker dissemination of update notifications.
  - iwwr 16y ago
    Although comparatively more concurrent connections. The 'hub' looks somewhat like an IRC server. In fact, you can emulate this behavior with IRC.
    - joshfraser 16y ago
      That's not actually the case. It's one ping per event. You don't keep the connection open.
      - iwwr 16y ago
        OK, the client must send periodic pings to the hub in order to receive updates.
        matthewsimon 16y ago
        Sheesh, did you bother to read the linked document? The "client" in pubsub registers a call-back URL with the server, and the server then POSTs new data to that URL. No periodic polling required -- that's the whole point of this.
        iwwr 16y ago
        Sorry, I was confused. I thought 'hubbub' was a drop-in replacement for RSS. Apparently, it's a meta-protocol built around RSS.
- dpritchett 16y ago
  Friendfeed provided a great example of the benefits here. The shorter your feedback loops between publishing and commenting, the more conversation you see around published content. http://jungleg.com/2009/07/27/pubsubhubbub-wordpress-feedburner-friendfeed-realtime-awesomeness/ http://jungleg.com/2009/07/27/pubsubhubbub-wordpress-feedbur...
- johns 16y ago
  Gowalla implements PuSH which lets you react to check-ins. Much better than constantly polling their API over and over again seeing if something has changed. I wish Foursquare would add this. It would open up a huge number of new use cases.
- michaelbuckbee 16y ago
  The switch from a polling architecture to a notification one greatly reduces the amount of over the wire traffic that has to take place. Consider the case of a blog where it's not updated for a month: every feed reader continues to continuously poll for the feed every hour of every day (though some are better about setting a timing decay, just checking for header last changed, etc.). With PubSubHub the traffic sent is 0. Some other things to consider: 1. The obvious case for this is blogs, but PubSubHub can be the foundation of other more interesting and new projects: consider a distributed Twitter like micro blogging service. I'm not saying building one is a good idea, just that it would be better built on PSH than polling. 2. People are lazy and feeds put in feedreaders are polled forever. I've worked on multiple systems where years later feed readers are still checking every hour for feeds that are missing, or just completely irrelevant.
- exit 16y ago
  > I also fail to see how this would reduce the load. no one in the protocol polls. the publisher pings the hub, the hub pings the n subscribers, vs n subscribers constantly polling a single publisher.
- rektide 16y ago
  hubs cannot insert data. the only function a hub serves is to assert that there is new data on the feed, and that clients should re-request the feed. a malicious hub could continually signal clients to keep fetching the feed, triggering a Denail-of-Service on the feed. that's really the only possible mis-use. hubs can also proxy. ycombinator could run their own hub that could subscribe to someone else's hub and notify by proxy all of it's clients. this negates a lot of the "big hub" mumbo jumbo you seem to fear. to answer your original confusion, the problem PubSubHubbub solves are two fold. foremost, 5 minutes is a long time in some worlds. second, every client subscribing has to pole every 5 minutes, asserting an etag and/or if-modified-since, which the server then has to look at and validate. with pubsubhubbub, results are "instant" and the server doesnt have any load when nothing new is being posted. this is indeed a big deal. to your last, a micro hub will have no additional load, so long as the micro hub has any subscribers.
MicahWedemeyer 16y ago
Is there something new here? PubSubHubbub has been out for a while. (Not trying to be a snob...just seeing if I'm missing something...)
- wyclif 16y ago
  Yeah, I thought exactly the same thing. What's new here?
- streeter 16y ago
  According to the google code updates page (http://code.google.com/p/pubsubhubbub/updates/list http://code.google.com/p/pubsubhubbub/updates/list), nothing new has happened since January 3rd. So no, nothing new looks to have happened.
doki_pen 16y ago
I love PuSH and we are using it on feedr.embed.ly. It was pretty simple to implement and the code is tight. The only thing that would save us a lot of work is if RSS feeds provided canonical links. We need a canonical link to match RSS feeds to our link metadata properly.
zwadia 16y ago
yawn

44 points by stanleydrew 16y ago

That looks quite complicated and the only benefit I can see is not having to wait 5 minutes for your RSS client to poll the source. Could anybody explain what are the true benefits over our current RSS/Atom polling systems? EDIT: I also fail to see how this would reduce the load. It just transfers the load to the "hub" node. If a big website runs its own hub node to serve its content, then there's no gain (you just had a layer of indirection and complexity). There could be big external hub, but what would be the point for them? Man-in-the-middle-ing the streams and inserting ads? Doesn't sound really great. And if a bazillion micro hubs emerge, then the overall load on the providers will pretty much remain the same.

stanleydrew 16y ago
Many many fewer HTTP requests.
- simias 16y ago
  I suppose webapps such as google reader query only once the source for all the users subscribed to a certain feed, effectively reducing the number of HTTP requests on the upstream provider. Isn't it a simpler and well tested solution to this problem? It basically makes the google server the Hub and your browser the subscriber to use the website's terminology.
  - jonasvp 16y ago
    You might say that Pubsubhubhub is the logical extension of the Google Reader concept. It allows you to "read" those feeds programmatically and enables the content site to notify Google Reader of updates so they get there quicker. Also, anyone can choose to be a hub, not just Google Reader. There's your simple and well tested solution wrapped in a standard for anyone to implement.
  - dpritchett 16y ago
    It's converting a two-tier one-to-many pull system with 5 minute polling to a three-tier one-to-many-to-many push (err.. PuSH) system. More efficient, allows for quicker dissemination of update notifications.
- iwwr 16y ago
  Although comparatively more concurrent connections. The 'hub' looks somewhat like an IRC server. In fact, you can emulate this behavior with IRC.
  - joshfraser 16y ago
    That's not actually the case. It's one ping per event. You don't keep the connection open.
    - iwwr 16y ago
      OK, the client must send periodic pings to the hub in order to receive updates.
      - matthewsimon 16y ago
        Sheesh, did you bother to read the linked document? The "client" in pubsub registers a call-back URL with the server, and the server then POSTs new data to that URL. No periodic polling required -- that's the whole point of this.
        iwwr 16y ago
        Sorry, I was confused. I thought 'hubbub' was a drop-in replacement for RSS. Apparently, it's a meta-protocol built around RSS.
dpritchett 16y ago
Friendfeed provided a great example of the benefits here. The shorter your feedback loops between publishing and commenting, the more conversation you see around published content. http://jungleg.com/2009/07/27/pubsubhubbub-wordpress-feedburner-friendfeed-realtime-awesomeness/ http://jungleg.com/2009/07/27/pubsubhubbub-wordpress-feedbur...
johns 16y ago
Gowalla implements PuSH which lets you react to check-ins. Much better than constantly polling their API over and over again seeing if something has changed. I wish Foursquare would add this. It would open up a huge number of new use cases.
michaelbuckbee 16y ago
The switch from a polling architecture to a notification one greatly reduces the amount of over the wire traffic that has to take place. Consider the case of a blog where it's not updated for a month: every feed reader continues to continuously poll for the feed every hour of every day (though some are better about setting a timing decay, just checking for header last changed, etc.). With PubSubHub the traffic sent is 0. Some other things to consider: 1. The obvious case for this is blogs, but PubSubHub can be the foundation of other more interesting and new projects: consider a distributed Twitter like micro blogging service. I'm not saying building one is a good idea, just that it would be better built on PSH than polling. 2. People are lazy and feeds put in feedreaders are polled forever. I've worked on multiple systems where years later feed readers are still checking every hour for feeds that are missing, or just completely irrelevant.
exit 16y ago
> I also fail to see how this would reduce the load. no one in the protocol polls. the publisher pings the hub, the hub pings the n subscribers, vs n subscribers constantly polling a single publisher.
rektide 16y ago
hubs cannot insert data. the only function a hub serves is to assert that there is new data on the feed, and that clients should re-request the feed. a malicious hub could continually signal clients to keep fetching the feed, triggering a Denail-of-Service on the feed. that's really the only possible mis-use. hubs can also proxy. ycombinator could run their own hub that could subscribe to someone else's hub and notify by proxy all of it's clients. this negates a lot of the "big hub" mumbo jumbo you seem to fear. to answer your original confusion, the problem PubSubHubbub solves are two fold. foremost, 5 minutes is a long time in some worlds. second, every client subscribing has to pole every 5 minutes, asserting an etag and/or if-modified-since, which the server then has to look at and validate. with pubsubhubbub, results are "instant" and the server doesnt have any load when nothing new is being posted. this is indeed a big deal. to your last, a micro hub will have no additional load, so long as the micro hub has any subscribers.

MicahWedemeyer 16y ago

Is there something new here? PubSubHubbub has been out for a while. (Not trying to be a snob...just seeing if I'm missing something...)

wyclif 16y ago
Yeah, I thought exactly the same thing. What's new here?
streeter 16y ago
According to the google code updates page (http://code.google.com/p/pubsubhubbub/updates/list http://code.google.com/p/pubsubhubbub/updates/list), nothing new has happened since January 3rd. So no, nothing new looks to have happened.

doki_pen 16y ago

I love PuSH and we are using it on feedr.embed.ly. It was pretty simple to implement and the code is tight. The only thing that would save us a lot of work is if RSS feeds provided canonical links. We need a canonical link to match RSS feeds to our link metadata properly.

zwadia 16y ago

yawn