6 ms·

I've just learned about this protocol tonight, but from skimming the specification, I think it's a well-thought-out, practical, minimal protocol. WebSub is a p

I've just learned about this protocol tonight, but from skimming the specification, I think it's a well-thought-out, practical, minimal protocol. WebSub is a protocol for people who want to implement the publish/subscribe pattern over HTTP callbacks (aka webhooks). Using webhooks means that subscribers don't need to have any kind of ongoing connection or session open to receive publishes. Subscribers are passive web servers and merely wait to receive an HTTP POST. No state, no connection, polling, or anything. The general model of HTTP callbacks is a simple scheme that's easy to implement using any programming language or platform out there, all of which have HTTP clients and servers capable of getting the job done with minimal fuss. I have actually built custom systems that worked using a very similar pattern as this protocol, where clients of a service pass in URLs where they'd like to be notified when an event occurs. Perhaps this is why I find myself nodding along when I read the protocol spec. There wasn't any standard way to model this, so I just invented something on the fly. You also see this pattern implemented in services like AWS SNS's support for HTTP [1], in Google Cloud PubSub, Twilio, etc. Each of these has an entirely custom protocol for PubSub over HTTP callback, and not something that's standard. They all tackle similar issues like preventing attackers from creating unauthorized subscriptions to URLs, but in different ways. WebSockets doesn't solve the same problem as WebSub. WebSockets require a continuous connection from a client to a service. An application will need to devise its own logic for resuming a session if the connection breaks. WebSub requires no active connection nor session. WebSub subscriptions could remain functional for months at a time (really indefinitely), with there being no communications whatsoever between messages. The system that initiates the subscription can be different than the one that receives the publishes, which is valuable because it means that messages don't all have to go to a single place. Publishes are sent to the domain name specified in the subscription URL. The subscribed web server could change regularly and everything will work as long as the DNS name keeps pointing to the right place. You could use multiple web servers to handle the subscription, by putting multiple servers in the DNS record, or you could use a load balancer in the same way as other web requests. This means you can scale easily. These are the kinds of benefits you get from building subscriptions on top of HTTP. All of the standard techniques and standard software "just work". I'm not an expert on XMPP, but I suspect XMPP would also be a bad fit for this use-case, and would also require continuous connectivity from the subscriber (please correct me if I'm wrong.) I think the same is true for MQTT but I'm not an expert on that either. As a person who has built and used multiple systems following this general abstract pattern, I think this is a good attempt at drafting a standard protocol. My impression reading the spec is that its designers had a good idea what problem they wanted to solve, and what kind of characteristics they wanted the solution to have, and came up with a protocol that succeeded in meeting those requirements. What's the objection to hub.secret? That facility doesn't seem essential to a minimal version of a protocol like this, but I understand why they included it. It provides a simple way for the subscriber to authenticate that the content they're receiving is legitimately the result of their subscription to the topic, and not e.g. an attacker's subscription, or an attacker system that's trying to impersonate the hub. How would you tackle this issue in a simpler way? (It would not be easy to solve this problem within the protocol using TLS, for example.) [1] https://docs.aws.amazon.com/sns/latest/dg/SendMessageToHttp.html https://docs.aws.amazon.com/sns/latest/dg/SendMessageToHttp....

antpls 8y ago
Thanks for the explanation. I still don't see the plus value compared to a simple pull-news-from-URL model. RSS with GET is already session-less, and remains functional for months, AND also works when the clients cannot receive incoming connection (mobile devices)
- detaro 8y ago
  And Pubsubhubbub/WebSub is a plain simple upgrade on top of that model, for when you do not want to rely on polling only, e.g. because you want updates delivered quickly (vs polling, where most implementations sensibly have logic to adapt their polling speed to posting rate of a resource). One of the benefits closed platforms have is that they can deliver posts inside the platform immediately, WebSub brings the option for that to feeds on the open web, without requiring subscribers to poll every <5 minutes and without requiring them to do large changes under the hood, e.g. introducing new non-HTTP protocols which can't be used on all hosting options. For end-devices other update mechanisms are useful as you say, and systems speaking them could hook onto Websub hubs to get notifications they then translate. E.g. your typical wordpress blog has no chance of offering a XMPP channel, but it can ping a WebSub hub since it's only HTTP.
  - antpls 8y ago
    If you want to "upgrade" the old server-client model (request-response) to a realtime instant bidirectional information flow (dialogue communication), you surely want to look at protocols such as XMPP.
    - detaro 8y ago
      Again, you exclude large parts of the web if you require something like XMPP. It totally makes sense to look into such things where it's viable, and you can bridge them with WebSub, but that was not in scope for pubsubhubbub (and WebSub is intentionally only pubsubhubbub with minor clarifications and cleanup). Pubsubhubbub was a (relative) success because everyone doing RSS feeds could easily add it with their existing tech stack.

antpls 8y ago

Thanks for the explanation. I still don't see the plus value compared to a simple pull-news-from-URL model. RSS with GET is already session-less, and remains functional for months, AND also works when the clients cannot receive incoming connection (mobile devices)

detaro 8y ago
And Pubsubhubbub/WebSub is a plain simple upgrade on top of that model, for when you do not want to rely on polling only, e.g. because you want updates delivered quickly (vs polling, where most implementations sensibly have logic to adapt their polling speed to posting rate of a resource). One of the benefits closed platforms have is that they can deliver posts inside the platform immediately, WebSub brings the option for that to feeds on the open web, without requiring subscribers to poll every <5 minutes and without requiring them to do large changes under the hood, e.g. introducing new non-HTTP protocols which can't be used on all hosting options. For end-devices other update mechanisms are useful as you say, and systems speaking them could hook onto Websub hubs to get notifications they then translate. E.g. your typical wordpress blog has no chance of offering a XMPP channel, but it can ping a WebSub hub since it's only HTTP.
- antpls 8y ago
  If you want to "upgrade" the old server-client model (request-response) to a realtime instant bidirectional information flow (dialogue communication), you surely want to look at protocols such as XMPP.
  - detaro 8y ago
    Again, you exclude large parts of the web if you require something like XMPP. It totally makes sense to look into such things where it's viable, and you can bridge them with WebSub, but that was not in scope for pubsubhubbub (and WebSub is intentionally only pubsubhubbub with minor clarifications and cleanup). Pubsubhubbub was a (relative) success because everyone doing RSS feeds could easily add it with their existing tech stack.