4 ms·

> Having a new HTTP verb for subscribing and publishing would seem like unnecessary complexity to me. Complexity for who? Introducing a secondary "hub" resour

by dnomad 8y ago

> Having a new HTTP verb for subscribing and publishing would seem like unnecessary complexity to me. Complexity for who? Introducing a secondary "hub" resource here is just accidental complexity. If I want to subscribe to resource A why am I talking to a different resource B? And once you introduce a secondary resource now you need yet another service discovery mechanism to support discovery of these pseudo-resource hubs. (Heaven forbid using an existing service discovery mechanism like RDDL.) Honestly stuff like this is just so poorly thought out it's difficult to understand why W3C stamps approval on this crap. There's no consideration given to alternate protocols like WebSockets or XMPP and there's no attempt to layer on top of existing standards in a meaningful way (hub.secret -- really???). Worst of all there's no real understanding here of what it means for a resource to change. The entire Content-Distribution model is geared towards just one very narrow use case. It's clear the W3C is all about being "inclusive" and "moving fast" there's real fear of "overthinking" things -- but seriously if this is the result we'd probably better off with better standards once a decade then this.

pjc50 8y ago
New HTTP verbs require support in the web server, and all proxies along the way, while using existing ones doesn't and is just part of the normal application framework.
fauigerzigerk 8y ago
>Introducing a secondary "hub" resource here is just accidental complexity. If I want to subscribe to resource A why am I talking to a different resource B? Think about what the hub has to do. It may have to notify millions of subscribers, deal with any errors, retry, etc. This is a very heavy duty messaging system that most publishers will not want to run themselves. And yet you want the publisher's domain name to be the well known resource that ultimately controls things. Publishers may be blogs hosted on small websites or even things like cars, phones, laptops or home appliances that are not always online or have to work under tight resource constraints. Publishers may wish to distribute their content through more than one hub. We don't even have to think of avoiding censorship to see why this increases availability. I think making it possible to split the roles of publisher and distributor is a very good idea. You can still decide to implement both roles on one server.
- antpls 8y ago
  Best explanation for me so far. Problem I see is that the same resources constraints could be on the Subscribers (limited connectivity, limited uptime) and the protocol does not address it?
  - fauigerzigerk 8y ago
    Good question. I don't know if the proposal addresses it directly, but as subscribers have to provide an HTTP endpoint for notifications, I would expect that subscribers would not normally be end user devices but rather gateways similar to SMTP servers or application APIs.
jcrites 8y ago
I've just learned about this protocol tonight, but from skimming the specification, I think it's a well-thought-out, practical, minimal protocol. WebSub is a protocol for people who want to implement the publish/subscribe pattern over HTTP callbacks (aka webhooks). Using webhooks means that subscribers don't need to have any kind of ongoing connection or session open to receive publishes. Subscribers are passive web servers and merely wait to receive an HTTP POST. No state, no connection, polling, or anything. The general model of HTTP callbacks is a simple scheme that's easy to implement using any programming language or platform out there, all of which have HTTP clients and servers capable of getting the job done with minimal fuss. I have actually built custom systems that worked using a very similar pattern as this protocol, where clients of a service pass in URLs where they'd like to be notified when an event occurs. Perhaps this is why I find myself nodding along when I read the protocol spec. There wasn't any standard way to model this, so I just invented something on the fly. You also see this pattern implemented in services like AWS SNS's support for HTTP [1], in Google Cloud PubSub, Twilio, etc. Each of these has an entirely custom protocol for PubSub over HTTP callback, and not something that's standard. They all tackle similar issues like preventing attackers from creating unauthorized subscriptions to URLs, but in different ways. WebSockets doesn't solve the same problem as WebSub. WebSockets require a continuous connection from a client to a service. An application will need to devise its own logic for resuming a session if the connection breaks. WebSub requires no active connection nor session. WebSub subscriptions could remain functional for months at a time (really indefinitely), with there being no communications whatsoever between messages. The system that initiates the subscription can be different than the one that receives the publishes, which is valuable because it means that messages don't all have to go to a single place. Publishes are sent to the domain name specified in the subscription URL. The subscribed web server could change regularly and everything will work as long as the DNS name keeps pointing to the right place. You could use multiple web servers to handle the subscription, by putting multiple servers in the DNS record, or you could use a load balancer in the same way as other web requests. This means you can scale easily. These are the kinds of benefits you get from building subscriptions on top of HTTP. All of the standard techniques and standard software "just work". I'm not an expert on XMPP, but I suspect XMPP would also be a bad fit for this use-case, and would also require continuous connectivity from the subscriber (please correct me if I'm wrong.) I think the same is true for MQTT but I'm not an expert on that either. As a person who has built and used multiple systems following this general abstract pattern, I think this is a good attempt at drafting a standard protocol. My impression reading the spec is that its designers had a good idea what problem they wanted to solve, and what kind of characteristics they wanted the solution to have, and came up with a protocol that succeeded in meeting those requirements. What's the objection to hub.secret? That facility doesn't seem essential to a minimal version of a protocol like this, but I understand why they included it. It provides a simple way for the subscriber to authenticate that the content they're receiving is legitimately the result of their subscription to the topic, and not e.g. an attacker's subscription, or an attacker system that's trying to impersonate the hub. How would you tackle this issue in a simpler way? (It would not be easy to solve this problem within the protocol using TLS, for example.) [1] https://docs.aws.amazon.com/sns/latest/dg/SendMessageToHttp.html https://docs.aws.amazon.com/sns/latest/dg/SendMessageToHttp....
apparentlymart 8y ago
> better standards once a decade Funnily enough, the first drafts of this protocol (back then, called PubSubHubbub) were written circa 2008, so this specification is about a decade in the making. At the time it was distributing content between a number of the bigger blogging/publishing platforms of the day, and also notifying search engines so they could update their indexes more quickly. If anything it seems like the standardization process was too long and missed the boat here (this particular problem is now most often solved by proprietary protocols), rather than being "rushed through". Can't deny that the world has changed a lot during the lifespan of this idea, though. Cellular-connected computers in our pockets were barely on the radar when this spec was first written. I'm sure some would argue that the burdens of publishing have now shifted on to the reader (probably battery powered, spotty connectivity) whereas in this spec's original universe the burdens were on the publisher (CDNs not yet as widespread, more independent publishing from web hotels, etc).

by dnomad 8y ago

pjc50 8y ago

New HTTP verbs require support in the web server, and all proxies along the way, while using existing ones doesn't and is just part of the normal application framework.

fauigerzigerk 8y ago

>Introducing a secondary "hub" resource here is just accidental complexity. If I want to subscribe to resource A why am I talking to a different resource B? Think about what the hub has to do. It may have to notify millions of subscribers, deal with any errors, retry, etc. This is a very heavy duty messaging system that most publishers will not want to run themselves. And yet you want the publisher's domain name to be the well known resource that ultimately controls things. Publishers may be blogs hosted on small websites or even things like cars, phones, laptops or home appliances that are not always online or have to work under tight resource constraints. Publishers may wish to distribute their content through more than one hub. We don't even have to think of avoiding censorship to see why this increases availability. I think making it possible to split the roles of publisher and distributor is a very good idea. You can still decide to implement both roles on one server.

antpls 8y ago
Best explanation for me so far. Problem I see is that the same resources constraints could be on the Subscribers (limited connectivity, limited uptime) and the protocol does not address it?
- fauigerzigerk 8y ago
  Good question. I don't know if the proposal addresses it directly, but as subscribers have to provide an HTTP endpoint for notifications, I would expect that subscribers would not normally be end user devices but rather gateways similar to SMTP servers or application APIs.

jcrites 8y ago

I've just learned about this protocol tonight, but from skimming the specification, I think it's a well-thought-out, practical, minimal protocol. WebSub is a protocol for people who want to implement the publish/subscribe pattern over HTTP callbacks (aka webhooks). Using webhooks means that subscribers don't need to have any kind of ongoing connection or session open to receive publishes. Subscribers are passive web servers and merely wait to receive an HTTP POST. No state, no connection, polling, or anything. The general model of HTTP callbacks is a simple scheme that's easy to implement using any programming language or platform out there, all of which have HTTP clients and servers capable of getting the job done with minimal fuss. I have actually built custom systems that worked using a very similar pattern as this protocol, where clients of a service pass in URLs where they'd like to be notified when an event occurs. Perhaps this is why I find myself nodding along when I read the protocol spec. There wasn't any standard way to model this, so I just invented something on the fly. You also see this pattern implemented in services like AWS SNS's support for HTTP [1], in Google Cloud PubSub, Twilio, etc. Each of these has an entirely custom protocol for PubSub over HTTP callback, and not something that's standard. They all tackle similar issues like preventing attackers from creating unauthorized subscriptions to URLs, but in different ways. WebSockets doesn't solve the same problem as WebSub. WebSockets require a continuous connection from a client to a service. An application will need to devise its own logic for resuming a session if the connection breaks. WebSub requires no active connection nor session. WebSub subscriptions could remain functional for months at a time (really indefinitely), with there being no communications whatsoever between messages. The system that initiates the subscription can be different than the one that receives the publishes, which is valuable because it means that messages don't all have to go to a single place. Publishes are sent to the domain name specified in the subscription URL. The subscribed web server could change regularly and everything will work as long as the DNS name keeps pointing to the right place. You could use multiple web servers to handle the subscription, by putting multiple servers in the DNS record, or you could use a load balancer in the same way as other web requests. This means you can scale easily. These are the kinds of benefits you get from building subscriptions on top of HTTP. All of the standard techniques and standard software "just work". I'm not an expert on XMPP, but I suspect XMPP would also be a bad fit for this use-case, and would also require continuous connectivity from the subscriber (please correct me if I'm wrong.) I think the same is true for MQTT but I'm not an expert on that either. As a person who has built and used multiple systems following this general abstract pattern, I think this is a good attempt at drafting a standard protocol. My impression reading the spec is that its designers had a good idea what problem they wanted to solve, and what kind of characteristics they wanted the solution to have, and came up with a protocol that succeeded in meeting those requirements. What's the objection to hub.secret? That facility doesn't seem essential to a minimal version of a protocol like this, but I understand why they included it. It provides a simple way for the subscriber to authenticate that the content they're receiving is legitimately the result of their subscription to the topic, and not e.g. an attacker's subscription, or an attacker system that's trying to impersonate the hub. How would you tackle this issue in a simpler way? (It would not be easy to solve this problem within the protocol using TLS, for example.) [1] https://docs.aws.amazon.com/sns/latest/dg/SendMessageToHttp.html https://docs.aws.amazon.com/sns/latest/dg/SendMessageToHttp....

apparentlymart 8y ago

> better standards once a decade Funnily enough, the first drafts of this protocol (back then, called PubSubHubbub) were written circa 2008, so this specification is about a decade in the making. At the time it was distributing content between a number of the bigger blogging/publishing platforms of the day, and also notifying search engines so they could update their indexes more quickly. If anything it seems like the standardization process was too long and missed the boat here (this particular problem is now most often solved by proprietary protocols), rather than being "rushed through". Can't deny that the world has changed a lot during the lifespan of this idea, though. Cellular-connected computers in our pockets were barely on the radar when this spec was first written. I'm sure some would argue that the burdens of publishing have now shifted on to the reader (probably battery powered, spotty connectivity) whereas in this spec's original universe the burdens were on the publisher (CDNs not yet as widespread, more independent publishing from web hotels, etc).