11 ms·
WebSub: Open protocol for distributed pub–sub communication on the internet
- arjun27 8y agoThis has come along way from "WebSub was previously known as PubSubHubbub"
- MickerNews 8y agoThe former is a much better name IMHO!
- friend-monoid 8y ago1. Why not a SUB HTTP request? And a PUB http request. The response URL could be a required header. 2. We have HEAD, can we do service discovery using HEAD? 3. Why not let a topic be a HTTP URL? “PUB /user/john/position HTTP/1.1\r\ndata...”. 4. Subscription expiration as a way to force subscribers to renew and upon renew get redirected to other servers is pretty cool. NATS has a special message (the INFO message) to do the same, but you might be in the middle of an important request-reply session you don’t want to abort. 5. The authors could have made this protocol very “non-http-ish” by implementing what amounts to Redis but in HTTP. I’m glad they didn’t. This still feels like HTTP, which is great.
- voltagex_ 8y agoMiddleboxes and bad HTTP servers/clients would explode, I guarantee it.
- friend-monoid 8y agoBut we have the PATCH method, which is... Non-standard, I think? And it works fine.
- icebraining 8y agoIt's been a standard for a few years: https://tools.ietf.org/html/rfc5789 https://tools.ietf.org/html/rfc5789
- lomnakkus 8y agoAlways use HTTPS?
- treve 8y agoI don't think we need more HTTP methods, unless a real case can be made for the existing ones not being good enough. The HTTP strongly discourages it. But I love the idea of topics being Uris (not just HTTP)
- jcrites 8y ago"topics" are indeed HTTP URLs: 1. Definitions Topic. An HTTP (or HTTPS) resource URL. However, you subscribe to a topic by interacting with a different "hub" URL, passing the "topic" URL as a parameter (`hub.topic`). Service discovery does appear to support HEAD requests. (See section 4.) Having a new HTTP verb for subscribing and publishing would seem like unnecessary complexity to me. Rather than ask "why not a new verb", I think a case would need to be made that a new verb is required, that the operation does not cleanly fit into the semantics of existing verbs. The existing verbs are capable of modeling quite a lot. With the protocol as they've described it, subscribing is just sending an HTTP POST to the hub URL, passing in the topic URL. That's a simple HTTP operation that a lot of clients and programs can be instructed to do easily. Requiring the use of a new HTTP verb will make interoperability difficult without apparent benefit.
- dnomad 8y ago> Having a new HTTP verb for subscribing and publishing would seem like unnecessary complexity to me. Complexity for who? Introducing a secondary "hub" resource here is just accidental complexity. If I want to subscribe to resource A why am I talking to a different resource B? And once you introduce a secondary resource now you need yet another service discovery mechanism to support discovery of these pseudo-resource hubs. (Heaven forbid using an existing service discovery mechanism like RDDL.) Honestly stuff like this is just so poorly thought out it's difficult to understand why W3C stamps approval on this crap. There's no consideration given to alternate protocols like WebSockets or XMPP and there's no attempt to layer on top of existing standards in a meaningful way (hub.secret -- really???). Worst of all there's no real understanding here of what it means for a resource to change. The entire Content-Distribution model is geared towards just one very narrow use case. It's clear the W3C is all about being "inclusive" and "moving fast" there's real fear of "overthinking" things -- but seriously if this is the result we'd probably better off with better standards once a decade then this.
- pjc50 8y agoNew HTTP verbs require support in the web server, and all proxies along the way, while using existing ones doesn't and is just part of the normal application framework.
- felixfbecker 8y agoThis reminds me a lot of the SUBSCRIBE method [0] in Microsoft’s WebDAV extensions (I think this is used for EAS?). You pass a Call-Back header with a URL that gets called with NOTIFY [1] [0] https://docs.microsoft.com/en-us/previous-versions/office/developer/exchange-server-2003/aa143117(v%3dexchg.65) https://docs.microsoft.com/en-us/previous-versions/office/de... [1] https://docs.microsoft.com/en-us/previous-versions/office/developer/exchange-server-2003/aa142938(v%3dexchg.65) https://docs.microsoft.com/en-us/previous-versions/office/de...
- deleted 8y ago[deleted]
- edhelas 8y ago"Hub deliver content to each subscriber with a POST" It's quite common on some social networks to have several thousands/millions subscribers for some pages/communities/accounts. Is it really wise to build a publish-subscribe delivery system on top of HTTP? This seems to be a huge overhead. In the meantime XMPP is already offering similar features (XEP-0060: Publish-Subscribe https://xmpp.org/extensions/xep-0060.html https://xmpp.org/extensions/xep-0060.html) for more than 10 years. It's implemented in several servers and can handle huge loads without problems (everything is handled in real-time through encrypted TCP sockets accross the network). We are building social networks on top of XMPP for several years now, you can check Movim (https://movim.eu https://movim.eu) and Salut à Toi (https://salut-a-toi.org/ https://salut-a-toi.org/) :)
- cbsmith 8y agoNot to mention PubSubHubub...
- erikrothoff 8y agoThis is the successor to PubSubHubbub. It’s pretty much the same API. https://websub.rocks https://websub.rocks is a great resource for implementors.
- zellyn 8y agoFrom the page linked: > WebSub was previously known as PubSubHubbub.
- mindcrime 8y agoIs it really wise to build a publish-subscribe delivery system on top of HTTP? I'm sure there is some use-case where this makes sense, but I agree with you. Probably most people wanting to do large scale pubsub should just be using XMPP, or possibly something like MQTT.
- friend-monoid 8y agoXMPP is an open TCP connection with its own protocol, like MQTT, Stomp, Redis, NATS, and all the *MQs. WebSub is mostly between HTTP servers, where one server can tell another that it wants a specific HTTP URL to be GET’d on a message. Each message requires a new TCP connection. So it will not handle very many messages - but I don’t think that’s the point.
- behrtam 8y agoSo it's an alternative to something like WAMP? (Web Application Messaging Protocol on top of websockets - https://wamp-proto.org/ https://wamp-proto.org/)
- friend-monoid 8y agoExcept you don’t have an open connection where you receive messages on. Instead, you register a callback url. It’s more like webhooks, but it’s it’s own thing, and not a setting on a page.
- TheChaplain 8y agoHow does this compare to ActivityPub used by Mastodon et al?
- erikrothoff 8y agoSee it more as an alternative to RSS. Aggregators subscribe and feed information to their users. The polling step is just switched to pushing.
- detaro 8y agoNot really an alternative, more of an addition to. The most common use case for Pubsubhubbub/WebSub is in combination with RSS/Atom feeds, informing subscribers about updates to those.
- tscs37 8y agoIt's part of the Social Group at W3C IIRC. WebSub solves problems around RSS, notably that clients have to poll a lot and smaller sites thusly need to maintain those requests. WebSub solves that by designating a hub that can handle this instead. Ie, you federate your blog feed to elsewhere. ActivityPub fills another niche, mostly everything around human interaction in social networks. WebSub could be used to feed data into ActivityPub networks.
- Communitivity 8y agoAfter reading the responses I don't see yet why the non-transport bits of WebSub can't be done with ActivityStreams and ActivityPub. That said, I haven't put the kind of time into it like the W3C Social Group has, nor am I recognized expert on it, so I'll hold further comment until I can finish and publish to Show HN a related proof-of-concept involving Peer-to-Peer topic based publish/subscribe using ActivityPub, that I've had on the back-burner for a while.
- gscott 8y agoJust bring back Fidonet
- deleted 8y ago[deleted]
- xg15 8y agoSo as with webhooks, this will require you to have a permanently reachable server sitting on the internet. Couldn't they at least have defined an alternative transport using websockets or SSE?
- icebraining 8y agoWebSub is essentially just the transport; the payload is usually Atom or RSS, but it's not specified. Had they included a websockets version, it would essentially be something almost completely different with the same name.
- EGreg 8y agoIs this compatible with new distributed crypto projects like scuttlebutt or dat, and if not, why not?
- icebraining 8y agoWebSub is just the standardization of PubSubhubbub, which has existed (and been used by publishers like Feedburner) for longer than those projects.
- nstart 8y agoSo essentially there's no difference right? When I was reading the spec I couldn't tell of any notable differences between the two texts.
- tonyg 8y agoIt is not, because it predates them both by several years.
- EGreg 8y agoThis makes me think the Web is dangerously close to becoming obsolete because it doesn’t embrace the new end to end encrypted, decentralized paradigms without central web servers and pins everything on http.
- optimuspaul 8y agoI really don't think that is even close to reality as the majority of users don't care all that much about e2e encryption (because they don't understand it) and that same user base has no idea what you are talking about with regard to decentralization. I would even venture to say that I'm not in a tiny minority of professionals that do care about e2e encryption and do understand decentralization but think that decentralization will never be much more than a science experiment. Users care about their experience and they want to have unencumbered conversations. The decentralized dreams impede those things.
- krn 8y agoSuperfeedr[1] (acquired by Medium in 2016[2]) was probably the biggest company yet, that based its entire business model on WebSub / PubSubHubbub protocol. [1] https://en.wikipedia.org/wiki/Superfeedr https://en.wikipedia.org/wiki/Superfeedr [2] https://techcrunch.com/2016/06/02/super-to-medium/ https://techcrunch.com/2016/06/02/super-to-medium/
- antpls 8y agoSo I read the comments but I still don't understand what problem this specification solves. Does it enable new use cases? Does it enable new trust models? Since it's only a server protocol, how do final users actually read the content? Edit : I don't understand the downvote. I use RSS daily to fetch news and I don't have problem with it. It's simple and deliver news to the edge (my mobile phone). I'm not saying WebSub is useless, I would like to understand what a 3 entities model brings to the table compared to a simple server-client delivery. What's more, the Subscriber entity cannot be a mobile device with the current network, because mobile internet providers block incoming GET requests. Therefore to fetch news, it has to be a pull model. Why not add a simple rational in the header of the spec, explaining the problem, the existing solutions, and why this new solution? Does it solve a security issue, a scalability issue or a trust issue?
- rakoo 8y agoWebSub is the successor to PubSubHubBub, which was designed to make distribution of RSS items more efficient (ie receive items as they are published on the host): - publisher would send updates, instead of having everyone poll - publisher would be protected from thundering herd if a content suddenly becomes popular - publisher and subscribers wouldn't need to exchange a full "page" of items when only one is needed
- antpls 8y ago> distribution of RSS items more efficient RSS is a 500kb static text file which is updated usually at most every hours. The whole file fits in server RAM and can already be served to thousands of people from cache with minimal CPU usage, or from Content Delivery Networks. A "full page of items" is easily compressed with gzip algorithms. Surely, the Internet isn't congested because of RSS...
- fenwick67 8y agoDon't forget about ETags.
- MuffinFlavored 8y ago
- ArtWomb 8y agoJust thinking beyond a replacement for RSS, this could be used for large-scale persistent virtual worlds. WebXR version of Second Life. GLTF as a content transmission format. As the Hub updates with a POST request, Subscribers callback could return local state. Hub becomes global state of truth. And there is no need to manage peer net or continuous WebSocket connections.
- Communitivity 8y agoXMPP has already been mentioned (thanks edhelas). What I didn't see mentioned was ActivityPub (https://www.w3.org/TR/activitypub/ https://www.w3.org/TR/activitypub/), or some of the advanced messaging features users have come to expect (e.g., store and forward, forms, automated route handlers). What I am curious about are the following questions? 1) What differentiates WebSub from XMPP? 2) What differentiates WebSub from ActivityPub? 3) How are you handling the N-squared delivery problem, if you are delivering content directly to each subscriber with HTTP POSTs? 4) Does WebSub currently support store and forward? If not, is that on a roadmap for a future version? 5) Same as 4, except for support for forms and form responses? Examples are a builtin Yes or No reply, or a builtin poll vote. 6) Same as 4, for automated message routing. 7) Why not have it be transport-agnostic, instead of mandating HTTP? And why HTTP? The growing trend is towards more decentralized. 8) How does this compare to Sir Tim Berners-Lee's SOLID (https://solid.mit.edu/ https://solid.mit.edu/)?
- smilbandit 8y agoI wish there was a pull aspect to it, like NEWNEWS with NNTP. If the subscriber is offline it seems that there doesn't seem to be a way to retrieve missing items except through the publisher.
- PaulHoule 8y agoWhat I don't get is what the motivation is for running the hub, other than "get bought by Medium?"
- sajal83 8y ago> The subscriber must be directly network-accessible and is identified by its Subscriber Callback URL Does this mean the subscriber needs to have a forwarded port open to the internet for this to work? Without IPv6, users behind NAT (and specifically behind CGNAT) wouldn't be able to use it.
- Muhammad-saleh 8y agoI think Websub is enhancement to XMPP. Both are asynchronous. Which means it should be faster than HTTP. Actually it's a push-based protocol where server sends single POST to the hub, the hub then sends that notification to all subscribers.