5 ms·
This is a nice writeup of the problems in using Webhooks for State Synchronization. I also noticed that the proposed solution is a pseudo IETF-style draft prot
by toomim 2mo ago
This is a nice writeup of the problems in using Webhooks for State Synchronization. I also noticed that the proposed solution is a pseudo IETF-style draft protocol called SCROLL... that happens to be remarkably similar to an actual IETF draft I am bringing to IETF 127 this November called "Braid-HTTP Subscriptions."
Both drafts request a subscription with a GET plus a header:
Scroll Request:
GET /scroll/feed/customers
Prefer: stream
Braid Request:
GET /customers
Subscribe:
In both systems, the GET leaves its response open to stream events. SCROLL responds with application/x-ndjson. Braid subscriptions are a 209 Multiresponse, with content-type application/http-history. This lets them support more than just JSON. You can send updates to the state of CSV, or PNGs, XML, HTML, plain text, or any media type.
The author noted that it's hard to get adoption. Well, the reason that Webhooks are so common is that they are bog-standard HTTP. For this to get adopted, we need to put it into bog-standard HTTP. So we need to go to the IETF, and and extend HTTP in a general way to support state synchronization. It should just work for any existing HTTP media type (not just JSON), and any resource/URL (not just special /scroll/* URLs), and any way of marking timestamps (not just the ordered strings proposed in SCROLL).
Then we can bake this stuff into HTTP, and thus into all our bog-standard libraries, utilities, and code, and you won't have to reimplement the same sync-logic-over-webhooks again, and again, and again.
Reach out if you're interested!
- weli 2mo agoI reached out through email :) Just one correction. My spec doesn't force /scroll/ URL's, just proposes it as a convention.
- bobbiechen 2mo agoIs it accurate to say this is something like long polling except you continue to hold the connection open for subsequent updates? Does this mean a server potentially needs to hold open a very large number of connections (one per client) even if there are no updates? And why formalize on HTTP rather than on a similar protocol over websockets?
- anamexis 2mo agoAnd isn't that just Server-Sent Events (SSE)?
- Joeri 2mo agoThere’s also the Linked Data Event Streams (LDES) standard, which is a way of hosting a log as a set of linked http documents, fetched via polling and link navigation. Is there really anything more needed than a webhook for the notification and an LDES for the log? https://semiceu.github.io/LinkedDataEventStreams/releases/1.0.0/index.html https://semiceu.github.io/LinkedDataEventStreams/releases/1....
- dzonga 2mo agothe solution is good but what tends to happen is webhook providers are not gonna work on such a solution - why - because it puts 'work' on them. hell this is without the proposal for a new protocol - just a 'GET' stream or paginated one like the author said. whereas with web hooks - a consumer has to do all the work - as the article above outlined.
- weli 2mo agoDoesn't webhooks put even more work on them? You need to create a webhook dashboard, sometimes with multiple environments and administration endpoints, a way to set HMAC secrets, retriability and deliverability mechanisms, a local tunnel CLI for development, etc. With scroll if you already own a event log system the only work you need to do is exposing it to the client securely, same API keys as the rest of your existing REST api's.
- RyoSaeba89 2mo ago[flagged]
- angrysponge 2mo agoI'm sorry, but I have to ask. bog-standard?
- Induane 2mo agoNever visited a standard bog?
- projektfu 2mo agocf. vanilla
- bowersbros 2mo agoIt's a British colloquial term meaning basic / expected / ordinary