4 ms·
Ask HN: WebSocket data architecture strategies: idempodent vs. incremental data?
How have you guys architected your data over websockets? Do you use certain strategies to reduce latency with less data over the wire and/or strategies to ensure local memory doesn't exceed a certain amount?
- colesantiago 5y agoHow are you currently doing this? did you try going for a 3rd party provider like Firebase, Ably or Pubnub?
- 2upmedia 5y agoPlanning stages, so no 3rd parties yet. Considering strategies because I need live connections with as low latency as possible so I'm probably going to be managing the websocket connections myself on the server side, unless there's something else that will give me the same benefits and nearly no latency.
- aranchelk 5y agoIf I understand the question: you don’t have to choose between idempotence and efficient incremental updates if you can give your application data a revision number. Send your incremental update(delta) with a reference to the app data’s last revision. Increment the revision number after applying each delta and reject any delta that doesn’t reference the current version on the server, they’ll only be applied at most once, so they’re idempotent. There are fancier versions of this scheme that can be devised based on the business rules for your application.
- 2upmedia 5y agoWow. Never heard of that one. Very useful thanks.
- 2upmedia 5y agoHappen to know of any places where I could find high level data architecture concepts like that? Or did you know that just by experience?
- aranchelk 5y agoI’m searching now and I see multiple books on designing distributed systems. They’re probably worth looking at, but I haven’t read any of them so I can’t make a specific recommendation. I’ve learned a lot from reading about the architecture of existing distributed systems e.g CouchDb, Git, and Bitcoin.
- 2upmedia 5y agoAwesome. Thanks.
- bem 5y agoI think you could find some inspiration in cryptocurrency exchanges. Most of them expose public websockets for prices, order books, trades, etc. They're high volume with lots of subscribers. For example, check out the docs for the Binance websockets (they use both snapshot and delta messages): https://github.com/binance/binance-spot-api-docs/blob/master/web-socket-streams.md https://github.com/binance/binance-spot-api-docs/blob/master... If you could tell me a little more about the data format, data volumes, number of subscribers, and how you get the data on the backend, I can try to give you some more concrete advice.