4 ms·
When I saw the headline, I thought it would be a neat article. Then when seeing the first data representation snippet, I thought "OK cool, that's what you conce
by vinkelhake 3y ago
When I saw the headline, I thought it would be a neat article. Then when seeing the first data representation snippet, I thought "OK cool, that's what you conceptually have". Then I realized that it was the actual data representation.
The article is still neat, but more for the presentation. Of course you can get a huge reduction in data size if what you are starting out with is just about the most bloated representation you can imagine.
https://thedailywtf.com/articles/The-Speedup-Loop https://thedailywtf.com/articles/The-Speedup-Loop
- jakelazaroff 3y agoAuthor here — the original data structure is basically the plain state you'd get by implementing these CRDTs. So I agree that the original data structure uses a lot of space, but I also think the post is broadly representative of the type of improvement you can get if you started with a naive implementation.
- beambot 3y agoWhat sort of savings would you get with basic compression algorithms...?
- jakelazaroff 3y agoLooks like someone else in this thread tried it out: https://news.ycombinator.com/item?id=37920255 https://news.ycombinator.com/item?id=37920255 TL;DR brotli does the best — it compresses the JSON down to 21.42kb, which is a 96.6% reduction. That's probably the most bang for the buck. If you take the final binary encoding format from the end of the post, though, brotli crunches it down to 5.99kb — a 99% reduction!
- masklinn 3y agoWould also be interesting to run-length-decode the final file and compress that, to see how much (or how little) the RLE help. Though I don’t think it’s super useful over the wire, LZ4 could also be an interesting thing to look at.
- dang 3y agoWhat would be an accurate and neutral title? (Edit: I've taken out the "98%" bit per the HN guidelines: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html)
- jakelazaroff 3y agoWhoops, sorry — I didn't realize that was against the guidelines!