4 ms·
I’ve been working on Prolly, a Rust implementation of a content-addressed ordered map built on prolly tree(intro from dolthub https://www.dolthub.com/docs/archi
by forhappy 2mo ago
I’ve been working on Prolly, a Rust implementation of a content-addressed ordered map built on prolly tree(intro from dolthub https://www.dolthub.com/docs/architecture/storage-engine/prolly-tree/ https://www.dolthub.com/docs/architecture/storage-engine/pro...)
A prolly tree is similar to a B+ tree, but its node boundaries are determined by the data rather than by insertion order. Each node is addressed by the hash of its contents, and updates create a new root while sharing unchanged nodes with older versions.
This makes it useful when an application needs more than basic key/value storage: cheap snapshots, efficient diffs, three-way merges, deduplication, and incremental sync between replicas.
Some use cases I’m exploring include local-first applications, versioned database indexes, Git-like filesystem snapshots, agent memory and event logs, and reproducible RAG indexes where the exact data snapshot used for an answer can be recorded.
It’s a storage primitive rather than a complete database. The goal is to provide the ordered-map layer and let applications choose their own storage backend, data model, and conflict policy.
The project is still evolving, and I’d appreciate feedback—especially about real-world use cases, the API, and what is missing.
- lifty 2mo agoGreat primitive, I’m a big fan of prolly trees! You mentioned git like filesystem snapshots? How would you retrofit that on an existing filesystem? Or you’re talking about writing a new one?
- forhappy 2mo agoThank you for your kind words, yes, I do have lots of ideas on top of prolly, actually I'm building some of them like crab(a serverless git remote, will be open-source soon), compass(a versioned code graph https://compass.crab.build/ https://compass.crab.build/), trail(a low-level version control tooling for AI agent https://github.com/crabbuild/trail https://github.com/crabbuild/trail) silo(a versioned S3 client to make S3-compatible bucket version-controllable https://github.com/crabbuild/silo https://github.com/crabbuild/silo) Prolly is a building block, there are sooo many use cases, I have the cookbook how developers can utilize prolly tree in their projects: https://github.com/crabbuild/prolly/blob/main/docs/cookbook.md https://github.com/crabbuild/prolly/blob/main/docs/cookbook....
- dtkav 2mo agoThank you for sharing this. I've been looking into prolly trees for relay.md - I'll try swapping this in and see if i can provide any perspective for real world CRDT use cases. I'm happy to see the various key policies.
- forhappy 2mo agoI'd be super happy to collaborate, let me know if you have any feedbacks ;)