I'm using "topic-based versioning"[0] and it solves most of the issues presented here. The principles are quite similar to what you would find in a database, wi

I'm using "topic-based versioning"[0] and it solves most of the issues presented here. The principles are quite similar to what you would find in a database, with a central data repository and a write-ahead log. Each "table" is organized in ordered, write-only topics (or folders) containing immutable messages (or files). Each operation is also adding a message into a special topic we use as log. Each participant have to remember the message id ("cursor") at which it was on the last sync and fetch new messages on the log first. Then, it's only a matter of applying the new ops until the desired point-in-time. - decentralized, private, efficient: you own a complete copy of all versioned datasets, and you can store deltas between versions since all messages are ordered in topics. Each sync is done like a "git pull --rebase". - reliable: If you follow the principles, conflicts are not possible. - collaborative: Here's the slightly difficult requirement. You can choose to defer message id ("cursor") allocation to another distributed service (like zookeeper, for example) and accept to be online to register new write operations on a topic ; or force the system to be linearizable. [0]: I wrote an article describing this design pattern here : https://medium.com/bcggamma/topic-based-versioning-architecture-for-scalable-ai-application-c926ffa92c1 https://medium.com/bcggamma/topic-based-versioning-architect... ! Any feedback welcomed !