3 ms·
> you should be able to take Lucene's nice fast immutable data structure and stuff blocks of it (at the term level or below) into FDB values very efficiently.
by derefr 9y ago
> you should be able to take Lucene's nice fast immutable data structure and stuff blocks of it (at the term level or below) into FDB values very efficiently.
That sounds a lot like Datomic's "Storage Resource" approach, too! Would Datomic-on-FDB make sense, or is there a duplication of effort there?
- wll 9y agoIt most definitely would. Datomic’s single-writer system requires conditional put (CAS) for index and (transaction) log (trees) roots pointers (mutable writes), and eventual consistency for all other writes (immutable writes) [0]. I would go as far as saying a FoundationDB-specific Datomic may be able to drop its single-writer system due to FoundationDB’s external consistency and causality guarantees [1], drop its 64bit integer-based keys to take advantage of FoundationDB range reads [2], drop its memcached layer due to FoundationDB’s distributed caching [3], use FoundationsDB watches for transactor messaging and tx-report-queue function [4], use FoundationDB snapshot reads [5] for its immutable indexes trees nodes, and maybe more? Datomic is a FoundationDB layer. It just doesn’t know yet. [0] https://docs.datomic.com/on-prem/acid.html#how-it-works https://docs.datomic.com/on-prem/acid.html#how-it-works [1] https://apple.github.io/foundationdb/developer-guide.html?highlight=developer%20guide#transactions-in-foundationdb https://apple.github.io/foundationdb/developer-guide.html?hi... [2] https://apple.github.io/foundationdb/developer-guide.html?highlight=developer%20guide#range-reads https://apple.github.io/foundationdb/developer-guide.html?hi... [3] https://apple.github.io/foundationdb/features.html#distributed-caching https://apple.github.io/foundationdb/features.html#distribut... [4] https://docs.datomic.com/on-prem/clojure/index.html#datomic.api/tx-report-queue https://docs.datomic.com/on-prem/clojure/index.html#datomic.... [5] https://apple.github.io/foundationdb/developer-guide.html?highlight=developer%20guide#snapshot-reads https://apple.github.io/foundationdb/developer-guide.html?hi...
- mvc 9y agoCan't see datomic itself ever doing that though because they'd have to support those features in all the backends.