4 ms·

I might need to look at Crux again for my prospective it was missing pull syntax and schema on write But reading from this thread that documents are converted

by slifin 6y ago

I might need to look at Crux again for my prospective it was missing pull syntax and schema on write But reading from this thread that documents are converted into EAVT for every attribute? Sounds interesting

dustingetz 6y ago
Without schema-on-write, the A in EAVT has no meaning. A tells us nothing about V. That makes it basically impossible for declarative systems like Hyperfiddle to understand your data structures and auto-generate UIs and such from the attributes. The fourth goal of Datomic is to "enable declarative data programming in applications" [1]. That's the design goal that makes Datomic so interesting to me, and Crux does not share it. Which is fine. Crux have given great reasons for doing things that way. https://www.infoq.com/articles/Datomic-Information-Model/ https://www.infoq.com/articles/Datomic-Information-Model/
- refset 6y ago
  Within Crux's new transaction functions you are able to run queries against the entire database and (very soon) also against speculative transactions, to enforce all manner of constraints and control whether the transactions succeed/fail. Schema-on-write is therefore a matter of funnelling all `put` & `delete` operations through these functions. I suspect someone could even create a library of transaction functions to emulate the entire Datomic data model, if so desired. However, I still believe schema-on-read is more desirable in general though, as you can simply extend a query to only care about the AV combinations which conform to some definition for that A. The Calcite SQL integration we've been building works like this. Conformance could also be processed asynchronously and documents be labelled as "conformed" as/when to reduce the burden during general queries. (also, hey!)
refset 6y ago
Things have been moving quickly in recent months! We've already done some prototyping on a pull syntax (using EQL) and will start proper work on it after the 1.9 release is out. Alongside some less-exciting changes 1.9 will include full transaction function support, so schema-on-write is much simpler to achieve now. It took us time figure out how to resolve it whilst not completely breaking bitemporality & eviction. Here are a few tests showing how it to use transaction functions now: https://github.com/juxt/crux/blob/b13b4da988ed7e91dc7685e5d5ab5c374194895f/crux-test/test/crux/tx_test.clj#L549 https://github.com/juxt/crux/blob/b13b4da988ed7e91dc7685e5d5... > documents are converted into EAVT for every attribute? The indexes are certainly schemaless, which means that every document attribute gets turned into a set of one or more triples that can be joined against without needing to declare anything about the attributes upfront, but internally the indexes don't follow the same EAVT pattern you may be familiar with from Datomic etc. checkout the ClojuTRE talk (linked in these comments) for a good overview on roughly what the internal indexes look like. It's a little out of date now, but still gives a strong impression for how things work today.

by slifin 6y ago

dustingetz 6y ago

Without schema-on-write, the A in EAVT has no meaning. A tells us nothing about V. That makes it basically impossible for declarative systems like Hyperfiddle to understand your data structures and auto-generate UIs and such from the attributes. The fourth goal of Datomic is to "enable declarative data programming in applications" [1]. That's the design goal that makes Datomic so interesting to me, and Crux does not share it. Which is fine. Crux have given great reasons for doing things that way. https://www.infoq.com/articles/Datomic-Information-Model/ https://www.infoq.com/articles/Datomic-Information-Model/

refset 6y ago
Within Crux's new transaction functions you are able to run queries against the entire database and (very soon) also against speculative transactions, to enforce all manner of constraints and control whether the transactions succeed/fail. Schema-on-write is therefore a matter of funnelling all `put` & `delete` operations through these functions. I suspect someone could even create a library of transaction functions to emulate the entire Datomic data model, if so desired. However, I still believe schema-on-read is more desirable in general though, as you can simply extend a query to only care about the AV combinations which conform to some definition for that A. The Calcite SQL integration we've been building works like this. Conformance could also be processed asynchronously and documents be labelled as "conformed" as/when to reduce the burden during general queries. (also, hey!)

refset 6y ago

Things have been moving quickly in recent months! We've already done some prototyping on a pull syntax (using EQL) and will start proper work on it after the 1.9 release is out. Alongside some less-exciting changes 1.9 will include full transaction function support, so schema-on-write is much simpler to achieve now. It took us time figure out how to resolve it whilst not completely breaking bitemporality & eviction. Here are a few tests showing how it to use transaction functions now: https://github.com/juxt/crux/blob/b13b4da988ed7e91dc7685e5d5ab5c374194895f/crux-test/test/crux/tx_test.clj#L549 https://github.com/juxt/crux/blob/b13b4da988ed7e91dc7685e5d5... > documents are converted into EAVT for every attribute? The indexes are certainly schemaless, which means that every document attribute gets turned into a set of one or more triples that can be joined against without needing to declare anything about the attributes upfront, but internally the indexes don't follow the same EAVT pattern you may be familiar with from Datomic etc. checkout the ClojuTRE talk (linked in these comments) for a good overview on roughly what the internal indexes look like. It's a little out of date now, but still gives a strong impression for how things work today.