3 ms·

This looks really cool! Perhaps you should show some nontrivial examples of (read) queries in RefineASM. Also, what are your plans (if any) for non primary in

This looks really cool! Perhaps you should show some nontrivial examples of (read) queries in RefineASM. Also, what are your plans (if any) for non primary indexing? Storage plans seem like a good fit, but will you transform RefineASM to maintain the indexes or what? What about using the indexes?

losfair 5y ago
There are some non-trivial examples of read and update queries in the test: https://github.com/losfair/RefineDB/blob/main/rdb-analyzer/src/data/treewalker/asm/asm_test.rs https://github.com/losfair/RefineDB/blob/main/rdb-analyzer/s... For example, here's a test that generates a list of integers, inserts the list into the database as set members, computes the sum and verifies it, and reads back the list: https://github.com/losfair/RefineDB/blob/ddac0d35b16c01105ddbe65f2b3605713fde4be4/rdb-analyzer/src/data/treewalker/asm/asm_test.rs#L392 https://github.com/losfair/RefineDB/blob/ddac0d35b16c01105dd... . The read-back part looks like: // Schema /* type Store { collection_a: set<Number>, collection_b: set<Number>, } type Number { @primary value: int64, } export Store store; */ graph main(root: schema): map { list1: list<int64>, list2: list<int64>, } { list1 = reduce(decompose_numbers) create_map create_list(int64) root.store.collection_a; list2 = reduce(decompose_numbers) create_map create_list(int64) root.store.collection_b; return m_insert(list1) list1 $ m_insert(list2) list2 create_map; } graph decompose_numbers(_unused: map{}, current: list<int64>, v: Number): list<int64> { return v.value : current; } And it's trivial to modify the reduce function `decompose_numbers` to return a more interesting aggregated value, e.g. the sum. The syntax is not yet very optimized for humans though (hence the name RefineAsm).
losfair 5y ago
On non-primary indexing: It's a nice thing to have but I haven't decided on the design yet. My current preference is to implement the non-primary index feature as a library once the human-friendly higher layer of RefineAsm (RefineQL) is built.
- voidmain 5y ago
  Consider thinking of index maintenance as part of your storage mapping layer. Basically you have redundant storage plans for the same schema. There is nothing particularly special about primary row indexes - that is just one physical representation of your schema, as are all indexes. You could have, say, a column oriented representation of the same data instead, or a covering index, or all of the above.

losfair 5y ago

There are some non-trivial examples of read and update queries in the test: https://github.com/losfair/RefineDB/blob/main/rdb-analyzer/src/data/treewalker/asm/asm_test.rs https://github.com/losfair/RefineDB/blob/main/rdb-analyzer/s... For example, here's a test that generates a list of integers, inserts the list into the database as set members, computes the sum and verifies it, and reads back the list: https://github.com/losfair/RefineDB/blob/ddac0d35b16c01105ddbe65f2b3605713fde4be4/rdb-analyzer/src/data/treewalker/asm/asm_test.rs#L392 https://github.com/losfair/RefineDB/blob/ddac0d35b16c01105dd... . The read-back part looks like: // Schema /* type Store { collection_a: set<Number>, collection_b: set<Number>, } type Number { @primary value: int64, } export Store store; */ graph main(root: schema): map { list1: list<int64>, list2: list<int64>, } { list1 = reduce(decompose_numbers) create_map create_list(int64) root.store.collection_a; list2 = reduce(decompose_numbers) create_map create_list(int64) root.store.collection_b; return m_insert(list1) list1 $ m_insert(list2) list2 create_map; } graph decompose_numbers(_unused: map{}, current: list<int64>, v: Number): list<int64> { return v.value : current; } And it's trivial to modify the reduce function `decompose_numbers` to return a more interesting aggregated value, e.g. the sum. The syntax is not yet very optimized for humans though (hence the name RefineAsm).

losfair 5y ago

On non-primary indexing: It's a nice thing to have but I haven't decided on the design yet. My current preference is to implement the non-primary index feature as a library once the human-friendly higher layer of RefineAsm (RefineQL) is built.

voidmain 5y ago
Consider thinking of index maintenance as part of your storage mapping layer. Basically you have redundant storage plans for the same schema. There is nothing particularly special about primary row indexes - that is just one physical representation of your schema, as are all indexes. You could have, say, a column oriented representation of the same data instead, or a covering index, or all of the above.