5 ms·

> It’s one of the most complex system one can develop The same could probably be said about the internal combustion engine, but it might soon be replaced by el

by krn 7y ago

> It’s one of the most complex system one can develop The same could probably be said about the internal combustion engine, but it might soon be replaced by electric batteries, which provide a much more elegant solution. I believe that "unbundled" databases, such as Crux[1], can become the electric batteries of the database world by making a lot of the current complexity irrelevant. [1] https://opencrux.com/ https://opencrux.com/

detaro 7y ago
What makes crux's internals less complicated than a more conventional database?
- tincholio 7y ago
  I suspect the gp was referring to it using either Kafka or RocksDB for storage.
- refset 7y ago
  1) the single-writer principle of the transaction log means there's no need for any transactional locking 2) the separation of reads and writes allows for elegant horizontal read-scaling without coordination/consensus 3) pluggable storage backends implemented as simple Clojure protocols (as the sibling comment mentions), which eliminates a large number of performance and durability concerns 4) combining schema-on-read with entity-attribute-value indexing means there's no need to interpret and support a user-defined schema 5) Datalog is simpler to implement and use than the full SQL standard or alternative graph query languages ...I work on Crux :)
  - simplify 7y ago
    Please tell us more about point 5!
    - refset 7y ago
      Crux uses a Worst-Case Optimal Join [0] algorithm with bitemporal indexes, and the Datalog-specific query layer is implemented in less than a thousand lines of Clojure: https://github.com/juxt/crux/blob/master/crux-core/src/crux/query.clj https://github.com/juxt/crux/blob/master/crux-core/src/crux/... SQL certainly provides a lot of bells and whistles but Crux has the advantage of consistent in-process queries (i.e. the "database as a value") which means you can combine custom code with multiple queries efficiently to achieve a much larger range of possibilities, such as graph algorithms like Bidirectional BFS [1]. [0] https://arxiv.org/pdf/1803.09930.pdf https://arxiv.org/pdf/1803.09930.pdf [1] https://github.com/juxt/crux/blob/master/docs/example/imdb/src/imdb/main.clj https://github.com/juxt/crux/blob/master/docs/example/imdb/s...
mamcx 7y ago
Whoa! somebody actually try to implement the log as the central component of a db! After read https://www.confluent.io/blog/turning-the-database-inside-out-with-apache-samza/ https://www.confluent.io/blog/turning-the-database-inside-ou... I wondered about that. I think that make the log first class and "plug" relational tables (optionally) will make a amazing database engine. In short, you PERSIST your commands: POST /City .. PUT /City .. DELETE /City .. and put listeners that decide if persist or not the commands, this allow to do: POST /SendMail (to:...) and have the flexibility to bundle the domain logic on top of the data logic in a single lang. This is my long term goal..

by krn 7y ago

detaro 7y ago

What makes crux's internals less complicated than a more conventional database?

tincholio 7y ago
I suspect the gp was referring to it using either Kafka or RocksDB for storage.
refset 7y ago
1) the single-writer principle of the transaction log means there's no need for any transactional locking 2) the separation of reads and writes allows for elegant horizontal read-scaling without coordination/consensus 3) pluggable storage backends implemented as simple Clojure protocols (as the sibling comment mentions), which eliminates a large number of performance and durability concerns 4) combining schema-on-read with entity-attribute-value indexing means there's no need to interpret and support a user-defined schema 5) Datalog is simpler to implement and use than the full SQL standard or alternative graph query languages ...I work on Crux :)
- simplify 7y ago
  Please tell us more about point 5!
  - refset 7y ago
    Crux uses a Worst-Case Optimal Join [0] algorithm with bitemporal indexes, and the Datalog-specific query layer is implemented in less than a thousand lines of Clojure: https://github.com/juxt/crux/blob/master/crux-core/src/crux/query.clj https://github.com/juxt/crux/blob/master/crux-core/src/crux/... SQL certainly provides a lot of bells and whistles but Crux has the advantage of consistent in-process queries (i.e. the "database as a value") which means you can combine custom code with multiple queries efficiently to achieve a much larger range of possibilities, such as graph algorithms like Bidirectional BFS [1]. [0] https://arxiv.org/pdf/1803.09930.pdf https://arxiv.org/pdf/1803.09930.pdf [1] https://github.com/juxt/crux/blob/master/docs/example/imdb/src/imdb/main.clj https://github.com/juxt/crux/blob/master/docs/example/imdb/s...

mamcx 7y ago

Whoa! somebody actually try to implement the log as the central component of a db! After read https://www.confluent.io/blog/turning-the-database-inside-out-with-apache-samza/ https://www.confluent.io/blog/turning-the-database-inside-ou... I wondered about that. I think that make the log first class and "plug" relational tables (optionally) will make a amazing database engine. In short, you PERSIST your commands: POST /City .. PUT /City .. DELETE /City .. and put listeners that decide if persist or not the commands, this allow to do: POST /SendMail (to:...) and have the flexibility to bundle the domain logic on top of the data logic in a single lang. This is my long term goal..