3 ms·
One of my dreams is building a hybrid of RDBMS and Protocol Buffers. It would look like a bunch of nested structs that can be kept in memory, fetched from disk
by cousin_it 10y ago
One of my dreams is building a hybrid of RDBMS and Protocol Buffers. It would look like a bunch of nested structs that can be kept in memory, fetched from disk or over the network. The schema would be kept in .proto files, and you'd be able to reload it at runtime (existing data would be handled according to protobuf schema evolution, because each field in each struct is numbered and old numbers aren't reused). Most joins would be gone because nested structs are allowed, but you'd still have an SQL-like language for more general queries (designing such a language when nesting is allowed is a bit tricky, but not very). Things like indices and transactions would also exist at every level (memory, disk, network) because they are useful at every level.
The end goal is eliminating impedance mismatch, because your current in-memory state, your RPCs and your big database would be described by the same schema language, somewhat strongly typed but still allowing for evolution. I have no idea if something like that already exists, though.
- weavie 10y agoThe problem is that what joins to what differs depending on the context for what you are looking for. Sometimes you join an order to supplier, other times it will be to customer and other times it will be to a list of products. Sometimes, all of them need to be included. With nested structs the parent/child relationship is fixed. If your query needs to invert this relationship, you essentially have to search through your entire database which will be incredibly slow and resource intensive. Either that, or you just store your data and their relationships separately and then allow for highly optimised searches to be conducted between these relationships according to the query and statistics you hold about the data ... but then you have just reimplemented an RDBMS..
- cousin_it 10y agoI guess the idea is to make the schema as readable as possible using nesting, and then use indices for the rest.
- TimJYoung 10y agoYep, it was one of the things used in quite a few systems before SQL became more widespread: https://en.wikipedia.org/wiki/MultiValue https://en.wikipedia.org/wiki/MultiValue
- jimktrains2 10y ago> One of my dreams is building a hybrid of RDBMS and Protocol Buffers. That's actually one my goal with https://github.com/jimktrains/drsdb https://github.com/jimktrains/drsdb. All communication would be done via protocol buffers and schemas could be created or loaded at runtime to be verified and used. The primary goal begin a distributed RDBMS written in rust.