3 ms·
What I was suggesting was moving the binary log replaying OUT of the database layer, which is where I imagined the efficiency losses might arise. I'm still fle
by hardwaresofton 10y ago
What I was suggesting was moving the binary log replaying OUT of the database layer, which is where I imagined the efficiency losses might arise.
I'm still fleshing this out in my head, but the idea is like: maintain a log of SQL commands that were to be done on the database at a layer right _before_ the database layer. then, as writes come in, they go to the layer-before, get replicated/distributed, and get copied to every node, and you could get easy (?) multi-master support. At the very least, you could do master-slave replication at the same cost that databases are doing it now, minus any inefficiencies from not using the specialized binary log each DB uses today. I guess the pre-database layer would do more gossiping than anything (in a multi-master type setup).
The real benefit here is that you would then be able to distribute any database that just supported SQL instantly (and even some that don't, with a properly written adapter), by composing it with another external (rather than internal) management process. There might even be some efficiencies to be gained if the databases are sharded -- localized data might go to the wrong node, but live there long enough to get reads and stuff like that.
I guess maybe instead of saying I'm thinking of building a layer on top, I'd rather pull binary log replication out of the databases themselves, and find some close-to-SQL log of commands that all DBs can share.