3 ms·
It looks like writes are buffered in an in-memory write ahead log before being written to object storage, which means that if the writer box dies, then you lose
by drodgers 2y ago
It looks like writes are buffered in an in-memory write ahead log before being written to object storage, which means that if the writer box dies, then you lose acknowledged writes.
I've built something similar for low-cost storage of infrequently accessed data, but it uses our DBMS (MySQL) for the WAL (+ cache of hot reads), so you get proper durability guarantees.
The other cool trick to use is to use Bε-trees (a relatively recent innovation from Microsoft Research) for the object storage compaction to minimise the number of write operations needed when flushing the WAL.
- quadrature 2y agoYou have the ability to choose your durability guarantee. You can choose to have synchronous writes, in which case the client blocks until the write is acknowledged. https://docs.rs/slatedb/latest/slatedb/config/struct.WriteOptions.html https://docs.rs/slatedb/latest/slatedb/config/struct.WriteOp...
- 0x1ceb00da 2y agoIs there something similar that caches recent changes locally if the device is offline and uploads them when it comes online?