3 ms·
OK, so cynicism warning here. After seeing what seems like oodles of sqlite and rocksdb plus a bespoke cross-machine coordinator databases posted on HN, is thi
by AtlasBarfed 4y ago
OK, so cynicism warning here.
After seeing what seems like oodles of sqlite and rocksdb plus a bespoke cross-machine coordinator databases posted on HN, is this like the new employment guaranty architecture? Do any of these have actual ability over something like ScyllaDB (I'm taking cassandra out because of the GC pauses probably being a problem)
RocksDB in particular is a single node SSD-tuned log-structured merge tree "database" written in a non-GC language. ScyllaDB is the same, but comes with the cassandra-protocol scaling and multiple datacenters.
Sure, go ahead and market your whatever, but if you're just (IMO) repackaging a log structured merge tree engine, with various newly implemented distributed transaction / conflict resolution coordination schemes .... you should probably show some good due diligence on if they are, you know, somewhat correct.
The Jepsen test suite basically shows every distributed data processing engine as flawed the first couple times. So much so I don't really trust your new fangled engine unless you pass Jepsen to some degree.
Is there some comparison metrics (lies, big lies, and benchmarks and all that) showing why these semi-bespoke databases are being used versus other, say, more proven, schemes?
- refset 4y agoXTDB is a deterministic, single-writer system that leans on existing tech to handle all the hard distributed & durability aspects (e.g. piggy-backing on Kafka for ACID + HA), so Jepsen would have relatively limited applicability but it couldn't hurt to sanity check that! I generally sympathise with your cynicism though that distributed consistency is much harder to get correct than most people realise. My own attraction to niche databases is the promise of better abstractions - the world desperately need those.