4 ms·
The popular databases we use today such as Postgres and Mysql are pretty terrible. Backups are extremely complicated to set up, and restoring from backups is ha
by gizmo 4y ago
The popular databases we use today such as Postgres and Mysql are pretty terrible. Backups are extremely complicated to set up, and restoring from backups is hard. Multimaster setups almost work, but not completely. You want a geo-distributed high availability database? Forget about it. The popular databases today just don't do the things every web startup wants from a database.
So I totally get why people want to make better database software, but at the same time, I don't understand how startups think the "move fast break things" approach will work here.
The value of a buggy, incomplete, poorly documented database is 0, and the value of a database system that actually works is infinite. It's totally discrete.
How this is not going to end up like every other database startup? Funding gets cut eventually because they can't demonstrate traction, but they can't demonstrate traction until the database product is actually way way better than the old dogs. Which will take 10 years if not 20.
MongoDB is the only new database that managed to break through, and it was a single-threaded document store without proper ACID properties. For a database, it was simple, but it would still regularly eat your data. Today, something like MongoDB would never get traction.
I wish these rising wave guys luck, but I don't see any indication they understand the obstacles they're facing.
- threeseed 4y ago> MongoDB is the only new database that managed to break through Snowflake, Cassandra, DuckDB, Clickhouse, Redis, InfluxDB, DynamoDB, BigQuery etc. There are many databases that have broken through they just are far more niche focused. > but it would still regularly eat your data No it didn't which is why it became so popular. The whole fsync saga was always overblown because (a) every client that shipped set it a safe default anyway and (b) it was fixed almost immediately.
- crabbone 4y agoI would love to hear more about "fsync saga" if you have any references. I know that PostgreSQL had "problems" with fsync(), but they never ended. They just accepted defeat, but so would everyone else who relies on filesystems to store their data. It's even more ingrained than that. The way hardware works, and, especially, the communication protocols around it (eg. SCSI or NVMe) are structured, fsync() is always going to be a problem.
- gizmo 4y agoSee this chain of articles: https://aphyr.com/posts/284-jepsen-mongodb https://aphyr.com/posts/284-jepsen-mongodb The mongodb organization repeatedly poo-pooed the issues with data loss, and only after repeated public exposure of the issues did they do anything about it. I don't know why people aggressively deny this when anybody can just google it and judge for themselves.
- threeseed 4y agoNo one denies anything or that the company handed it well. But this was never a real-world issue. All of the clients set safe defaults and the documentation was very clear about default fsync behaviour.
- crabbone 4y agoThanks! This was very interesting. But, did they actually fix this? The three posts are about the problems, but they go up to 3.X version of MongoDB, and the most recent seems to be at 6.X.