6 ms·
Ask HN: Why are there no easy-to-use highly-available open source databases?
I'm a single engineer who wants to run a database by myself, no cloud vendors involved. I don't really care about scalability to a point, I don't really care about the API to interact with the database. I just want something that I can set up on multiple machines with minimal effort and have the loss of one of those nodes not cause catastrophic failures. I want the experience of something like etcd without being limited to a few gigabytes of data in the cluster (a Terabyte or maybe a couple hundred Gigabytes as a limit would probably be fine). I've spent a lot of time looking through the various options and wrote up my thoughts on why each of them wasn't a good fit.
Postgres: A great database but HA options either have far too many moving parts, require manual intervention, are proprietary, or are Fake Open Source. Seems like there's lack of interest in addressing this because companies that sponsor development generate a lot of value off complex operations
MySQL: This database has some odd behavior but that would be fine if it weren't Fake Open Source
MariaDB: HA Options are poor or abandoned
SQL Server: Proprietary
Oracle: haha
CockroachDB: Fake Open Source
YugabyteDB: Fake Open Source. Special shout out here for not even linking to instructions for how to build the database in the readme.
MongoDB: Proprietary, and even before the license change it was Fake Open Source
Cassandra: Not fun to run, and even though I said I didn't care about API I don't necessarily love how it works. But it comes probably the closest?
ScyllaDB: Fake Open Source
TiDB: Fake Open Source
Singlestore: Fake Open Source
FoundationDB: This one comes close but its beginning as a proprietary database really hurt its community, which is way smaller now than it should be. Could grow into something great if more folks got behind it.
etcd: not suitable for use above a couple GB of data
What do I mean by Fake Open Source? A project that has a large percentage of its contributors beholden to a single organization/entity to me is not really open source in spirit. I'm looking for a project where I can feel confident my contributions won't effectively end up behind some proprietary license down the line if/when the VC backed organization that primarily sponsors development decides it needs to protect itself from AWS. If there's an "Enterprise" product and the organization calls the source code for the main project the "Community Edition" or something like it, it's not Real Open Source. If a single organization shuts down and contributions fall off a cliff (https://github.com/rethinkdb/rethinkdb/graphs/contributors) it's not Real Open Source. There are lots of Real Open Source projects with great communities of users/contributors, but many of the newer databases don't have legitimate open source development communities behind them, in my opinion.
I've probably missed some examples. Mostly, I wonder why there hasn't been a general purpose open source database that does the operations stuff as well as the proprietary databases do. Am I missing something?
- jka 5y agoYep, your assessment seems broadly accurate to me. I was going to suggest Cassandra until I saw it in the list. I'm {interested in/optimistic about} FoundationDB too, although haven't had a chance to use it in practice yet. Out of curiosity: what would your preferred choice(s) be to fit these requirements using existing proprietary products?
- database_guy 5y agoFrom my research I think Cockroach, Yugabyte, and Scylla are all interesting and solid products depending on the exact API/guarantees you're looking for. For products I don't even have the option to self-host I think DynamoDB and Cloud Spanner are great products, probably among others.
- jka 5y agoCool, thanks - yep, DynamoDB seems fairly reliable, and that's also good to hear another vote in support of Cloud Spanner. Glad to have learned about rqlite from this thread as well.
- PaulHoule 5y agoI'd say Oracle has been a better steward of MySQL than Sun was back when Sun was a separate company.
- endisneigh 5y agoI’m not understanding why it matters if it’s fake open source or not. Even with “real open source” there’s no guarantee your contributions will be included. Or that the license won’t change in the future.
- jakobdabo 5y agoI'm not sure if Percona's HA solutions are any better than what MariaDB offers, but it's not in your list, so maybe worth mentioning. https://www.percona.com/services/support/high-availability https://www.percona.com/services/support/high-availability https://www.percona.com/blog/2021/04/14/percona-distribution-for-mysql-high-availability-with-group-replication-solution/ https://www.percona.com/blog/2021/04/14/percona-distribution...