3 ms·

Firstly, to the OP, that is an excellent post-mortem. Introspective, goes deep, generally confidence inspiring even though a part of it is literally "we were in

Firstly, to the OP, that is an excellent post-mortem. Introspective, goes deep, generally confidence inspiring even though a part of it is literally "we were incompetent and didn't know it", well, lessons learned. I'd like to add some colour, although I've never used RethinkDB so take it all with a large pile of salt. lobster_johnson says: > Rethink doesn't even have transactions The post-mortem dwells heavily on how RethinkDB focused on correctness and was beaten by MongoDB which is, to put it mildly, not concerned about correctness. But Rethink is a JSON DB that apparently doesn't have transactions and which on its web page says "Query JSON documents with Python, Ruby, Node.js or dozens of other languages". Here are three technologies I do not strongly associate with correctness: * JSON * Schemaless databases * Scripting languages Rethink's own FAQ says "RethinkDB is not a good choice if you need full ACID support or strong schema enforcement". So in the database market there are of course lots of users who do strongly value correctness, but they tend to use databases and technologies that are statically typed (Java/C#) and database engines that have are ACID and provide schemas. Very few of them want a database that doesn't provide ACID, doesn't provide schema enforcement and which appears to either not support standard industrial languages at all, or is at least distinctly uninterested in them. In other words Rethink targeted the hipster startup market of companies building social networks, ride-hailing apps and other things where correctness is not important and low-correctness tech dominates, but tried to sell a product that was oriented around correctness. This seems like a market/product mismatch.

lobster_johnson 10y ago
That's a good point. Competing with MongoDB is difficult because "correctness" alone is not a selling point. It's the "worse is better" phenomenon as a marketing challenge. That said, I don't think lack of correctness itself is a design goal. I think it's accidental, or at least an intermediate state before we have something better. Case in point: I'm currently working on an open-source (it will be released soon) data store that uses JSON to express "documents" of data, but also has schemas, joins and transactions. It uses PostgreSQL internally, but completely hides the underlying storage. The developers who are currently using it for actual work are pleasantly surprised about how nice it is to work with: You get to work with rich, structured data in its natural shape without needing some kind of ORM to mediate between an app's data model and the database's, and at the same time it's very strictly validated without being pushy (it has gradual typing: you can start out schemaless and then tighten the schema). When we designed it, we wanted to build something that reflected how people actually use a database, and I think we're achieving that. (I'm looking forward to make it public and share it on HN.) So it's certainly possible to have the best of both worlds. It's just that the direction that the world moved in with "NoSQL" temporarily moved us off course for a little while. It started out as a natural reaction to scaling issues, and in that sense I think it might have been a necessary step while people figured out what they were doing. We're seeing the same evolution in the ways of people's thinking with regard to dynamically/statically typed languages, too.

lobster_johnson 10y ago

That's a good point. Competing with MongoDB is difficult because "correctness" alone is not a selling point. It's the "worse is better" phenomenon as a marketing challenge. That said, I don't think lack of correctness itself is a design goal. I think it's accidental, or at least an intermediate state before we have something better. Case in point: I'm currently working on an open-source (it will be released soon) data store that uses JSON to express "documents" of data, but also has schemas, joins and transactions. It uses PostgreSQL internally, but completely hides the underlying storage. The developers who are currently using it for actual work are pleasantly surprised about how nice it is to work with: You get to work with rich, structured data in its natural shape without needing some kind of ORM to mediate between an app's data model and the database's, and at the same time it's very strictly validated without being pushy (it has gradual typing: you can start out schemaless and then tighten the schema). When we designed it, we wanted to build something that reflected how people actually use a database, and I think we're achieving that. (I'm looking forward to make it public and share it on HN.) So it's certainly possible to have the best of both worlds. It's just that the direction that the world moved in with "NoSQL" temporarily moved us off course for a little while. It started out as a natural reaction to scaling issues, and in that sense I think it might have been a necessary step while people figured out what they were doing. We're seeing the same evolution in the ways of people's thinking with regard to dynamically/statically typed languages, too.