4 ms·
I mostly agree with what you're saying, but I also think there's enormous value in making easy things be really easy. Even with today's state of the art adding
by coffeemug 14y ago
I mostly agree with what you're saying, but I also think there's enormous value in making easy things be really easy. Even with today's state of the art adding a shard, dealing with consistency issues, adding replicas, etc. is relatively hard. Perhaps not in a computer-sciency sense (all the problems are fairly well understood), but in an operational sense. Lots and lots of work needs to be done even with systems like MongoDB, let alone with MySQL. And once you're done with that, you can't really run complicated queries easily, so you have to solve that problem.
We set out to make these things be really easy (whether we succeed or not remains to be seen). We want the users not to have to deal with these issues at all whenever possible. You should be able to set up a cluster, add shards, and run cross-shard joins and aggregation in five minutes.
Of course once that problem is solved, there are tougher problems like high-performance cross-document distributed ACID, but I think the industry as a whole is relatively far away from that right now. (there are some solutions to this - e.g. Clustrix, but they require specialized hardware which makes it out of reach for most developers)
- nlavezzo 14y agoCongratulations on shipping - looks like a very well thought out product. Regarding your last comment on high-performance distributed ACID, that's what we've built at FoundationDB, although FoundationDB is a key value store so transactions are multi/cross-key instead of cross-document.
- Nitramp 14y agothere are tougher problems like high-performance cross-document distributed ACID, but I think the industry as a whole is relatively far away from that right now Megastore and Spanner solve that problem, with varying tradeoffs: http://research.google.com/pubs/pub36971.html http://research.google.com/pubs/pub36971.html http://research.google.com/archive/spanner.html http://research.google.com/archive/spanner.html
- erichocean 14y agoOur internal database does too (with a different design than Spanner, but stuff still comes "online" atomically for everyone across the globe at the same time, with similar latency). Unlike FoundationDB, and like Spanner, we're doing it with complex object graphs, not just key-values, and we also do it with consistent secondary indexing (I'm not sure if Spanner supports this or not). This isn't "the future", this is now. People are doing it, and have been for awhile. If you're going to "rethink the database", distributed global consistency should be at the top of your list today. RethinkDB seems like its merely "rethinking" Mongo. The main benefit of global consistency, of course, is ease of use. Global consistency is so much easier to reason about and write code for!
- nlavezzo 14y agoHi Erich - I'm curious... you refer to your "internal database" doing distributed ACID but state that you're not with Google. Can you say who you're with? It's interesting to us to know who is also working on this problem.