3 ms·

It's about using the right tool for the job. It seems most of the companies you mentioned have decided that relational databases are actually the right tool.

It's about using the right tool for the job. It seems most of the companies you mentioned have decided that relational databases are actually the right tool. You mentioned... Google: Proprietary stack for document-orientated systems (search, GMail), use MySQL to run AdWords (their revenue business). http://xooglers.blogspot.com/2005/12/lets-get-real-database.html http://xooglers.blogspot.com/2005/12/lets-get-real-database.... Amazon: Major Oracle customer. Functionality is managed by modular teams, each of which has their own architectural choices, so difficult to generalize. Certainly in the early days was entirely Oracle (and look, it's a story about Oracle failing!) http://glinden.blogspot.com/2006/03/early-amazon-oracle-down.html http://glinden.blogspot.com/2006/03/early-amazon-oracle-down... Want you to use SimpleDB, but difficult to find Amazon groups actually eating the dog-'food'. eBay: Oracle. http://www.addsimplicity.com/downloads/eBaySDForum2006-11-29.pdf http://www.addsimplicity.com/downloads/eBaySDForum2006-11-29... LinkedIn: Oracle & MySQL. Custom caching layer on top to get round the 'graph' problems you talked about. http://hurvitz.org/blog/2008/06/linkedin-architecture http://hurvitz.org/blog/2008/06/linkedin-architecture OK, what about some others: Facebook: MySQL. Memcached for caching. Custom data stores for images and for inbox search (so non-relational for document storage again) http://glinden.blogspot.com/2008/05/scaling-facebooks-databases.html http://glinden.blogspot.com/2008/05/scaling-facebooks-databa... Wikipedia: MySQL. They're document storage, so it is possible to get document storage to work on MySQL. http://venublog.com/2008/04/16/notes-from-scaling-mysql-up-or-out/ http://venublog.com/2008/04/16/notes-from-scaling-mysql-up-o... Maybe it's just not sexy to stand up at a conference and say 'We architected this system to run on a relational database, in the normal proven way, and it runs wonderfully in production. We scaled it using partitioning, in the normal way. We're generating real money with the system.' You only hear a biased sample, which is why you think the companies you named are good examples of people that use non-relational databases.

wheels 18y ago
We seem to be oscillating between "key value stores are the way!" and "relational databases are the way!" Both have their places. If my notes were read as seeming to advocate the end of relational databases, please excuse me because that was certainly not my intent. Of course all of those companies use relational databases. Often they're the best tools for the job and in an IT landscape as large as their respective setups, of course they're the right tool for several jobs. But they've also got problem domains where they've decided RDBMSes are not the best tools for the job, which was my point to begin with. This thread started off with the assertion, basically, that if you weren't using a RDBMS to store everything, you probably didn't know anything about databases and I feel like we've (including the original poster) been slowly narrowing in on something closer to reality. As for the examples on the non-RDBMS side, BigTable, SimpleDB, Voldemort (+) and, well, the guy that I met with last week that wrote eBay Germany's search system and told me they had their own non-RDBMS storage system. + http://blog.linkedin.com/2009/03/20/project-voldemort-scaling-simple-storage-at-linkedin/ http://blog.linkedin.com/2009/03/20/project-voldemort-scalin...
- justinsb 18y ago
  Let's take the Rails approach: you don't need a non-relational database. If you're smart enough to know why this isn't true in your case, and you need to step outside the recommended framework, great. Your application is in the unusual 0.1% (but are you sure you're not in the 1% of the 99.9% that's just made a mistake?) The problem is, all the big names like to name-drop interesting projects, and people assume that Amazon is built on SimpleDB, or Google is built on BigTable, or LinkedIn is built on Voldemort, or Facebook is built on Cassandra, when in reality none of them really are. Odds are, you're not an Amazon, Google, LinkedIn, Facebook, and the odds are even more against you being in that sliver of functionality where they're using this cool tech.
  - wheels 18y ago
    I feel like you're arguing with a straw-man here. I've not said that most sites should avoid relational databases. I've said that there exist applications where RDBMSes are less than ideal and graph / key-value systems seem to be a better fit. We seem to agree there. We can argue over how large that slice is, but since we'd both be making numbers up, it seems rather pointless. Your business seems to be invested in pushing out commodity RDBMSes. Mine is focused on applications that need super fast graph traversal. It's no shocker that we have a different take on this. :-)
    - justinsb 18y ago
      Just read your profile & followed the link to your company. I like the product you guys are building, and certainly understand where you're coming from now. I just take exception to the name-dropping of big-companies as examples of companies that have decided against relational databases, when they've only done so in incredibly narrow niches of very large systems. It would be more accurate to say they've decided against traditional filesystems, yet we don't see articles proclaiming the death of the filesystem. This is the real straw man argument, I'd suggest. I'm sure there are applications out there for whom graph databases are ideal; the only way to get real numbers is to try to bring a product to market, so I wish you well!

wheels 18y ago

We seem to be oscillating between "key value stores are the way!" and "relational databases are the way!" Both have their places. If my notes were read as seeming to advocate the end of relational databases, please excuse me because that was certainly not my intent. Of course all of those companies use relational databases. Often they're the best tools for the job and in an IT landscape as large as their respective setups, of course they're the right tool for several jobs. But they've also got problem domains where they've decided RDBMSes are not the best tools for the job, which was my point to begin with. This thread started off with the assertion, basically, that if you weren't using a RDBMS to store everything, you probably didn't know anything about databases and I feel like we've (including the original poster) been slowly narrowing in on something closer to reality. As for the examples on the non-RDBMS side, BigTable, SimpleDB, Voldemort (+) and, well, the guy that I met with last week that wrote eBay Germany's search system and told me they had their own non-RDBMS storage system. + http://blog.linkedin.com/2009/03/20/project-voldemort-scaling-simple-storage-at-linkedin/ http://blog.linkedin.com/2009/03/20/project-voldemort-scalin...

justinsb 18y ago
Let's take the Rails approach: you don't need a non-relational database. If you're smart enough to know why this isn't true in your case, and you need to step outside the recommended framework, great. Your application is in the unusual 0.1% (but are you sure you're not in the 1% of the 99.9% that's just made a mistake?) The problem is, all the big names like to name-drop interesting projects, and people assume that Amazon is built on SimpleDB, or Google is built on BigTable, or LinkedIn is built on Voldemort, or Facebook is built on Cassandra, when in reality none of them really are. Odds are, you're not an Amazon, Google, LinkedIn, Facebook, and the odds are even more against you being in that sliver of functionality where they're using this cool tech.
- wheels 18y ago
  I feel like you're arguing with a straw-man here. I've not said that most sites should avoid relational databases. I've said that there exist applications where RDBMSes are less than ideal and graph / key-value systems seem to be a better fit. We seem to agree there. We can argue over how large that slice is, but since we'd both be making numbers up, it seems rather pointless. Your business seems to be invested in pushing out commodity RDBMSes. Mine is focused on applications that need super fast graph traversal. It's no shocker that we have a different take on this. :-)
  - justinsb 18y ago
    Just read your profile & followed the link to your company. I like the product you guys are building, and certainly understand where you're coming from now. I just take exception to the name-dropping of big-companies as examples of companies that have decided against relational databases, when they've only done so in incredibly narrow niches of very large systems. It would be more accurate to say they've decided against traditional filesystems, yet we don't see articles proclaiming the death of the filesystem. This is the real straw man argument, I'd suggest. I'm sure there are applications out there for whom graph databases are ideal; the only way to get real numbers is to try to bring a product to market, so I wish you well!