3 ms·
This is very impressive, but at some point it becomes more cost effective to rack up more reasonably sized commodity hardware with a "big data" database, no? (
by jimbokun 9y ago
This is very impressive, but at some point it becomes more cost effective to rack up more reasonably sized commodity hardware with a "big data" database, no?
(Using "big data" to mean sacrificing low latency joins and full transaction support, with the caveats that many "big data" systems are making progress at getting those things to work.)
Of course you can shard, but there is a significant maintenance cost to that as well, and requires your data model can be easily sharded.
- falcolas 9y agoUnder 1 TB is still commodity hardware with many of these DBMS. 5-10 TB is commodity hardware plus a few extra drives. Sharding, or splitting out into "big data" databases has its own cost, a cost which is frequently greater than the $20k worth of hardware.
- jimbokun 9y agoAn example in the article was over 37TB, and another was 205TB containing "hundreds of thousands of uncompressed InnoDB tables". Maybe that's a system that could have been better served by something like Cassandra? (Depending, of course, on access patterns, complexity of queries, etc. etc.)