3 ms·
Use TokuMX (I don't have a stake in TokuMX, I'm just a happy customer). It's mostly a drop-in for Mongo 2.6 (though converting data is a downtime operation). To
by cheald 12y ago
Use TokuMX (I don't have a stake in TokuMX, I'm just a happy customer). It's mostly a drop-in for Mongo 2.6 (though converting data is a downtime operation). To your point specifically, we're running aggregation stuff on collections with >1 billion documents and it hums along great. Massively better disk usage, clustering keys, ACID guarantees, document-level locking, tunable compression, and a bunch of other small improvements make it a really excellent choice.
TokuMX does have one significant problem in that the technology they use for indexes experiences performance degradation over time if you do lots of deletes from the head of the collection (ie, if you're processing data that gets inserted and deleted roughly chronologically), but it is fixable with reIndex() (which is an async operation analogous to OPTIMIZE TABLE). That's really the only large problem we've had with it, and it's surmountable with regular maintenance. In the case of monotonically-ordered data though, you can address the problem with partitioned collections, which lets you split a collection up into chunks, and then drop chunks as they become unnecessary, without having to do expensive things to the whole collection.
It looks like WiredTiger should provide many of the same benefits, so you may evaluate that first before taking the plunge, but I haven't personally compared it with our tokumx setup yet.
Outside of that, you probably need to start sharding your dataset. Figure out a shard key that will let you distribute your data well, and start splitting your load among multiple replica sets. Figure out if your issues are IO, CPU, or whatever - there's a lot of documentation on how to do this, so I won't rehash it here, but in general, you first need to understand what is slow before you can fix it. Chances are very good that your culprit is disk IO, though (because it always is in databases), and you can probably get some of that performance back with more RAM (to keep more of your working set in RAM at a given point), faster disks, and/or a striped disk array. Depending on your platform, some kernel tuning may help, as well, though it's going to be specific to your platform and setup.
You might also look at other tools and see if they're better-suited for your purposes. We use Elasticsearch as our ad hoc query layer on top of TokuMX, and are doing some stuff with Cassandra and BigQuery for bulk analysis of historical data. I'm not suggesting that you replace your whole stack, but if you're trying to shove a square peg through a round hole, you're just going to be frustrated. Scaling is all about specializing your toolset to fit your problem domain - there is almost never a single solution that is the best thing for all use cases. If you need to add another tool to the stack to offload one part of your requirements, do it!