4 ms·
I've been playing around with the Aggregation Framework lately (using the release candidate). The performance seems to be pretty reasonable, especially when co
by _jmar777 14y ago
I've been playing around with the Aggregation Framework lately (using the release candidate). The performance seems to be pretty reasonable, especially when compared to similar tasks with the old MR framework. A quick and dirty benchmark number in case anyone is interested:
* Obligatory unscientific, probably not meaningful, etc. disclaimer.
Mongo Version: 2.2.0-rc1
Hardware: MBP, Snow Leapord, 2.2 GHz Intel Core i7, 8 GB mem
Data: Single collection with 500k records (machine generated time-series event data)
Query Pipeline:
[
{
$match: { ts: { $gte: 1293858000000, $lt: 1296536400000 } }
},
{
$group: {
_id: 'aggregations',
sum: { $sum: '$foo' },
num: { $sum: 1 },
avg: { $avg: '$bar' }
}
}
]
Results: The time range matched against above matches 42,466 documents within the collection. The average response time over 50 runs is 419ms. Not exactly "Big Data OLAP" stuff just yet, but plenty fast enough for most use cases involving reasonably small sets of data. Great job to the MongoDB team!
- mathias_10gen 14y agoOut of curiosity, did you have an index on the ts field that you are matching with? By the way, if you (or anyone else for that matter) come up with useful benchmarks I'd love to get a copy of them at mathias@10gen.com. I have a few of my own, but I'd like to get some real-world workloads from the community to test potential optimizations against.
- _jmar777 14y agoYes, I had a single index: db.events.ensureIndex({ ts: 1 }); I'll try to clean up my benchmark code a little, throw it in a gist, and then I'll send it your way.
- _jmar777 14y agoUpdate: already forwarded this to Mathias, but posting here in case anyone else wanted to see the full benchmark test: https://gist.github.com/3518344 https://gist.github.com/3518344