3 ms·

MongoDB Aggregation I: Counting and Grouping

22 points by _pius 17y ago

stingraycharles 17y ago
This is pretty interesting: looks like they're laying down the foundations of their Map/Reduce framework with this -- this approach to their Group function is one of the most flexible I've seen so far! In essence, they have implemented their Group function as a real simple Map/Reduce job, in which the "Map" phase is skipped partially Instead of completely decoupling the Map and Reduce phase, a Reduce is performed right after each object is Mapped: you perform a Reduce directly on the source object. Regular Map/Reduce frameworks usually decouple the Map and Reduce phase completely: you perform a Map on the source object, and a Reduce on all the emitted Map objects. This of course brings a lot of overhead (lots of Map objects emitted with just the value "1").
- JeffJenkins 17y ago
  Mongo actually supports map/reduce now: http://www.mongodb.org/display/DOCS/MapReduce http://www.mongodb.org/display/DOCS/MapReduce I was a bit disappointed that you couldn't set up a function to update "map" objects when the collection it was generated from is updated or added to. That was my favourite feature from CouchDB
  - stingraycharles 17y ago
    Ah, I should've looked better -- I didn't realize they actually already supported Map/Reduce by now -- good news! I was a bit disappointed that you couldn't set up a function to update "map" objects when the collection it was generated from is updated or added to. That sounds interesting, but it makes me wonder: what happens when an updated object is mapped again, as you describe? An update means that a previously mapped entry is deleted, so you should unreduce (inflate?) the previous map operation (which in many cases is not possible). How are these things handled in Couch ?
    - njl 17y ago
      In CouchDB, Map/Reduce is the primary way that you access the database. The first time you access a "view", it will do a Map/Reduce on the entire database. On subsequent access, it will redo the Map on new records. To make this work, the action of the Map function needs to be immutable with respect to the record; this is intellectually pleasing but incredibly annoying in practice. I haven't really used the Map/Reduce functionality of Mongo yet, but I have poked at it a little. Basically, with MongoDB, you do a one-time transform from one "collection" of records to another. It doesn't automatically update the way CouchDB does, but I can do things like joins that I couldn't do with CouchDB. Once again, Mongo's practicality makes me happy, while Couch gives me reason to admire but avoid.

22 points by _pius 17y ago

This is pretty interesting: looks like they're laying down the foundations of their Map/Reduce framework with this -- this approach to their Group function is one of the most flexible I've seen so far! In essence, they have implemented their Group function as a real simple Map/Reduce job, in which the "Map" phase is skipped partially Instead of completely decoupling the Map and Reduce phase, a Reduce is performed right after each object is Mapped: you perform a Reduce directly on the source object. Regular Map/Reduce frameworks usually decouple the Map and Reduce phase completely: you perform a Map on the source object, and a Reduce on all the emitted Map objects. This of course brings a lot of overhead (lots of Map objects emitted with just the value "1").

JeffJenkins 17y ago
Mongo actually supports map/reduce now: http://www.mongodb.org/display/DOCS/MapReduce http://www.mongodb.org/display/DOCS/MapReduce I was a bit disappointed that you couldn't set up a function to update "map" objects when the collection it was generated from is updated or added to. That was my favourite feature from CouchDB
- stingraycharles 17y ago
  Ah, I should've looked better -- I didn't realize they actually already supported Map/Reduce by now -- good news! I was a bit disappointed that you couldn't set up a function to update "map" objects when the collection it was generated from is updated or added to. That sounds interesting, but it makes me wonder: what happens when an updated object is mapped again, as you describe? An update means that a previously mapped entry is deleted, so you should unreduce (inflate?) the previous map operation (which in many cases is not possible). How are these things handled in Couch ?
  - njl 17y ago
    In CouchDB, Map/Reduce is the primary way that you access the database. The first time you access a "view", it will do a Map/Reduce on the entire database. On subsequent access, it will redo the Map on new records. To make this work, the action of the Map function needs to be immutable with respect to the record; this is intellectually pleasing but incredibly annoying in practice. I haven't really used the Map/Reduce functionality of Mongo yet, but I have poked at it a little. Basically, with MongoDB, you do a one-time transform from one "collection" of records to another. It doesn't automatically update the way CouchDB does, but I can do things like joins that I couldn't do with CouchDB. Once again, Mongo's practicality makes me happy, while Couch gives me reason to admire but avoid.