4 ms·
CERN’s use of MongoDB for Large Hadron Collider data
- ihodes 16y agoI'm curious to know why they chose to use MongoDB? Is it because its free & fast? Or is it truly faster than a propriety SQL system? Either way, it's nice to see that computation in science isn't stuck in the 90's, which is sometimes how it feels.
- ergo98 16y agoThey use it as a trivial caching system of pre-aggregated data.
- brown9-2 16y agoCorrect me if I'm wrong but couldn't any number of other caching solutions (such as memcached) be used in this situation? In other words, are there any features specific to MongoDB that they are actually making use of?
- ergo98 16y agoThey don't give much details, but it sounds like a very superficial need that could be accommodated by Redis, Memcached, AppFabric, and on and on. Kind of surprised that they chose MongaDB as it's far from the fastest competitor.
- dschn 16y agoJust speculation, but the built-in map/reduce is probably quite useful.
- ergo98 16y agoPerhaps, but the description essentially said that there is a custom python layer that aggregates the data, and they punt it into MongoDB with some SQL extraction pattern simply to store it into that silo. e.g. If I run a query for NAME LIKE 'BLAH%' it stores that resultset. If I run NAME LIKE 'BLAP%' it stores that resultset. If someone else comes and runs either of them, only a direct match will pull my prior results. It's okay, but it's no big win or case study.
- ehwizard 16y agoThey needed to be able to do some queries on various indexes, so key/value stores wouldn't work. Also MongoDB tends to be only 10% slower then memcache, so not sure why you think its not fast...
- kristina 16y agoThey go into it in the actual paper, if you're interested. They wanted a document store because their data is flexible (all different sizes and shapes), complex, and they needed secondary indexes. They didn't need transactions and MongoDB was very fast.