4 ms·
Some informative links, in case you haven't seen them already: * http://www.quora.com/What-caused-Foursquares-downtime-on-October-4-2010 http://www.quora.com/W
by cubes 16y ago
Some informative links, in case you haven't seen them already:
* http://www.quora.com/What-caused-Foursquares-downtime-on-October-4-2010 http://www.quora.com/What-caused-Foursquares-downtime-on-Oct...
* http://blog.foursquare.com/2010/10/05/so-that-was-a-bummer/ http://blog.foursquare.com/2010/10/05/so-that-was-a-bummer/
* https://groups.google.com/group/mongodb-user/browse_thread/thread/66752f49af68619?pli=1 https://groups.google.com/group/mongodb-user/browse_thread/t...
Someone on this thread suggested that Foursquare's performance problem is related to the calculation of badges. I'm not sure where this idea came from. I haven't seen any mention of the issue being related to badges in first or secondhand sources.
That said, Mongo DB's map/reduce operation would not be a reasonable solution at this time. Mongo DB's map/reduce performance is, at present, somewhat lacking because it runs via the javascript engine which is currently single threaded. I know there are plans to improve performance of Mongo DB's javascript engine by switching to V8, but I don't know if V8 is multithreaded.
Often design decisions that look bad in hindsight get baked in early. By the time you realize that an alternate design would yield better performance, there may be too much data to migrate so you just have to live with it.
- megaman821 16y agoI am not sure if badge calculation is the bottleneck. Foursquare did say the calculation is why they have to keep everything in memory though. I just assumed that they sharded by user to keep all of the user's check-ins in the user document and that they are doing calculation with that set of data.
- whakojacko 16y agoharryh mentioned it here: http://news.ycombinator.com/item?id=1769909 http://news.ycombinator.com/item?id=1769909 Computing the badges online (ie at each checkin time) requires having the whole db in ram for acceptable performance.