I look at it this way: you need your 'collection' side of things to be as lightweight as possible. I would 1) collect request data in memory 2) 'post-process' a

I look at it this way: you need your 'collection' side of things to be as lightweight as possible. I would 1) collect request data in memory 2) 'post-process' and move that data periodically into your db (once a minute, once a second, whatever) - allowing you to basket that data however you want and 3) move that pre-processed data back into memory to add into your responses as needed. That way the most expensive work would get done in a dignified, scheduled manner. I don't know anything about MongoDB but if it's not MySQL that's a good start. Disclaimer: I don't know what I'm talking about.