Absolutely this. If you're in the AWS ecosystem then move away from the bare-ish metal and onto the tools they provide for doing this stuff. MongoDB in particul

Absolutely this. If you're in the AWS ecosystem then move away from the bare-ish metal and onto the tools they provide for doing this stuff. MongoDB in particular really doesn't seem to be relevant to the problem at all. Throw all the data into Kinesis, and then push the bits you need into Redshift the other end. If you want to keep a lot of data for ad-hoc queries but only use a subset for regular queries then consider setting up separate dense compute and dense storage Redshift clusters as appropriate. As far as pricing goes there's a hell of a discount for going reserved once you've got things set up the way you want them. Once you've got that running and the immediate fires are out you may want to look at Spark to pre-compute some of the reports (you can still push the computed data into Redshift tables to save changing the reporting too much), although I don't think spark as a service isn't really there in AWS right now, and running up an EMR cluster is seriously going to cost you. Bigtable provides similar capabilities to Redshift in the google ecosystem, and looks okay although I wasn't blown away by it.