Exactly. The (stealthy) thing we're doing doesn't have much data at all since we're not doing anything with images or video. Data for 1M users would be ~40GB,

Exactly. The (stealthy) thing we're doing doesn't have much data at all since we're not doing anything with images or video. Data for 1M users would be ~40GB, of which only 8GB of that would be relevant at any single point in time. Well within the realms of in memory caching. DB writes (our writes are batchy based on external events that we're not in control of) would be our biggest bottleneck but we offset them with MEMORY based mysql tables to hold the 'in progress' data and then update the disk based tables for all users in off-peak times. We can speed up DB writes with more spindles, and then by an order of magnitude by moving to SSD(s). A single DB server at ~$5000 to handle the data for 10M users and, if necessary, a number of mysql slave servers. Subsequent scaling is just sharding. Compared to the serving that data via HTTP[s] the DB side of our application is easy.