At Fashiolista we went through a few phases. A.) In the early days we simply pulled everything together by querying the database which worked quite well. B.)

At Fashiolista we went through a few phases. A.) In the early days we simply pulled everything together by querying the database which worked quite well. B.) After that we switched to using redis and in a similar approach to twitter's pushed only to active users and pulled the feed for inactive users. C.) Now our requirements slowly changed and it became harder to fit everything in memory (or fallback) that's why we switched to Cassandra. Feedly offers you a framework, but allows you to make your own decisions regarding the tradeoffs. - you can change the fanout to hit only active users (twitter, yahoo paper approach) - you can chose to store the full activity in the feed or only an id (memory usage vs extra lookups) - you can customize the priority of tasks These are all things we've done at one point or another. You can chose your own approach with Feedly. For Fashiolista we currently use celery with rabbitmq and have different queues and worker clusters for low and high priority fanout tasks. In your example with Lady Gaga memory usage would go up on the rabbitmq server. Since rabbit also stores to disk it won't run out easily. After that the celery machines will handle the updates and autoscaling will kick in more machines if needed.