Yes, it would be a challenge to make it work, and work fast! Lots of numbers to crunch! But it's definitely possible, you could e.g. use idle threads to collate

Yes, it would be a challenge to make it work, and work fast! Lots of numbers to crunch! But it's definitely possible, you could e.g. use idle threads to collate data in the background while not handling requests, and serve results leveraging the collated database. The simplest way would probably be to cluster the users according to their preferences and serve your results according to which cluster a user belongs to. More complex mechanisms, like detailed preference correlation graphs might be possible, but would consume huge amounts of storage. But I really fancy this idea, and I do seriously believe it to be one of the next big breakthroughs which are going to happen. Unfortunately I do currently not have the time to handle such a big assignment, so I decided to throw the idea out there, free for the grabs. I want to have that thing, and I don't care much who's gonna make the big bucks from it! But try to convince anyone that an openly touted idea is actually worth something. People still seem believe that if you're not secretive about your ideas, they probably suck... :-(