4 ms·
The key point is that you've done all your processing in batch, and you're only displaying the most prominent or interesting properties and links for each recor
by rjurney 16y ago
The key point is that you've done all your processing in batch, and you're only displaying the most prominent or interesting properties and links for each record. Hadoop/Pig/Python together are much more powerful than SQL if you don't have a real-time requirement, and by the time we get to the key/value store all the data processing is done. If you think you have a real-time analytic requirement... well you may, but quite possibly you really don't.
Getting to that point through batch processing can be hard, but the infrastructure is ideally suited to it. NoSQL doesn't impede you whatsoever. It enables you to think correctly about packaging your data for recursive consumption in trivial interfaces.
Real-time large scale graph processing isn't possible, or is very hard but... its not really needed to do amazing things.