3 ms·
I have my own definition of big data or maybe this is the next "after big data" and that is "streaming data". Data being generated non stop at a high enough ra
by rdtsc 14y ago
I have my own definition of big data or maybe this is the next "after big data" and that is "streaming data".
Data being generated non stop at a high enough rate that it doesn't make sense to store it. You can only analyse, extra relevant statistics or some features and move on.
Storing it is just putting in a huge buffer and as new data comes in the old data falls of the end.
In some situations where products are up 24/7 in multiple time zones, there is no time for offline batch processing. By the time the batch has finished there is newer possibly bigger batch and so on.