4 ms·
Yes, sure - the "big data" issues are essentially about data above a certain point where very different techniques for processing it (e.g. cluster computing and
by PeterisP 6y ago
Yes, sure - the "big data" issues are essentially about data above a certain point where very different techniques for processing it (e.g. cluster computing and algorithms to split and merge partial results of computing that's not trivially parallelizable) start to make sense or even become required. If technology progress grants a huge increase in available memory and computing power and algorithm efficiency, then that boundary moves upwards.
There are early "big data" publications on datasets that decidedly shouldn't be treated as "big data" today because now it's entirely appropriate to process them with simple methods within the RAM on a decent server or in some cases even on my laptop.