4 ms·
Yeah like others point out this is small data. My rules are: 1) Does it fit in a desktop computer memory? If it does it is "small data". 2) Does it fit on a
by rdtsc 12y ago
Yeah like others point out this is small data. My rules are:
1) Does it fit in a desktop computer memory? If it does it is "small data".
2) Does it fit on a hard drive in a desktop? Then it is just "data". Or medium data.
3) If you need a cluster or some centralized network storage to fit and manage it, you might have big data.
4) Next level up is streaming data. Your data doesn't even sit anywhere because it is accumulated faster than you could ever processes it.
- texthompson 12y agoMy usual rule of thumb is "does AWS sell a single instance big enough to store and process data in real time?" If you can't find an instance big enough, you have big data.
- semi-extrinsic 12y agoThe best definition I've seen of Big Data is "the database indices don't fit in memory on the beefiest single server you have access to". (And this doesn't mean you can claim big data because you only have access to your 4 GB RAM laptop.)