3 ms·
IMHO when discussing "Big Data" you have to go up PB (in total) and some large numbers (in daily) to reach "Big Data". If the data can be stored inside a commer
by hnthrowaway0315 5y ago
IMHO when discussing "Big Data" you have to go up PB (in total) and some large numbers (in daily) to reach "Big Data". If the data can be stored inside a commercial Xeon box then it's not even "Medium Data".
- dylan604 5y agoto me, Big Data is more of a concept than just a description of the amount of data. So ontop of the storage, it's also the analysis and usage of that data. So Big Data services could be just as useful to a small company with a mere 100GB of data once they learn how to squeeze the juice from the berries they've collected.
- funcDropShadow 5y agoBut, if it is 100 GB of data, Big Data tools are the wrong tools, because they are inefficient at that size. Most Big Data come with lot's of restricitions and complications which are fine if they are the only alternative. But don't ever think about using Cassandra instead of Postgres if you have less than 10 TiB. Even over 10 TiB it is not clear when to switch to something more scalable.
- dylan604 5y agoAgain, this comment is focused on the size of the data on storage and how much data there is. To me, this is just a single aspect of "Big Data". To me, "Big Data" is also what information can be gleaned from that data that is being stored whether it be Postgres, NoSQL, Cassandra, etc. That part of it is only relevant in conversations about "how much data" is available to the "Big Data" processes. People paying for "Big Data" are excited about "how much data" they have other than a bragging point. What they are paying for is the information that can be garned for having that data.