4 ms·
There have been conversations in my job where I refer to a 40GB file as "big" and someone else will go "that's nothing." Well, obviously there are bigger files
by mfisher87 12y ago
There have been conversations in my job where I refer to a 40GB file as "big" and someone else will go "that's nothing." Well, obviously there are bigger files somewhere, but that doesn't mean 40GB is nothing.
"Big" is always a relative term. Anything is big in the right perspective, so to say that something "isn't big" is almost always wrong. We should really try to find a term with a more absolute meaning.
- andybak 12y agoI think a rough definition might be something like "datasets big enough to cause problems with standard tools on standard hardware" - i.e. where you need to change your tools to get things done. Still pretty vague but it captures what I think many people are getting at.
- blumkvist 12y agoBig data does not refer only to the size of the dataset. It's variance and velocity, in addition to volume.
- jghn 12y agoI was going to say something similar except for me it's a function of size and multidimensionality. We're sitting on several PB but it's generally in two dimensions - standard analysis techniques work, we just have problem moving the data around efficiently and processing it. I like your inclusion of velocity, I'm including that in my definition.