1 terabyte is definitely small data. There's only one company consistently doing big data and that's, of course, the goog. I like the dynamic definition that bi

1 terabyte is definitely small data. There's only one company consistently doing big data and that's, of course, the goog. I like the dynamic definition that big data is only what is not easily handled by existing tools. The tools of the past few years (hive, cascalog, elastic search, etc) and some of the emerging ones (shark, impala, Druid, etc) have radically raised the bar for what problems smaller shops can handle. We're handling a very modest small data set (1,000 messages/sec) with just a few people. That's thanks to the awesome tools coming out of this fruitful (and overhyped) industry.