4 ms·

How Big Data Became So Big

52 points by sew 14y ago

edhallen 14y ago
Good look at the history of the marketing term big data. I think this fits squarely in the realm of big data hype, or at least a review of why it is big data hype (something I've blogged about here http://www.klaviyo.com/blog/2012/07/16/the-curse-analytics-big-data-hype/ http://www.klaviyo.com/blog/2012/07/16/the-curse-analytics-b...). The SAS discussion makes it especially clear that it's viewed as a trend. But - posts like these frequently jump to the top of hacker news, and there's clearly a feeling that access to new analyses and more data is going to change the world. I think this is probably right. I'd love to hear more discussion of how big data has already started this process. Examples I can think of (and I'd love to hear more): - weather analysis for farmers (weatherbill / the climate corporation) - marketing (the target pregnancy example - http://www.nytimes.com/2012/02/19/magazine/shopping-habits.html?pagewanted=all http://www.nytimes.com/2012/02/19/magazine/shopping-habits.h...) More examples?
- asanwal 14y ago
  CB Insights (http://www.cbinsights.com/mosaic/ http://www.cbinsights.com/mosaic/) is assessing the health of private companies using public data. B2B sales teams, supply chain/procurement professionals and investors use CBI's private company analytics to sell more, purchase smarter and invest more intelligently. disclosure: I'm the founder so I'm very biased. We're hiring btw.
crayola 14y ago
The buzz around big data, let's not forget, is partly fuelled by bullshit generated by vendors of big data solutions and analytic firms. It probably does not really need to be repeated, but I guess it cannot hurt. Data being big rarely causes it to be much more interesting. There can be much more actionable information in a few hundred records than in billions, depending on context, proper experimental design, and so on. Also, an often overlooked point is that precision of statistical estimators (e.g. width of confidence intervals) typically scales with the square root of sample size; yet the cost of processing an additional observation is typically constant (e.g. with distributed systems like hadoop). So in many cases there are decreasing returns to data size.
Tichy 14y ago
I'm already getting allergic to the term. Every couple of months there is a new term for the same thing to fuel a new hype cycle (machine learning, computational intelligence, data mining,...). Not that it isn't an important thing, and certainly more and more data is becoming available. Now lets get back to work.
drucken 14y ago
Painful reading an HN front page article like this. An entire article around use of two-word (marketing) nomenclature with no informative value in the article at all...

52 points by sew 14y ago

Good look at the history of the marketing term big data. I think this fits squarely in the realm of big data hype, or at least a review of why it is big data hype (something I've blogged about here http://www.klaviyo.com/blog/2012/07/16/the-curse-analytics-big-data-hype/ http://www.klaviyo.com/blog/2012/07/16/the-curse-analytics-b...). The SAS discussion makes it especially clear that it's viewed as a trend. But - posts like these frequently jump to the top of hacker news, and there's clearly a feeling that access to new analyses and more data is going to change the world. I think this is probably right. I'd love to hear more discussion of how big data has already started this process. Examples I can think of (and I'd love to hear more): - weather analysis for farmers (weatherbill / the climate corporation) - marketing (the target pregnancy example - http://www.nytimes.com/2012/02/19/magazine/shopping-habits.html?pagewanted=all http://www.nytimes.com/2012/02/19/magazine/shopping-habits.h...) More examples?

asanwal 14y ago
CB Insights (http://www.cbinsights.com/mosaic/ http://www.cbinsights.com/mosaic/) is assessing the health of private companies using public data. B2B sales teams, supply chain/procurement professionals and investors use CBI's private company analytics to sell more, purchase smarter and invest more intelligently. disclosure: I'm the founder so I'm very biased. We're hiring btw.

crayola 14y ago

The buzz around big data, let's not forget, is partly fuelled by bullshit generated by vendors of big data solutions and analytic firms. It probably does not really need to be repeated, but I guess it cannot hurt. Data being big rarely causes it to be much more interesting. There can be much more actionable information in a few hundred records than in billions, depending on context, proper experimental design, and so on. Also, an often overlooked point is that precision of statistical estimators (e.g. width of confidence intervals) typically scales with the square root of sample size; yet the cost of processing an additional observation is typically constant (e.g. with distributed systems like hadoop). So in many cases there are decreasing returns to data size.

Tichy 14y ago

I'm already getting allergic to the term. Every couple of months there is a new term for the same thing to fuel a new hype cycle (machine learning, computational intelligence, data mining,...). Not that it isn't an important thing, and certainly more and more data is becoming available. Now lets get back to work.

drucken 14y ago

Painful reading an HN front page article like this. An entire article around use of two-word (marketing) nomenclature with no informative value in the article at all...