4 ms·

Big Data will flood the planet in 2012

41 points by zeratul 15y ago

loeg 15y ago
"Source: EMC" (EMC sells "Big Data" storage.)
- mberning 15y ago
  My thoughts exactly. "Big Data" gets a lot of lip service from hardware and software vendors, but for the most part they are pushing solutions to problems that 99.9% of people do not have. Most shops can't even get a basic data warehouse and BI tool up and running AND have their customers/users be happy with the solution. The idea that those same teams now need to start using special tools to analyze their 'big data' and hire data engineers/data scientists to do it is absurd.
  - thematt 15y ago
    I agree completely. However, I also think that those shops that do choose to parse and interpret massive amounts of data in new ways will have decisive advantages over those that don't.
zeratul 15y ago
I'm still trying to figure out what people mean by "big data". I'm full of hope that they don't mean "big storage" or "big hard drives filled with big bytes". The data by it self does not have any value. Maybe some need to store data just for legal purposes. If ever sued they can show the records. The value comes from summarizing the data into something that you can use it for making decisions. So my hope is that media are really saying: "there will be a lot of data to analyze - start practicing now."
- deleted 15y ago
  [deleted]
- Helianthus 15y ago
  It seems to be a confusing comparison to Big Oil or Big Pharma etc. In other words, it's a buzzphrase coined by people who appreciate catchiness over actual meaning: the people who are a few layers removed from making important decisions and have little else to do but talk incessantly about an obvious trend.
ahalan 15y ago
TheEconomist>> Hadoop, a new type of database, can be used to sift through big data streams in real time. HN>> Hadoop is not a new type of database, and it cannot be used to sift through big data streams in real time. Related>> Why big data is better than small data : http://www.quora.com/Why-the-current-obsession-with-big-data http://www.quora.com/Why-the-current-obsession-with-big-data
deleted 15y ago
[deleted]
nhebb 15y ago
Most important, firms will discover that they can extract value from the data. A study by the McKinsey Global Institute, a think-tank, found that analysing health-care data could yield $300 billion-worth of savings in America alone. I smell BS. Sounds like they're priming the pump for big consulting contracts.
- DevX101 15y ago
  I read the report. I think they had the big ideas right, but the quantification of the opportunity was a bit hand wavey. The biggest chunk of that $300 billion was clinical operations, which they said would lead to a $165 Billion savings. They never explained HOW they came up with this $165 billion number, they just asserted it. This category would include things like an IBM Watson helping doctors to diagnose patients, or automated programs to help avoid adverse reactions when the doctor prescribes a drug that he shouldn't have. As important as I think these technologies are, I highly doubt they'll be reducing healthcare expenditure in the U.S. by $165 billion.
- zeratul 15y ago
  Actually, it's more than that (e.g., http://www.nejm.org/doi/full/10.1056/NEJMp1109649 http://www.nejm.org/doi/full/10.1056/NEJMp1109649) but there are two serious bottlenecks: you need medical experts and the data might be too noisy. This means that any medical data mining project will cost a lot of money and half-way in the project you will find out that your data has too much noise. Most people don't want to take that risk, despite that benefits might be enormous. TL;DR: yes it's BS but nothing that high-risk/high-ROI startups couldn't deal with.
tybris 15y ago
Big Data, yes. Big Conclusions, no.

41 points by zeratul 15y ago

loeg 15y ago

"Source: EMC" (EMC sells "Big Data" storage.)

mberning 15y ago
My thoughts exactly. "Big Data" gets a lot of lip service from hardware and software vendors, but for the most part they are pushing solutions to problems that 99.9% of people do not have. Most shops can't even get a basic data warehouse and BI tool up and running AND have their customers/users be happy with the solution. The idea that those same teams now need to start using special tools to analyze their 'big data' and hire data engineers/data scientists to do it is absurd.
- thematt 15y ago
  I agree completely. However, I also think that those shops that do choose to parse and interpret massive amounts of data in new ways will have decisive advantages over those that don't.

zeratul 15y ago

I'm still trying to figure out what people mean by "big data". I'm full of hope that they don't mean "big storage" or "big hard drives filled with big bytes". The data by it self does not have any value. Maybe some need to store data just for legal purposes. If ever sued they can show the records. The value comes from summarizing the data into something that you can use it for making decisions. So my hope is that media are really saying: "there will be a lot of data to analyze - start practicing now."

deleted 15y ago
[deleted]
Helianthus 15y ago
It seems to be a confusing comparison to Big Oil or Big Pharma etc. In other words, it's a buzzphrase coined by people who appreciate catchiness over actual meaning: the people who are a few layers removed from making important decisions and have little else to do but talk incessantly about an obvious trend.

ahalan 15y ago

TheEconomist>> Hadoop, a new type of database, can be used to sift through big data streams in real time. HN>> Hadoop is not a new type of database, and it cannot be used to sift through big data streams in real time. Related>> Why big data is better than small data : http://www.quora.com/Why-the-current-obsession-with-big-data http://www.quora.com/Why-the-current-obsession-with-big-data

deleted 15y ago

[deleted]

nhebb 15y ago

Most important, firms will discover that they can extract value from the data. A study by the McKinsey Global Institute, a think-tank, found that analysing health-care data could yield $300 billion-worth of savings in America alone. I smell BS. Sounds like they're priming the pump for big consulting contracts.

DevX101 15y ago
I read the report. I think they had the big ideas right, but the quantification of the opportunity was a bit hand wavey. The biggest chunk of that $300 billion was clinical operations, which they said would lead to a $165 Billion savings. They never explained HOW they came up with this $165 billion number, they just asserted it. This category would include things like an IBM Watson helping doctors to diagnose patients, or automated programs to help avoid adverse reactions when the doctor prescribes a drug that he shouldn't have. As important as I think these technologies are, I highly doubt they'll be reducing healthcare expenditure in the U.S. by $165 billion.
zeratul 15y ago
Actually, it's more than that (e.g., http://www.nejm.org/doi/full/10.1056/NEJMp1109649 http://www.nejm.org/doi/full/10.1056/NEJMp1109649) but there are two serious bottlenecks: you need medical experts and the data might be too noisy. This means that any medical data mining project will cost a lot of money and half-way in the project you will find out that your data has too much noise. Most people don't want to take that risk, despite that benefits might be enormous. TL;DR: yes it's BS but nothing that high-risk/high-ROI startups couldn't deal with.

tybris 15y ago

Big Data, yes. Big Conclusions, no.