4 ms·
"I suggest that ‘Big Data’ analyses are no more prone to this kind of problem than any other kind of analysis." The notion that 'big data' is just as susceptib
by onion2k 13y ago
"I suggest that ‘Big Data’ analyses are no more prone to this kind of problem than any other kind of analysis."
The notion that 'big data' is just as susceptible to bad statistical analysis if you ignore dynamics in the incoming data is entirely true. In that sense 'big data' is the same as every other kind of analysis. But there is another notable difference between 'big data' analysis and any other kind: the marketing. 'Big data' solutions are often sold as something you can just plug in to your infrastructure, set up some data feeds, and out pops an insightful trend analysis telling you things about your customers that you could never have understood with Excel alone. That is what's wrong with 'big data'.
Used properly, by an intelligent statistician ("data scientist") who knows their field well, big data tools are very useful for doing the things that statisticians do only faster. But that is all. Big data tools don't magically do clever statistical analysis on their own, and they are not a replacement for statisticians, despite what some people seem to think.
- walshemj 13y agoIt just makes practible analysis that before would have been to costly before. I suspect most of the backlash is from MBA students who could make a spread sheet draw purty pictures but crash and burn when required to do any real work. By that argument Lyons tea shops should have never have built LEO - after all they had a manual system that could track the cost of a bun down to fractions of a farthing (1/4 of a penny)