4 ms·
Little innovation in "little data"? What about the whole academic field of Statistics? Most of the innovation is about how to make maximal use of limited/censor
by 11001 13y ago
Little innovation in "little data"? What about the whole academic field of Statistics? Most of the innovation is about how to make maximal use of limited/censored/missing data? I don't hear "big data" folks talking much about multiple imputations, structural equation modeling, mixed-effects regressions, etc.
- glutamate 13y agoMichael Jordan (the other Michael Jordan) wrote a nice article about how the long tail of big data is little data and hierarchical models etc. should work really well there. http://bayesian.org/sites/default/files/fm/bulletins/1106.pdf http://bayesian.org/sites/default/files/fm/bulletins/1106.pd...
- hackula1 13y agoAs someone who writes predictive analytics software, I can say that you are dead on. Most of analytics is a process of doing one of two things: 1) Extrapolation of data where samples are lacking. 2) Taking big data and making it little data so that you can actually comprehend it. My goal in practically every algorithm/tool I write is to take a several billion records and condense it into something that can be put in a spreadsheet, put on a chart, or rendered on a map. Analysts roll their eyes every time some big data guru releases another map with 7 trillion points on it. "Oh so you took 3 weeks rendering a map that looks like yet another population map, when you could have rendered this instantaneously with a choropleth?"