4 ms·
I really like this article. It reflects a historical perspective that I find lacking in most tech media, where the limits of data and method are almost always
by randcraw 8y ago
I really like this article. It reflects a historical perspective that I find lacking in most tech media, where the limits of data and method are almost always overlooked, and it's imagined that the world is entirely different today than it was in 1990. In fact, the world is very much the same today -- it's the COMPUTERS that are different (and the network). We're just looking at it through glasses with tinted lenses.
(BTW, I've also worked with petabyte-scale social media data, and the author's suggestion that we often don't ask clear questions or use clearly discriminate methods is spot on. Too often we imagine that deeper dives into higher resolution data will yield more and better information. But we very rarely test that assumption.)
I think the core problem is, we seldom take the effort to understand the limits of our big data or its sources. And that's because, in the end, nobody really cares if our datamined conclusions are real or imaginary, just as long as the graphs look cool.
- grigjd3 8y agoI also work at a petabyte scale web media company and what I've learned is we're not selling accuracy. We allowing ad campaign managers to use buzzwords with their employers.
- triplee 8y agoSo much this. It's why data engineering ends up being more and more useful, even if it's not sexy. Automating the boring parts of dumping all the ugly data and cleaning up the stuff that is sampled is a better use of resources.