2 ms·
I've found in my data-mining experience that the most interesting data (at least on the Web) is not particularly easy to parse, even if you write something that
by danso 15y ago
I've found in my data-mining experience that the most interesting data (at least on the Web) is not particularly easy to parse, even if you write something that automates a form's POST submissions. The second difficult part is normalizing it, as much web/text data is formatted for display to humans, which is quite different than data in easily analyzable form.
So given that, it's just worth learning enough program to do loops, conditionals, and regexes to get what you want.
- St-Clock 15y agoThis is so true. I've been mining mailing lists and framework documentation for my Ph.D. and most of my effort and time was spent normalizing the data. Once that was done, classifying content and linking concepts was relatively easy...