4 ms·
One aspect of Aaron Swartz's case that I found both interesting and way under-reported, was that he supposedly was going to perform meta-analysis against the co
by pasbesoin 11y ago
One aspect of Aaron Swartz's case that I found both interesting and way under-reported, was that he supposedly was going to perform meta-analysis against the collected documents. And that a previous meta-analysis of a similar sort had revealed some surprising and useful results.
With all the pieces locked up, it is difficult if not impossible to do such science and statistical analysis. In other words, it is antithetical to the nature of science.
I didn't see/read enough to confirm this, in Schwartz's case. Nonetheless, I think the idea and the opinion are valid.
- dalke 11y agoThe Google n-gram project is an example of how one might extract useful information by indexing large numbers of documents. You might, for example, look at trends in the use of a given scientific instrument, or with a bit of NP, when the field switched from using "computer" to mean a machine instead of a person. It was an eye-opener to see how gendered the scientific papers were in the 1960s, where the scientist is invariably a "he" and the secretary/typist is a "she". Some simple questions I have are: how have those trends changed over time? Is it different for different subfields? It's impossible to do that on my own when the papers cost even $1 per copy.
- pasbesoin 11y agoAlso interesting, and more concerning, is that such meta-analyses are revealing biases (in the science), systemic deficiencies, and fraud. Very valuable and pertinent information to have.
- amelius 11y agoSee also: https://en.wikipedia.org/wiki/Biomedical_text_mining https://en.wikipedia.org/wiki/Biomedical_text_mining
- austinjp 11y agoTangentially related: is anyone here familiar with automated analysis of research papers? I feel this is a field that would perfectly lend itself to machine learning, or similar approaches. It would require extraction of meaningful data, be therein lies the rub. There are several large projects looking into collecting/sharing scientific data, mainly focusing on persuading and empowering researchers to share their full data sets. But what can we do where the raw data is not available? Is anyone working on ways to reliably extract data from the millions of PDFs in research databases? There are several open-ended questions here. If anyone knows of work in these areas, or is interested in this, I'd be very keen to talk.
- vasar 11y agoOnly materials related, but at http://www.citrine.io/ http://www.citrine.io/ they do something like that. I recently attended a talk given by one of the team members (Bryce) and they seemed to be quite open to discussions.
- dalke 11y agoThe field is called text mining or literature data mining. https://en.wikipedia.org/wiki/Text_mining https://en.wikipedia.org/wiki/Text_mining
- davidkretch 11y agoAndrej Karpathy (OpenAI) does this to an extent with machine learning papers from arXiv: http://www.arxiv-sanity.com/ http://www.arxiv-sanity.com/