3 ms·
It's been a backburner project of mine to data-mine the publicly-available clinical study information...while not all results are reported, there are useful met
by danso 14y ago
It's been a backburner project of mine to data-mine the publicly-available clinical study information...while not all results are reported, there are useful meta-analyses that can be done (i.e. which companies are least likely to report, under what circumstances, such as study size and disease, are studies likely to omit info?).
It's not immediately evident, but ClinicalTrials.gov does provide their database in a structured format...probably the most convenient endpoint is this giant sitemap, which they have generated for scraping uses:
http://www.clinicaltrials.gov/ct2/about-site/crawling http://www.clinicaltrials.gov/ct2/about-site/crawling
- ZoFreX 14y agoYou might be interested in the work the Cochrane Collaboration does: http://www.cochrane.org/ http://www.cochrane.org/
- primaryobjects 14y agoI've actually written a search crawler for clinicaltrials.gov. Their XML format is fairly convenient for parsing. I never thought for mining those types of data-points though. I suppose you could look at the study completion status and whether results are posted or not http://www.clinicaltrialweb.com http://www.clinicaltrialweb.com
- cai 14y agoMy experience of the status on clinicaltrials.gov is that it isn't always accurate. I don't think too much effort is put into keeping the data current.
- danso 14y agoI think there's a lot of potential for meta-analyses...do a Google search for those that have been done by academia...I've seen some use the ClinicalTrials data as a proxy for seeing which diseases seem profitable enough to cure, or to see trends in the methodology of trial being done. It seems like an area in which a tech-savvy person could make inroads, as some of the scholarly research may not be done with the most efficient forms of data-gathering (i.e. not many scientists are programmers)
- merinid 14y agoThey also actually have a structured format for download (CSV from an Oracle DB Dump).