4 ms·
It's all part of DocumentCloud, which means API access to the text of all that Guantanamo data. See how easy this can be: curl http://www.documentcloud.org
by conesus 15y ago
It's all part of DocumentCloud, which means API access to the text of all that Guantanamo data. See how easy this can be:
curl http://www.documentcloud.org/api/search.json?q=group:nytimes
"total": 2821,
"page": 1,
"per_page": 10,
"q": "group:nytimes",
"documents": [{
"id": "86275-isn-10015-abd-al-rahim-al-nashiri-jtf-gtmo",
"title": "ISN 10015 - Abd al Rahim al Nashiri - JTF-GTMO Detainee Assessment",
"access": "public",
"pages": 15,
"description": null,
"source": null,
"created_at": "Sun, 24 Apr 2011 15:50:19 +0000",
"updated_at": "Mon, 25 Apr 2011 15:12:53 +0000",
"canonical_url": "http://www.documentcloud.org/documents/86275-isn-10015-abd-al-rahim-al-nashiri-jtf-gtmo.html",
"resources": {
"pdf": "http://s3.documentcloud.org/documents/86275/isn-10015-abd-al-rahim-al-nashiri-jtf-gtmo.pdf",
"text": "http://s3.documentcloud.org/documents/86275/isn-10015-abd-al-rahim-al-nashiri-jtf-gtmo.txt",
"thumbnail": "http://s3.documentcloud.org/documents/86275/pages/isn-10015-abd-al-rahim-al-nashiri-jtf-gtmo-p1-thumbnail.gif",
"search": "http://www.documentcloud.org/documents/86275/search.json?q={query}",
"page": {
"text": "http://www.documentcloud.org/documents/86275/pages/isn-10015-abd-al-rahim-al-nashiri-jtf-gtmo-p{page}.txt",
"image": "http://s3.documentcloud.org/documents/86275/pages/isn-10015-abd-al-rahim-al-nashiri-jtf-gtmo-p{page}-{size}.gif"
}
}
},
...
]}
Look at documents[0].resources.text:
curl http://s3.documentcloud.org/documents/86275/isn-10015-abd-al-rahim-al-nashiri-jtf-gtmo.txt
There you go. Now go build that tree and report back. :-)
- mrcharles 15y agoOh, very cool. Thanks for the heads up-- If I get some free time I'll do this.
- jashkenas 15y agoI'm afraid that only a handful (< 20, so far) of the newly released documents have been redacted and published thus far. You won't get too far trying to build that tree.
- mrcharles 15y agoSure but I can start writing code now, in anticipation of the full set.
- mrcharles 15y agoSo I was fiddling a bit. Sadly, despite being able to skip the OCR step, there's obviously a lot of OCR errors in the text as is, which would make reliably stripping out data somewhat more difficult. The reality is that I don't really have the time to commit to this unless everything worked perfect. Ah well, maybe someone else will do it.