7 ms·

13M Pages of Declassified CIA Documents Posted Online

200 points by happy-go-lucky 10y ago

SeaDude 10y ago
Interesting searches of the cache include "LSD", "hacker", "Martin Luther King", and "psilocybin". After those searches, the resource was suddenly unavailable.
- spuiszis 10y ago
  I haven't gotten the search to work at all, assuming that search bar is for the documents (the UX isn't great). I've been getting "CIA.gov is Temporarily Unavailable" each time. That said, there are some interesting (and mildly terrifying) documents you can find from some general browsing of the categories.
- deleted 10y ago
  [deleted]
- throwanem 10y ago
  It's federal IT. I'm going to guess overload due to extremely resource-inefficient implementation before I guess "uh oh, somebody's curious about MKULTRA, better pull the search".
ChuckMcM 10y ago
Well sort of: CIA.gov is Temporarily Unavailable CIA.gov is temporarily unavailable. We apologize for the inconvenience. Please try again later. Posted: Aug 27, 2012 04:31 PM Last Updated: Aug 27, 2012 04:31 PM That was the result of my search 'Nicaraguan contra'
- retSava 10y ago
  Considering they went from about four computers accessing the db, to the Internet, I'm leaning more towards the reddit/HN/whatever-effect :) Here's to hoping the "someone who got called in to fix it" makes a tremendous work in scaling it.
  - throwanem 10y ago
    Federal IT. Betting it's chock-full of easy wins for a savvy dev, and those do tend to turn up with a GS number from time to time. They rarely seem to stick, though, which I gather is part of what 18F is trying in a small way to change.
- eatbitseveryday 10y ago
  Search another term, like 'keyhole' and it succeeds immediately. Switch back and forth and you can see it's deliberate.
  - dawnerd 10y ago
    Or just cached...
pizza 10y ago
link: https://www.cia.gov/library/readingroom/collection/crest-25-year-program-archive https://www.cia.gov/library/readingroom/collection/crest-25-... Search function not working - redirected url is "https://www.cia.gov/$(SERVE_NS_)/library/readingroom/collection/crest-25-year-program-archive" https://www.cia.gov/$(SERVE_NS_)/library/readingroom/collect... ?
hackuser 10y ago
I hope someone downloads the contents quickly, before another administration takes it offline or 'reclassifies' the information.
- 0xcde4c3db 10y ago
  I wonder if that's actually the motivation behind opening it up now rather than weeks or months from now. There are apparently efforts underway at other federal and federally-funded bodies (e.g. EPA, NOAA, NASA, some federally-funded college departments) to get backups of raw data into safe places in case Congress or the Trump administration orders it taken offline or even deleted [1]. [1] https://www.washingtonpost.com/news/energy-environment/wp/2016/12/13/scientists-are-frantically-copying-u-s-climate-data-fearing-it-might-vanish-under-trump/ https://www.washingtonpost.com/news/energy-environment/wp/20...
  - leereeves 10y ago
    Funny that they only worry about that now that Republicans are taking over. Reportedly, Obama ordered just such a deletion of records. http://thehill.com/blogs/congress-blog/homeland-security/268282-dhs-ordered-me-to-scrub-records-of-muslims-with-terror http://thehill.com/blogs/congress-blog/homeland-security/268...
    - knz 10y ago
      Is there anything in this article that actually links the deletion to President Obama or explains why the deletion was ordered? Or even any evidence that this happened? Edit to be clear: I'm not implying the Obama administration did not do things like this but this read more like an anecdote by someone with political leanings. We also have no information about why they were deleted or modified and don't know if the author had access to the full picture. Perhaps they were merged with another record, therefore improving the "haystack" etc.
      - leereeves 10y ago
        The author made that connection: "the Obama administration was ordering they be wiped away." I haven't seen any followup on this story. Liberal media seems to have ignored it entirely, conservative media reports it without further details. The author, Philip Haney, has been a vocal critic, appearing on Fox and writing numerous articles, so there's been ample opportunity for the Obama administration to respond if there was indeed an explanation such as you suggest, but I haven't seen any response.
elastic_church 10y ago
They should put it in a torrent
- sebleon 10y ago
  They likely won't, but would be awesome if someone here on HN did!
- erelde 10y ago
  Torrents are only for criminals. /s (edit: double sarcasm in fact)
- yq 10y ago
  Since the landing page required everyone interested to enter a keyword to search, the back-end server can record all query and ip-addrs. I doubt if WikiLeaks.org use this exact same method to study what document attract traffic|attention. WikiLeaks' new releases are not signed by PGP anymore, I take that as either a Warrant Canary or some kinds of warning. Of course, if everything packed in a torrent, then there's no way to conduct such a study.
  - gruez 10y ago
    You can iterate through all the documents by going to https://www.cia.gov/library/readingroom/search/site https://www.cia.gov/library/readingroom/search/site, going through each year, and within each year, going through all the result pages.
deleted 10y ago
[deleted]
BuuQu9hu 10y ago
The printing hack was pretty nice :)
JorgeGT 10y ago
Tip: use the codeword "keyhole" to browse interesting satellite reconnaissance photos: https://www.cia.gov/library/readingroom/docs/CIA-RDP78B04560A002000010029-2.pdf https://www.cia.gov/library/readingroom/docs/CIA-RDP78B04560...
- lallysingh 10y ago
  Ref: https://en.m.wikipedia.org/wiki/Keyhole,_Inc https://en.m.wikipedia.org/wiki/Keyhole,_Inc and https://en.m.wikipedia.org/wiki/Corona_(satellite) https://en.m.wikipedia.org/wiki/Corona_(satellite)
- digler999 10y ago
  > March 1964 > Approved for release 2001/8/21
lostboys67 10y ago
The declassified studies in indigence are an interesting read - I remember one where they documented some of the kooks that wrote to the DCI.
mack73 10y ago
The redacted parts in the documents have codes, e.g.: (b)(1) (b)(3) NatSecAct What are those codes refering to? Are only names redacted? For you who have trouble searching, here's a document I just stumbled on while browsing through the content: https://www.cia.gov/library/readingroom/docs/0006541525.pdf https://www.cia.gov/library/readingroom/docs/0006541525.pdf It is about interrogation techniques used by USA (power drills were mentioned). Scary stuff. I wonder, if you train a machine with content from major (all?) news sites, could it then, in theory, predict what was redacted to some degree? I have an itch to scrape this content. Maybe I shouldn't.
- andrefrancisco 10y ago
  They are codes for FOIA exemptions. http://www.foiadvocates.com/exemptions.html http://www.foiadvocates.com/exemptions.html
bootload 10y ago
CREST has been available to the public for a while now, though not online. ODCI/DI would be interesting to look into but I won't hold my breath finding anything outlining the bastardly of the early 60s. [0] First terms I looked up, "Budget Hawkeye" for 1963-64. Found some details, hard numbers were redacted. NPIC ^grew^ a lot post '63 judging by the budget requests. Lots of files referencing "Brass Knob" in '63 (daily Cuban flyover). The other interesting details were NPIC monthly memos detailing the daily operations. Some interesting finds - David A. Phillips ('81) https://www.cia.gov/library/readingroom/document/cia-rdp90-00845r000201240009-0 https://www.cia.gov/library/readingroom/document/cia-rdp90-0... - Howard E. Hunt ('85) https://www.cia.gov/library/readingroom/document/cia-rdp90-00845r000201240009-0 https://www.cia.gov/library/readingroom/document/cia-rdp90-0... - Jim Garrison ('67) https://www.cia.gov/library/readingroom/document/cia-rdp90-00965r000402730006-3 https://www.cia.gov/library/readingroom/document/cia-rdp90-0... - Film 'Executive Action' ('73) https://www.cia.gov/library/readingroom/document/cia-rdp88-01365r000300070003-7 https://www.cia.gov/library/readingroom/document/cia-rdp88-0... [0] https://news.ycombinator.com/item?id=12324273 https://news.ycombinator.com/item?id=12324273 [1] https://www.cia.gov/library/readingroom/document/cia-rdp76-00183r000400030140-4 https://www.cia.gov/library/readingroom/document/cia-rdp76-0...

200 points by happy-go-lucky 10y ago

Interesting searches of the cache include "LSD", "hacker", "Martin Luther King", and "psilocybin". After those searches, the resource was suddenly unavailable.

spuiszis 10y ago
I haven't gotten the search to work at all, assuming that search bar is for the documents (the UX isn't great). I've been getting "CIA.gov is Temporarily Unavailable" each time. That said, there are some interesting (and mildly terrifying) documents you can find from some general browsing of the categories.
deleted 10y ago
[deleted]
throwanem 10y ago
It's federal IT. I'm going to guess overload due to extremely resource-inefficient implementation before I guess "uh oh, somebody's curious about MKULTRA, better pull the search".

ChuckMcM 10y ago

Well sort of: CIA.gov is Temporarily Unavailable CIA.gov is temporarily unavailable. We apologize for the inconvenience. Please try again later. Posted: Aug 27, 2012 04:31 PM Last Updated: Aug 27, 2012 04:31 PM That was the result of my search 'Nicaraguan contra'

retSava 10y ago
Considering they went from about four computers accessing the db, to the Internet, I'm leaning more towards the reddit/HN/whatever-effect :) Here's to hoping the "someone who got called in to fix it" makes a tremendous work in scaling it.
- throwanem 10y ago
  Federal IT. Betting it's chock-full of easy wins for a savvy dev, and those do tend to turn up with a GS number from time to time. They rarely seem to stick, though, which I gather is part of what 18F is trying in a small way to change.
eatbitseveryday 10y ago
Search another term, like 'keyhole' and it succeeds immediately. Switch back and forth and you can see it's deliberate.
- dawnerd 10y ago
  Or just cached...

pizza 10y ago

link: https://www.cia.gov/library/readingroom/collection/crest-25-year-program-archive https://www.cia.gov/library/readingroom/collection/crest-25-... Search function not working - redirected url is "https://www.cia.gov/$(SERVE_NS_)/library/readingroom/collection/crest-25-year-program-archive" https://www.cia.gov/$(SERVE_NS_)/library/readingroom/collect... ?

hackuser 10y ago

I hope someone downloads the contents quickly, before another administration takes it offline or 'reclassifies' the information.

0xcde4c3db 10y ago
I wonder if that's actually the motivation behind opening it up now rather than weeks or months from now. There are apparently efforts underway at other federal and federally-funded bodies (e.g. EPA, NOAA, NASA, some federally-funded college departments) to get backups of raw data into safe places in case Congress or the Trump administration orders it taken offline or even deleted [1]. [1] https://www.washingtonpost.com/news/energy-environment/wp/2016/12/13/scientists-are-frantically-copying-u-s-climate-data-fearing-it-might-vanish-under-trump/ https://www.washingtonpost.com/news/energy-environment/wp/20...
- leereeves 10y ago
  Funny that they only worry about that now that Republicans are taking over. Reportedly, Obama ordered just such a deletion of records. http://thehill.com/blogs/congress-blog/homeland-security/268282-dhs-ordered-me-to-scrub-records-of-muslims-with-terror http://thehill.com/blogs/congress-blog/homeland-security/268...
  - knz 10y ago
    Is there anything in this article that actually links the deletion to President Obama or explains why the deletion was ordered? Or even any evidence that this happened? Edit to be clear: I'm not implying the Obama administration did not do things like this but this read more like an anecdote by someone with political leanings. We also have no information about why they were deleted or modified and don't know if the author had access to the full picture. Perhaps they were merged with another record, therefore improving the "haystack" etc.
    - leereeves 10y ago
      The author made that connection: "the Obama administration was ordering they be wiped away." I haven't seen any followup on this story. Liberal media seems to have ignored it entirely, conservative media reports it without further details. The author, Philip Haney, has been a vocal critic, appearing on Fox and writing numerous articles, so there's been ample opportunity for the Obama administration to respond if there was indeed an explanation such as you suggest, but I haven't seen any response.

elastic_church 10y ago

They should put it in a torrent

sebleon 10y ago
They likely won't, but would be awesome if someone here on HN did!
erelde 10y ago
Torrents are only for criminals. /s (edit: double sarcasm in fact)
yq 10y ago
Since the landing page required everyone interested to enter a keyword to search, the back-end server can record all query and ip-addrs. I doubt if WikiLeaks.org use this exact same method to study what document attract traffic|attention. WikiLeaks' new releases are not signed by PGP anymore, I take that as either a Warrant Canary or some kinds of warning. Of course, if everything packed in a torrent, then there's no way to conduct such a study.
- gruez 10y ago
  You can iterate through all the documents by going to https://www.cia.gov/library/readingroom/search/site https://www.cia.gov/library/readingroom/search/site, going through each year, and within each year, going through all the result pages.

deleted 10y ago

[deleted]

BuuQu9hu 10y ago

The printing hack was pretty nice :)

JorgeGT 10y ago

Tip: use the codeword "keyhole" to browse interesting satellite reconnaissance photos: https://www.cia.gov/library/readingroom/docs/CIA-RDP78B04560A002000010029-2.pdf https://www.cia.gov/library/readingroom/docs/CIA-RDP78B04560...

lallysingh 10y ago
Ref: https://en.m.wikipedia.org/wiki/Keyhole,_Inc https://en.m.wikipedia.org/wiki/Keyhole,_Inc and https://en.m.wikipedia.org/wiki/Corona_(satellite) https://en.m.wikipedia.org/wiki/Corona_(satellite)
digler999 10y ago
> March 1964 > Approved for release 2001/8/21

lostboys67 10y ago

The declassified studies in indigence are an interesting read - I remember one where they documented some of the kooks that wrote to the DCI.

mack73 10y ago

The redacted parts in the documents have codes, e.g.: (b)(1) (b)(3) NatSecAct What are those codes refering to? Are only names redacted? For you who have trouble searching, here's a document I just stumbled on while browsing through the content: https://www.cia.gov/library/readingroom/docs/0006541525.pdf https://www.cia.gov/library/readingroom/docs/0006541525.pdf It is about interrogation techniques used by USA (power drills were mentioned). Scary stuff. I wonder, if you train a machine with content from major (all?) news sites, could it then, in theory, predict what was redacted to some degree? I have an itch to scrape this content. Maybe I shouldn't.

andrefrancisco 10y ago
They are codes for FOIA exemptions. http://www.foiadvocates.com/exemptions.html http://www.foiadvocates.com/exemptions.html

bootload 10y ago

CREST has been available to the public for a while now, though not online. ODCI/DI would be interesting to look into but I won't hold my breath finding anything outlining the bastardly of the early 60s. [0] First terms I looked up, "Budget Hawkeye" for 1963-64. Found some details, hard numbers were redacted. NPIC ^grew^ a lot post '63 judging by the budget requests. Lots of files referencing "Brass Knob" in '63 (daily Cuban flyover). The other interesting details were NPIC monthly memos detailing the daily operations. Some interesting finds - David A. Phillips ('81) https://www.cia.gov/library/readingroom/document/cia-rdp90-00845r000201240009-0 https://www.cia.gov/library/readingroom/document/cia-rdp90-0... - Howard E. Hunt ('85) https://www.cia.gov/library/readingroom/document/cia-rdp90-00845r000201240009-0 https://www.cia.gov/library/readingroom/document/cia-rdp90-0... - Jim Garrison ('67) https://www.cia.gov/library/readingroom/document/cia-rdp90-00965r000402730006-3 https://www.cia.gov/library/readingroom/document/cia-rdp90-0... - Film 'Executive Action' ('73) https://www.cia.gov/library/readingroom/document/cia-rdp88-01365r000300070003-7 https://www.cia.gov/library/readingroom/document/cia-rdp88-0... [0] https://news.ycombinator.com/item?id=12324273 https://news.ycombinator.com/item?id=12324273 [1] https://www.cia.gov/library/readingroom/document/cia-rdp76-00183r000400030140-4 https://www.cia.gov/library/readingroom/document/cia-rdp76-0...