6 ms·

Ask HN: How far ahead could the NSA be?

6 points by graderjs 5y ago

For clarity, the NSA or the "classified world" in general, which includes all the corporate entities as well. Question was prompted by a remark by the following source, to the effect that, "In the course of my research, I had an acquaintance who worked for the NSA as a civilian during the 1960s, and he offhandedly said they were running computers with a clock speed of 650Mhz at that time." This source will be poo-pooed by many here I think, because we can probably look up Wikipedia for this person and find words like "debunked", "pseudoscience", "crank", or other such terms that when encountered by many people act like the "Jedi mind trick" and foreshorten any further consideration. Nevertheless, it seems there could be classified breakthroughs in technology that many years later make their way into the public market. I think it's interesting to speculate how far ahead secret tech could be. Source: Richard Dolan, Aerial Phenomena with Richard Dolan, S2 Ep8, "Is there a breakaway civilization?"

usednet 5y ago
Really, really far ahead. Just imagine the infrastructure they have right now collecting all of the internet activity in the entire US through fiber backbones. I remember reading that Britain some years ago was collecting 200+ PB of data daily and storing it for a month. That blew my mind, one can only imagine what they’re doing here in the US.
- Mister_X 5y ago
  Storing data accomplishes nothing unless the agency involved is organizing, collating and sharing that data with others. Do you really believe "they" have the time and money to do that with that much data, considering 99.999% of it is dross? I sure don't.
  - usednet 5y ago
    Yes, absolutely I do. They have protocol analyzers that reverse engineer wire traffic to put it into semistructured form for processing with MapReduce.
Mister_X 5y ago
Never heard of Richard Dolan and just tried to listen to an interview with him but couldn't make it through the first 5 minutes, he's clearly a UFO conspiracy monetizer whackadoodle.
- graderjs 5y ago
  Heh :) predictable response, as I said above. I like the word wackadoodle tho, it's an interesting name. I do recommend you try that episode, you may have a different perspective or you may feel more of the same.
eesmith 5y ago
650Mhz CPUs in the 1960s is nonsense. Cray sold lots of machines to the NSA. ("HPC is dominated by a full line of systems from Cray Research" - https://publishing.cdlib.org/ucpressebooks/view?docId=ft0f59n73z&chunk.id=d0e12308&toc.id=d0e12308&brand=ucpress https://publishing.cdlib.org/ucpressebooks/view?docId=ft0f59... ). The 1970s era Cray-1 was 80 MHz, and Cray was pushing the limits of CPU technology. You really think the NSA had the Cray installation, with NSA's own OS (Folklore) and more, and had an internal system which was >10x more powerful, that no one has heard of, with no information leak about that technology to the wider world? Whackadoodle conspiracy nonsense. Or rather, what's more likely, that that statement is true, or that Dolan and/or the NSA civilian got something wrong?
- graderjs 5y ago
  Interesting document. I remember seeing a document before that had been declassified about Cray at some US government agency in the 60s/70s. I think it was NSA. I'll try to track it down. I'm just thinking it had some technical details redacted...could it be possible that some of the MHz were redacted and were higher than the published 80 MHz figures? Maybe, right? Anyway, also interesting is you both (you and another commenter) used the terms "conspiracy" and "whackadoodle" :) First time I've seen the word whackadoodle, it's a pretty interesting name. And now I'm wondering if it's a common combiner with "conspiracy". Used often? edit: I just looked up the whackadoodle. Turns out it's a noun, too, huh. I see it's more common in the US, maybe that's why I never heard this before :) Reminds me of yankeedoodle (or, seemingly unrelatedly, "wanker", from the whacka - for masturbate, and the doodle for penis. Maybe that's just my Australian transliteration, haha). Anyway, I see the word is so interesting there was a WSJ article[0] on it from 2019. Woosh, straight over my head with that one, heh (was that too much of a 90s phrase?) :) [0]: https://www.wsj.com/articles/wackadoodle-a-ludicrous-word-for-ludicrous-times-11566575623 https://www.wsj.com/articles/wackadoodle-a-ludicrous-word-fo... - A ludicrous word for ludicrous times.
  - eesmith 5y ago
    Not by an order of magnitude! Surely you've heard of gamers who add special cooling devices so they can overclock the CPU. Increase the frequency and power consumption increases. Decrease the frequency and power consumption decreases. You don't need as much cooling. And those Crays needed special cooling. Quoting Wikipedia for the Cray-1: > each circuit board was paired with a second, placed back to back with a sheet of copper between them. The copper sheet conducted heat to the edges of the cage, where liquid Freon running in stainless steel pipes drew it away to the cooling unit below the machine. The first Cray-1 was delayed six months due to problems in the cooling system Any computer engineer working on a severely underclocked Cray is going to wonder why it has so much unused cooling. From a business sense, think how many more sales you could have without including needlessly exotic Freon cooling that requires developing new welding techniques. > declassified about Cray at some US government agency in the 60s/70s Cray was close to the NSA, yes. That's no secret. The NSA and related agencies funded a lot of supercomputing research. The popcount instruction, implemented in the Cray-1 and other machines, even has the nickname "The NSA Instruction". https://vaibhavsagar.com/blog/2019/09/08/popcount/ https://vaibhavsagar.com/blog/2019/09/08/popcount/ and https://news.ycombinator.com/item?id=27470529 https://news.ycombinator.com/item?id=27470529 . > ... interesting is you both (you and another commenter) used .. Don't assume independence between commenters. I deliberately quoted Mister_X when I used the term "whackadoodle". Try https://www.urbandictionary.com/define.php?term=Whackadoodle https://www.urbandictionary.com/define.php?term=Whackadoodle . FWIW, the term "whackadoodle" was used in the 1990s: ] (1995) It's been said that you've got to be a real "whackadoodle" to dive those truly deep sites; this book will not encourage anyone to be a "whackadoodle!" - https://archive.org/details/diveontarioguide0000kohl/page/46/mode/2up?q=Whackadoodle https://archive.org/details/diveontarioguide0000kohl/page/46... ] (1990) Ellen Beach Yaw, Maria Galvany, Erna Sack, and Miliza Korjus among others varied in the extent to which they considered an elitist theatre career part of or a mere adjunct to their appeal as whackadoodle artistes. - https://archive.org/details/dementedworldoft00mord/page/32/mode/2up?q=Whackadoodle https://archive.org/details/dementedworldoft00mord/page/32/m... .

graderjs 5y ago
The document I remembered is I believe this one: https://media.defense.gov/2021/Jul/01/2002754533/-1/-1/0/6586785-NSA-KEY-ROLE-IN-MAJOR-DEVELOPMENTS-IN-COMPUTER-SCIENCE.PDF https://media.defense.gov/2021/Jul/01/2002754533/-1/-1/0/658... I'm thinking an interesting paragraph is: Already by the early 1960s, automation completely overshadowed : manual processing. Starting in the middle of that decade, NSA began purchasing : commercially developed computers in addition to building its own units. Agency : programmers often wrote specialized software that extended the cryptologic capabilities : of these on-the-shelf systems. The computer of choice was the IBM 7010. NSA also : used the minicomputers PDP-1 and PDP-10 as well as the UNIVAC 490, along with a : software developed at NSA known as RYE. The Honeywell Corporation also developed : the Honeywell 316 to accept manual Morse data from 128 different sources ; simultaneously and record it directly onto magnetic tape. As well, collection systems : even in remote sites became digitally based. Furthermore, computers also allowed for k the collection of a wider variety of signals, including ____________________ By 1968 NSA had over 100 computers, organized in complexes according to the type of processing performed. NSA's five acres of equipment became the largest collection of advanced computers in the United States and probably in the world. Another one is: There were other important computer firsts at NSA. A different supercomputer was developed by the Thinking Machine Corporation. This computer was known as the CM-5, or FROSTBURG. Built at a cost of $25 million, the CM-5 had a storage capacity of 500 gigawords and could perform 65 billion calculations per second. Designed to address higher-mathematics-related questions, FROSTBURG was one of the first true parallel processing computers. Also worth mentioning, when instaJJed the PACE 10 was the first analog desktop computer in use at NSA. This computer required the hand wiring of different panels for each type of mathematical calculation to be performed. Of further note, NSA developed many unique and pioneering software systems and databases, such as SOLIS, which held all NSA electronic product reports. One possible thing that happened is someone maybe translated the compute into layman's terms and in translation the 'flops' and 'MHz' was confused. So 65 billion ops per second in parallel, somehow became a processor with a clock speed of 650 MHz. Maybe there's not much difference between those two measurements: I'm not a supercomputer expert after all, and anyway I'm not definitely saying that's exactly what happened--I still believe they had secret tech way ahead of any documented in declassified releases--but it's one possibility. Ultimately, this thread was probably only going to be about speculating. Without any multiply corroborated first-hand information, most people can't say for sure one way ro the other. Thank you for that mildly interesting comment and those links. I appreciate the 90s refs! :) I think your reasoning about the economics, and business are definitely solid. But that would have to assume that such an advance was an actual product line known to the Cray business in general, and not a special compartmentalized project developed in secret. The latter case I don't think overarching business drivers would have much impact because of the veil of secrecy, and they wouldn't be, by definition, strapped for cash for such a project anyway. Ah, so you copied the other commenter! Hah :) Interesting. Whackadoodle is such a popular word that you're copying someone using it, and I never heard of it. I was going to search UB but I thought their descriptions are a little extreme, seems like it in this case too. Anyway, this is not the fun discussion of possible future tech breakthroughs that I'd hoped to encourage here, but what the hell let's roll with it. Anyway, what I really wanted to see here was lovely speculations about what possible things could they have had, how far they could have been ahead! Maybe next time! :)
- eesmith 5y ago
  > 65 billion ops per second in parallel, somehow became a processor with a clock speed of 650 MHz. That specific example is impossible. The CM-5 was 1990 technology. For all you know, that civilian could have said 650 kHz. > Without any multiply corroborated first-hand information, most people can't say for sure one way ro the other. We have that. We have lots of primary sources from that era which have been declassified. You and I both linked to pages with some of that historical information. We have people talking about their experiences. You can even see FROSTBURG in a museum. And yet you regard a single second- or third-hand, unverifiable claim by Dolan as having any meaningful weight? Why? > not a special compartmentalized project developed in secret. Then you know nothing about Cray Research, and are seemingly coming up with ways to patch your conspiracy theory - because what you are suggesting is a conspiracy theory. For your conspiracy theory to work, just where is this special compartmentalized project supposed to be located? Seymour Cray put CRI's R&D site in his home town of Chippewa Falls. Wikipedia informs me the population of the town in 1970 was 12,351. You don't hide a second project in a town like that. And Cray put it there so he wouldn't have to deal with business meetings. If Cray had another project, somewhere besides his home town, then he would either 1) be off-site often ... so why locate Cray's public R&D there at all?, or 2) have someone else better than Cray[1] working at the other site. It makes no sense unless you think there's thousands of people engaged not just in a conspiracy of silence, but also who managed to prevent any hint of tech transfer between the projects. > this is not the fun discussion of possible future tech breakthroughs Then as a suggestion, don't bring up cockamamie [2] conspiracies about topics you know little about ("I'm not a supercomputer expert") in the hopes of getting that discussion. There's a lot of literature about the history of supercomputing, the NSA, and related project. Read that literature to learn what sorts of multiple corroboration exist. Then you can seriously evaluate if alternative stories are a better fit to the evidence. [1] The blurb for "The supermen : the story of Seymour Cray and the technical wizards behind the supercomputer" (book available from https://archive.org/details/supermenstoryofs0000murr https://archive.org/details/supermenstoryofs0000murr ) says: > "After a rare speech at the National Center for Atmospheric Research in Boulder, Colorado, in 1976, programmers in the audience had suddenly fallen silent when Cray offered to answer questions. He stood there for several minutes, waiting for their queries, but none came. When he left, the head of NCAR's computing division chided the programmers. 'Why didn't someone raise a hand?' After a tense moment, one programmer replied, 'How do you talk to God?'" -from The SUPERMEN The Story of Seymour Cray and the Technical Wizards behind the Supercomputer Your conspiracy theory requires coming up with someone more clever than Cray as the tech lead, who never talk about it, and nothing ever became known about it. Except for an odd slip by a civilian. [2] This time I chose a word my father would use.

graderjs 5y ago
Thank you for continuing to engage! :) This is a good to great opportunity for me to practice my skills. I'll reply in a bit :)
- graderjs 5y ago
  > 650Mhz CPUs in the 1960s is nonsense. Prove it. :) > Don't assume independence between commenters. Am I? Heh :) Your independence is you can choose what to say. > That specific example is impossible. The CM-5 was 1990 technology. How do you know what's impossible? Are you an insider to NSA's SAPs? > We have that. We have lots of primary sources from that era which have been declassified. What we have is the declass story. The classified story could be different. > Why? Because of the nature of SCI SAPs. > Then you know nothing about Cray Research, and are seemingly coming up with ways to patch your conspiracy theory - because what you are suggesting is a conspiracy theory. You’re insisting SAPs are “conspiracies”. Hahaha :) > For your conspiracy theory to work, just where is this special compartmentalized project supposed to be located? The location of a SAP would be classified, obviously. > the population of the town in 1970 was 12,351. You don't hide a second project in a town like that. Pine Gap is a small town, many SAPs are hidden there. Los Alamos is a small town, many SAPs are there, including the Manhattan Project's Fat Man. > And Cray put it there so he wouldn't have to deal with business meetings….better than Cray[1] working at the other site. As the public face Cray is not going to be deep in any SAP, as it's a security mistake to allocate the senior leader to a technical role in an SAP, and a resource misallocation because the leader will have many other commitments. > It makes no sense unless you think there's thousands of people engaged not just in a conspiracy of silence, but also who managed to prevent any hint of tech transfer between the projects. You just described exactly how TS/SCI SAPs work > Then as a suggestion, don't bring up cockamamie [2] conspiracies about topics you know little about ("I'm not a supercomputer expert") in the hopes of getting that discussion. That’s the best way to get an answer. Just say, “I don’t know, I’m interested”, and people who show up. > There's a lot of literature about the history of supercomputing, the NSA, and related project. Read that literature to learn what sorts of multiple corroboration exist. Then you can seriously evaluate if alternative stories are a better fit to the evidence. I don’t need to read that. I can just enjoy it, thinking about it, looking at it. > Your conspiracy theory requires coming up with someone more clever than Cray as the tech lead, who never talk about it, and nothing ever became known about it. Not really, as the location is fine, and SAPs will be staffed with many experts more clever than Cray in their niche. The public face will not be deep into an SAP and probably wouldn't even be cleared for it. Do you think is Elon cleared for and deep into every SAP SpaceX has going?
  - eesmith 5y ago
    > Prove it. :) You have yet to explain why a single comment, heard second- or third-hand, about a putative 650 MHz CPU in the 1960s, is so believable that you would rather believe there's a secret project a least a decade ahead of the rest of the world, than believe it was misheard or falsified. The only people I know who do that are conspiracy theorists. > The classified story could be different. Yes, and you could be living inside a Truman-like artificial environment, where everything around you is a carefully constructed zoo of what humans used to be like. Now prove you aren't. Or, Richard Dolan could be a cockamamie conspiracy theorist. Prove he isn't. > How do you know what's impossible? Because I have an idea of how much work it takes to make something like a Cray and what materials technology was like. Integrated chips were still new. The Apollo Guidance Computer famously was the first computer based on silicon ICs, and it was about the level of an Apple ][. The earlier discrete systems would be speed of light limited. 650 MHz means light travels 46 cm in vacuum. Which is roughly the size of the Apollo Guidance Computer. And we have many stories about how hard it was to build the AGC, and how much work it took to program it. Your conspiracy theory requires a system which is about 300x more powerful than the Apollo program could develop, despite access to the world's public resources. It seems you would rather make up a fake history about the secret world than enjoy the amazing work that actually did happen. > Manhattan Project's Fat Man The Manhattan Project is quite instructive about how hard it is to keep things a secret. Quoting http://blog.nuclearsecrecy.com/2013/03/29/narratives-of-manhattan-project-secrecy/ http://blog.nuclearsecrecy.com/2013/03/29/narratives-of-manh... ] There were leaks about the Manhattan Project (and atomic bombs and energy in general) printed in major press outlets in the United States and abroad. It was considered an “open secret” among Washington politicos and journalists that the Army was working on a new super-weapon that involved atomic energy just prior to its use. Don't forget the Soviets knew about it too. > many experts more clever than Cray in their niche In the niche field of high performance computing, there was no one better than Cray. Again, 'How do you talk to God?' That's one of the more annoying things about conspiracy experts. They demean the history we know about by presuming there must be a secret history so much better that brilliant people like Cray are but shadows of these unnamed people who never boast about themselves, never write memoirs, and whose works are never de-classified, even now when 1GHz laptops are passé. And they don't care to learn more about actual history to understand when something doesn't have the ring of truth.

graderjs 5y ago
I haven't looked at it yet. Hold up, let me read and get back to your message from sometime Friday. I've got a very busy time until then. :)
- deleted 5y ago
  [deleted]
- graderjs 5y ago
  I'm sorry for my delay in replying you, I need to set up a customer demo this weekend. I really have no time for this until I can properly read it and reply. Probably Monday! Thanks, hope you have a good weekend. :)
  - graderjs 5y ago
    Hello eesmith, I finally got to your comment. I'm sorry about my delay, I was so busy and I think this is important so I wanted to set aside some time where I could really think about it and respond properly. I hope you enjoy! :) > Prove it. :) You have yet to explain … you would rather believe there's a secret project a least a decade ahead of the rest of the world, than believe it was misheard or falsified. Ha :) I think a decade is easy and the truth is further. I’m satisfied with the explanation of SAPs, and if you’re not, you know your interpretation is not something I can’t help you with, right? :) The only people I know who do that are conspiracy theorists. Then you don’t know me, heh :) Yes… you could be living inside a Truman-like artificial environment…a carefully constructed zoo of what humans used to be like. Ha :) Is that why you find it hard to accept that SAPs imply much unknown, because you think it radically alters your reality? Could that not also be a good thing to refresh your perspective, and unblock you to speculate on larger possibilities? Hard to dispute that classified history diverges, and seems a stretch to imply that SAP existence means reality is a simulation. Richard Dolan could be a cockamamie conspiracy theorist…Prove he isn't. Does a label alter the truth or thinking? Labels can function as a Jedi mind trick to discourage more thinking tho. Are the prevalence of such labels why you find it hard to accept this, because the labels connote shame or craziness? Because I have an idea… how much work it took to program it. Knowledge based on one (public, I assume, heh :) ?) compartment. The existence of SAPs implies much unknown publicly, why insist that public knowns proscribe all unknowns? Your conspiracy theory requires a system which is about 300x more powerful… seems you would rather make up a fake history about the secret world than enjoy the amazing work that actually did happen. People can enjoy being inspired by both. :) There were leaks about the Manhattan Project … just prior to its use. Did those leaks come from Los Alamos? The project occurred at many sites. That SAPs are hidden in small towns seems hard to dispute, heh :) Don't forget …Cray. Again, 'How do you talk to God?' No one public at any rate. That people were better in their niche, seems hard to dispute. That's one of the more annoying things about conspiracy experts. They demean… even now when 1GHz laptops are passé. Being inspired by public history can prompt people to speculate, which can be seen as admiration of public breakthroughs, not undermining. And they don't care…doesn't have the ring of truth Will your “ring of truth” be the same as someone who has worked in SAPs?
    - eesmith 5y ago
      > Did those leaks come from Los Alamos? The project occurred at many sites. That SAPs are hidden in small towns seems hard to dispute, heh :) Since you don't care to 1) present your sources, nor 2) verify my sources ... which answer your question .. , nor 3) address the speed-of-light problem I mentioned earlier, I give up.
      - graderjs 5y ago
        Well thank you for the challenging and somewhat enjoyable sparring session. Word to the wise I think you picked a very good time to bow out for yourself. I look forward to coming across you here again in future. Good day!
      - graderjs 5y ago
        Verify your sources? That source is rubbish with respect to this question, I looked and it doesn't say anything about leaks from Los Alamos. Your 'source' answers nothing, sorry. I presented my sources: my intelligence and intuition, plus the existence of SAPs. The speed of light problem? Did the speed of light change much in the decades between these breakthroughs? I don’t think so, and I don't think it's the blocker. You’re saying the unknown is impossible. I’m saying the unknown may be. Your claim is stronger and requires more rigorous proof. More importantly, I see through your behavior and your claims. You have a pattern: you dress up in rhetoric that projects authority, while misframing any other views as crazy but underneath such abuse your points are weak, and not even logical. Your projected your "conspiracy theory" obsession onto this discussion, and said it 20 times. I never said conspiracy (a secret plot to do something bad). And the only theory I put here is about projects, what happens in them. Yet from your misframing, every SAP is a supposedly a conspiracy while you act like their existence is impossible--yet SAPs are fact. Ignoring that, and obsessively repeating the "Jedi mind trick" mantra of "conspiracy theory" as a term of abuse as if it will shame anyone with a view different to yours into silence, or somehow banish the truth, is crazy. And it's bad, and abusive. All because you're insecure, overcompensating by trying to "prove" you're right, and other's "wrong", and uncomfortable with saying, "I don't know". But if you remove all the intimidating, framing rhetoric, you points are just a lot of nonsense and lack of logic. You get a scared person, being mean to others, refusing to open their mind in the face of logic, and insisting on abusively imposing their small minded view on other people, because they’re too threatened by the reality that there is much they don’t know. Basically it seems like what your saying is: SAPs are not possible to be 10 years more advanced than public knowledge, SAPs cannot even really exist because it’s too hard to keep something secret, and anyone who suggests that SAPs could be far ahead, are actually just “cockamamie conspiracy theorists” intent on not paying due credit and admiration to the people who deserve it. And anything secret can be predicted absolutely by what you already know. But actually it sounds like you have a conspiracy theory...about conspiracy theorists, and the limits of your knowledge. And it seems like you're in the parallel universe here because in everything you're saying it's like you pitted against the evil conspiracy theorists of the world who you are imagining are everywhere and are the cause of all the world's ills, while also simultaneously also being sort of ineffective and crazy, and yet you, the hero, the one righteous truther, are the last beacon of truth guarding against this. Pathetic. And your imagined bogeys and imagined crusade doesn't justify any of your abuse. You speak with such authority--earn and act like the airs you merely project. You haven't. You need integrity, you don't show that here. And you've got abusive disdain for different views, it doesn't make you strong, it makes you weak. It doesn't make you smart, it makes you the crazy one. You said my views are crazy. I'm pushing back. I'm saying you're crazy. I didn't want to say this, I wanted you to be different but you weren't. And I chose to say it, to expose the truth of what you did and said. I'm not gonna let that stuff slide and pretend I don't see what I see, I'm gonna call it out. That's what I'm doing. I treated you very good and was very fair--more than fair, I was super generous--and you just kept chanting your abusive claptrap, while retreating into a narrow view of the pretending what you know proscribes all that was, is and can be--while I discussed the unknown, which was the point of the question. You kept puffing up your weak points with pompous abusive rhetoric. And when I didn't play your game, you gave up. And now, once it's over, I'm saying what you did. Thank you for your time, and this opportunity to say it clearly.
deleted 5y ago
[deleted]
graderjs 5y ago
Hello eesmith, did you see my reply? I'm sorry I just wanted to thread it down the page, rather than branch it! :)