18 ms·
Ken Thompson really did launch his "trusting trust" trojan attack in real life
- foobarian 4y agoHeh, as far back as then Ken T. already knew the score: ) writing to news just causes more ) misunderstandings in the future. there ) is no way to win. Pretty amazing insight!
- xmprt 4y agoI don't think it's some super novel insight by him. The news has existed for many centuries and at least to some extent, it has created misunderstandings as long as it has existed.
- foobarian 4y agoSee, exactly! He meant writing to newsgroups, which were relatively new at that time. Or did he? :-)
- segfaultbuserr 4y agoOne can find precursors of nearly every aspect of the modern social media back in newsgroups from the Usenet era during the 1980s and 1990s, including cat memes. Unix developer Rob Pike even invented role-play trolling in the 1980s on Usenet (mostly manually, but an automated bot was also tried). Back then it was a harmless hoax, and today on the modern web it's now causing massive troubles. [0] Old-timers like Ken Thompson clearly have understood the nature of a social network since a long time ago. [0] https://en.wikipedia.org/wiki/Mark_V._Shaney#History https://en.wikipedia.org/wiki/Mark_V._Shaney#History
- aliqot 4y agoOff-topic but, I'm noticing a lot of anime waifu and furry-type illustrations on tech blogs lately on HN. Can someone cooler/younger tell me: Is this the hand-off to the new generation, or is there a meta-meme I missed?
- junon 4y agoThe former.
- aliqot 4y agoThanks, take care.
- some_furry 4y agoI mean, sorta. I'm in my mid 30's. I started blogging about cryptography and security under my furry handle (and with blog posts adorned with furry art) at the start of the pandemic. It gave me something to do that was both productive and fun.
- LanternLight83 4y agoJust wanted to say that I love your blog!!!
- some_furry 4y agoThanks <3
- meibo 4y agoIndividualism is big for those generations, and identifying yourself with your favorite anime girl/pop culture element or the character you designed with the elements you like is a way to differentiate yourself from others. The furry/anime communities also tend to be pretty technically inclined, so I imagine there's just some natural overlap.
- bombcar 4y agoIndividualism is big ... so be like everyone else and have an anime avatar. The loop continues.
- an1sotropy 4y agoIt is great to see this from the source (or close to it). Some searches on the text of Thompson's message also led to these pre-existing sources (not linked from niconiconi's blog) of a post by Jonathan Thornburg: https://www.mail-archive.com/cryptography-digest@senator-bedfellow.mit.edu/msg02776.html https://www.mail-archive.com/cryptography-digest@senator-bed... https://groups.google.com/g/sci.crypt/c/PybcCHi9u6s/m/b-7U1y9QBZMJ?pli=1 https://groups.google.com/g/sci.crypt/c/PybcCHi9u6s/m/b-7U1y... btw is there any public archive of ~old usenet (say, through 1999)? I was trying to remember things I learned from on alt.2600 but groups.google.com says it's "banned".
- obi1kenobi 4y agoWhen you say "searches" are you referring to regular web search (e.g. Google) or something more specialized? Just wondering if there's a good way to programmatically search "old" records like these.
- an1sotropy 4y agosorry didn't meant to be obscure - it was just google searches, but I found that if I used quotes google was less useful than if I searched for a longer string without quotes (I think others on HN have noted the quality of google searches seems to be decreasing). In my case I searched for fyi: the self reproducing cpp was installed on OUR machine and we enticed the unix support group but without quotes. I know it's quixotic but I kind of wish altavista was still working (now yahoo owns the domain name).
- deleted 4y ago[deleted]
- chris_st 4y agoSomeone (Uunet, maybe?) put out a bunch of usenet CDs back in the 90's... don't know how far they went back (or which groups they had. IIRC, it was pretty much everything except binary groups). It'd be challenging to find them.
- 4y ago
- naillo 4y agoThat anime girl really makes it hard to hide that you're doing work unrelated things in the office.
- hbn 4y agoIf you squish the browser window enough it'll go into a mobile view where she jumps to the bottom
- bentcorner 4y agoAh - thank you. The comments here make a lot more sense now. Anyone care to explain why it doesn't show up anywhere for desktop users? Html has this: <footer id="footer"> <img id="footer-image" src="/img/niconiconi.png" width="0%" /> </footer>
- hbn 4y agoOn desktop you see the image as a background on the <body> @media (min-width: 770px) body { ... background-image: url(/img/niconiconi.png); ... } And on mobile you see that footer @media (min-width: 770px) #footer-image { width: 0%; }
- SllX 4y agoIt’s okay. She’s got a copy of “The C Programming Language”. We’re in the clear!
- Attrecomet 4y agoAnd whatever the site did disabled ublock's "block element". Luckily umatrix still works and I was able to block images from the site
- AlwaysRock 4y ago> That anime girl really makes it hard to hide that you're doing work unrelated things in the office. What do you mean?
- 4y ago
- dolmen 4y agoBut can we trust this post? How can we be sure that Ken (working for Google) didn't infect the toolchain used for Chrome to propagate that legend?
- selectnull 4y agoThat's easy: try it with Firefox.
- wyldfire 4y agoPretty sure Firefox uses Chrome as its toolchain these days. Nice try son, but it's turtles all the way down. EDIT: just kidding - no "chrome-headless-c++ -c firefox.o firefox.cpp" (yet).
- guhidalg 4y agoI did check this to make sure I was still living in a sane universe, Firefox doesn't use Blink https://en.wikipedia.org/wiki/Comparison_of_browser_engines https://en.wikipedia.org/wiki/Comparison_of_browser_engines
- tristor 4y ago> Pretty sure Firefox uses Chrome as its toolchain these days. It does not. There is no shared code between Firefox and Chrome. They use completely different rendering engines with independent histories (Chrome uses Blink originated from WebKit originated from KHTML, Firefox uses Gecko originated from Netscape originated from Mosaic). The only shared component is that Firefox utilizes public APIs for Google SafeBrowsing. Disclaimer: ex-Mozillian
- wyldfire 4y agoOh sorry. Chrome is not a toolchain in mostly any sense. Except for its extraordinary flexibility as browsers are in general able to execute code. So I thought it was obviously a joke that one would use a browser to compiler another browser. The "on trusting trust" attack regards using your compiler as a mechanism to infect compiled executables -- including compilers themselves, and their generated code. I didn't mean to suggest that the two browsers shared any code.
- denton-scratch 4y agoThanks for digging-up and posting that; an interesting historical artifact.
- bombcar 4y agoThis goes to show the importance of multiple independent toolchains, and multiple variations on debuggers. The more there are, the harder it would be to successfully execute this attack for any length of time. However, it could and certainly has been done in specific targeted cases I bet.
- noobermin 4y agoPlease tell this to all those people who want gcc to vaporize and have all of us use clang. Competition is good.
- golemotron 4y agoFunny how the article says Ken Thompson popularized the attack. There's good reason to say he invented it.
- ProjectMoonShot 4y agoKarger and Schell in 1974 conceptualised the idea of trap doors built into compilers in their Multics Security Review. Although Ken Thompson is responsible for popularising the idea through his Turing award speech in 1984. Edit: Ken Thompson mentions the paper in the acknowledgements of his Turing award speech. """ Acknowledgment. I first read of the possibility of such a Trojan horse in an Air Force critique [4] of the security of an early implementation of Multics. I cannot find a more specific reference to this document. I would appreciate it if anyone who can supply this reference would let me know. """
- golemotron 4y agoThanks.
- Thoreandan 4y agoThanks for the reference! Found the 1974 paper https://web.archive.org/web/20030410020522/https://www.acsac.org/2002/papers/classic-multics-orig.pdf https://web.archive.org/web/20030410020522/https://www.acsac... and the "Thirty Years Later: Lessons from the Multics Security Evaluation" followup https://web.archive.org/web/20030410095057/https://www.acsac.org/2002/papers/classic-multics.pdf https://web.archive.org/web/20030410095057/https://www.acsac... Always neat to have lost sources show up eventually. Librarians rule.
- Javipok 4y ago
- lisper 4y agoTo me, the historical reconstruction aspect of this is at least as interesting as the attack. One of my more obscure hobbies is studying the history of ancient texts, and it is fascinating to watch the process of losing primary historical sources play out in front of my eyes: > However, in 1995, Usenet poster Jay Ashworth, citing personal communications with Ken Thompson, provided strong evidence of the existence of a real-world experiment of this attack. Unfortunately, the full Usenet message is missing on the web. There are only quoted snippets of this Usenet post circulated around various blogs, reducing its authenticity. > In 2021, I’ve rediscovered the full Usenet message after a search effort in multiple Usenet archives. My success was partial - it was still a repost by someone else, and I was unable to find the original message. However, this repost contains the full Usenet message, including complete headers and message body, with the poster name and its Message-ID, establishing the authenticity of the post beyond reasonable doubts.
- routerl 4y agoThe data archaeologists of the future are, right now, acquiring and developing what will become the skills of their profession. Very cool to watch, indeed.
- jll29 4y agoIt will be hard for future (>700 years away from now) historians to discover a lot about what we thought today: writing personal and professional letters (e.g. "letters to the editor" in learned journals, letters to an uncle living in a different city) and diaries is happening less and less compared to the last 200-300 years. Perhaps we should go ahead and have a few hundred thousand emails printed with a special lasting ink on velum to pass it on to our successors ("Codex Electronicus"). On reflection, my own inbox is perhaps rather too nerdy - it would introduce a strong selection bias to posterity's view about us.
- Godel_unicode 4y ago> is happening less and less Do you have any evidence for this? Even assuming it’s true, doesn’t the volume of podcast/YouTube/email/facebook/HN posts more than make up for it?
- noobermin 4y agoTrusting trust is so old that this probably has been discussed before, but isn't it possible to "break it" by either disassembly or just looking at the elf with a hex editor? I know you can theoretically hack the disassembler too if you'd like, but after some point it becomes onerous.
- Sebb767 4y ago> but after some point it becomes onerous. That's exactly the point - a sufficiently deep supply chain attack can avoid detection just because no one bothers to look that deep.
- naniwaduni 4y agoOn the other hand, it is generally considered pretty hard to predict arbitrary changes in the future.
- enriquto 4y agoThere is this 2009 PhD thesis, and associated articles, where they explain how to "counter" trusting trust by using a set of independent compilers (even assuming that each compiler may be infected): https://dwheeler.com/trusting-trust/ https://dwheeler.com/trusting-trust/ This is an automatic process: you compile each compiler with the others a few times and compare the outputs. At the end it gives a criterion to decide which compilers contain trojan horses.
- selfhoster11 4y agoI always had issues with this particular counter, because it assumes that you cannot create a sufficiently good back door-creating AI/heuristic machine that can also fit in the unused spaces in our systems and binaries without being noticed. That’s a big ‘if’, looking ahead into the deep future especially that our knowledge of autonomous agents and storage keeps growing.
- dwheeler 4y ago
- mosburger 4y agoI know this probably falls into the category of red-baiting hysteria, but sometimes I've wondered (not seriously, in more of a spy-vs-spy, campy, tinfoil hat way) if the Kotlin programming language is a secret plot by the Russian government to embed backdoors in everything compiled with it. How many people have done a deep inspection of the compiled product? (Insert balloon boy meme of the conspiracy theory guy at his bulletin board) Of course, the Russians would probably be justified in wondering the same thing about a programming language created in the United States.
- Vizdun 4y agois the only reason for this wondering the fact Kotlin was made by czech developers?
- mosburger 4y agoI thought a lot of it was developed at JetBrains' (now shuttered) St. Petersburg R&D site? Could be wrong, though.
- masswerk 4y ago> Of course, the Russians would probably be justified in wondering the same thing about a programming language created in the United States. Now I want to have a closer look at S/360 and S/370 system tapes… ;-)
- deleted 4y ago[deleted]
- nwsm 4y ago> Keywords: horse, mouth These message keywords cracked me up.
- whatthem 4y ago
- dang 4y ago"Please don't pick the most provocative thing in an article or post to complain about in the thread. Find something interesting to respond to instead." "Please don't complain about tangential annoyances—things like article or website formats, name collisions, or back-button breakage. They're too common to be interesting." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- PoignardAzur 4y ago> "Please don't complain about tangential annoyances—things like article or website formats, name collisions, or back-button breakage. They're too common to be interesting." That one happens a lot on HN. I'm not sure I'd want it to happen less.
- avgcorrection 4y agoI agree with this sentiment. This is very inappropriate considering how children can consume C as a gateway language to memory unsafety.
- tristor 4y agoThis page is getting flagged as malware by OpenDNS, but here's an archive link: https://archive.ph/UjaMd https://archive.ph/UjaMd
- trustingtrust 4y agoOne of my favourite papers for sure. It’s mind boggling the way it was presented.
- GuB-42 4y agoThe attack was successfully performed in the wild by the virus "Win32/Induc.A" The virus looks for a Delphi installation, modifies the SysConst.pas file, which is the source code of a part of the standard library and compiles it. After that, every program compiled by that Delphi installation will contain the virus. The virus does nothing else, it is therefore harmless if you don't have Delphi installed. It resulted in many software vendors releasing infected executables without realizing it, sometimes claiming false positives. After all, the executable was not tampered with, the compiler was.
- segfaultbuserr 4y agoIn 2015, a malicious copy of Xcode, XcodeGhost, also performed a similar attack and infected iOS apps from a dozen of software companies in China. Globally, 4000 apps were found to be affected. It was not a true Thompson Trojan, as it doesn't infect development tools themselves, but it did show toolchain poisoning can indeed cause substantial damages. https://en.wikipedia.org/wiki/XcodeGhost https://en.wikipedia.org/wiki/XcodeGhost
- cduzz 4y agoAh, but was a version of Xcode compiled with this version, and did that child version of Xcode also have the trojan code?
- segfaultbuserr 4y agoVery unlikely, and I don't know any Xcode that runs on iOS on mobile devices, so I said it was not a true Thompson Trojan.
- scoot 4y agoWas it "a dozen software companies", or "dozens of software companies"? The current wording says neither.
- Joker_vD 4y agoI personally was infected by it when I was studying at university! It came in when QIP updated... and I did have Delphi 5.0 installed at that time. Quick googling tells me this happened in August, 2009... which was 13 years ago. Quomodo fugit tempus!
- deleted 4y ago[deleted]
- andyjansson 4y agoAlso see David A. Wheeler's work on countering trojan horse attacks: https://dwheeler.com/trusting-trust/ https://dwheeler.com/trusting-trust/
- segfaultbuserr 4y agoTo me, the most interesting part of Wheeler's work is formal verification. As an extra argument, he converted his verbal arguments into a set of logical statements, and then used a theorem prover to show the DDC argument is flawless (within its assumptions).
- aaaaaaaaaaab 4y agoWhat if the theorem prover was compiled with an infected compiler?
- segfaultbuserr 4y agoI remember seeing a post on MathOverflow, the OP asked about the consequences of using malicious code to fool a theorem prover to certify lies and falsehoods. It's a valid question and has deep philosophical implications. Unfortunately, mathematicians are not tech workers, so they were not impressed, and closed the question as off-topic. I personally think the main reason resposible for the lack of enthusiasm from mathematicians is that formal methods are rarely used in our society, and mathematicians in general (with the exception of logicians) also do not really value formal axiomatic systems as the something especially important for setting a standard of truth. If formal methods are used in decision and policymaking in the far future, the picture will be different. Nevertheless, right now, malicious proofs are just a hypothetical thought experiment. https://mathoverflow.net/questions/63816/consequences-of-technically-proving-anything-in-coq-on-at-least-linux-exploiti https://mathoverflow.net/questions/63816/consequences-of-tec...
- dwheeler 4y agoI did think of that :-) ... and I explained how I countered that in the paper and during my public defense. First, the proofs were verified by a separate prover that was itself independently formally verified. Second, the proofs were manually verified by multiple people. Once you know how to read first order logic notation (which is easier to learn than most programming languages), it's not hard to verify the steps by hand. The paper walks through the key parts. For more details see the paper :-).
- Izkata 4y agoShort story about an AI keeping itself hidden through this attack: https://www.teamten.com/lawrence/writings/coding-machines/ https://www.teamten.com/lawrence/writings/coding-machines/
- Jabrov 4y agoDamn this story was soo good!
- thret 4y agoThe only unbelievable part of this short story is that they wrote a compiler in assembly! Very enjoyable. Reminds me of Rob Miles' AI tasked with acquiring stamps.
- danuker 4y agoReminds me of https://www.decisionproblem.com/paperclips/index2.html https://www.decisionproblem.com/paperclips/index2.html
- albrewer 4y agoI think this is my favorite incremental game of all time. I thought it wasn't really possible to tell a story through gameplay in a game like this but this one and Kittens game[0] have shown me it's possible. [0]: https://kittensgame.com/web/ https://kittensgame.com/web/
- deleted 4y ago[deleted]
- kingforaday 4y agoThe technical ingenuity is remarkable for the time period but I love how it still required social engineering to execute: ) we enticed the "unix support group" ) (precursor to usl) to pick it up ) from us by advertising some ) non-backward compatible feature."
- k0stas 4y agoI got obsessed with this paper recently, to the point where I have read most of the "Unknown Air Force Document" that Thompson references with giving him the idea of Trojan horse. The document was later identified and is declassified and publicly available [1]. > If one reads the original paper, one only finds a description of this attack as a thought experiment, leading one to conclude that any claim of a real-world attack by Thompson was an urban myth due to exaggeration. This is true although Thompson gives some tantalizing hints in the paper. In the introduction, he writes " I would like to present to you the cutest program I ever wrote." So he definitely wrote it and at least played around with it. Later on in the "Moral" section, he writes "The moral is obvious. You can't trust code that you did not totally create yourself. (Especially code from companies that employ people like me.)" This appears to be an admission but not quite strong or direct enough to validate he implemented and used the Trojan horse so it is great to read this post. [1] https://csrc.nist.gov/csrc/media/publications/conference-paper/1998/10/08/proceedings-of-the-21st-nissc-1998/documents/early-cs-papers/karg74.pdf https://csrc.nist.gov/csrc/media/publications/conference-pap...
- an1sotropy 4y agoThanks so much for sharing the older MULTICS report; this is fascinating stuff.
- XorNot 4y agoThis is a good moment to note that http://bootstrappable.org/ http://bootstrappable.org/ exists and is one of the low level defenses OSS can provide against this problem. With the minimal set of binary blobs that can be audited we can reasonably reconstruct whole toolchains from scratch.
- the_clarence 4y agoKen Thompson really did something great in there
- deleted 4y ago[deleted]
- pabs3 4y agoWe really need bootstrappable and reproducible builds to be more popular: https://bootstrappable.org/ https://bootstrappable.org/ https://reproducible-builds.org/ https://reproducible-builds.org/
- moring 4y agoCould you elaborate? I can't find anything there that would prevent a Thompson-style backdoor.
- pabs3 4y agoStarting with 512 bytes of machine code plus source and building an entire Linux distro from scratch without any existing binaries means that it is exceedingly unlikely that such a backdoor would be possible.
- d110af5ccf 4y agoWell it shifts the target a bit. Instead of infecting the compiler you'd try to infect one of the core OS components (presumably in the kernel) and have it detect when those were being compiled and insert itself. Probably also infect any compilers you detect while you're at it as well. This would probably be a bit more difficult to pull off. If someone did pull it off, fully reproducible builds ought to make it readily detectable at least in the absence of some extreme rootkit contortions.
- pabs3 4y agoThere are no Linux/other kernel binaries involved in the bootstrappable builds scenario either, only the 512-byte bootstrap seed of machine code, which is written as commented hex, which you then input manually and run. In theory that bootstrap seed could be backdoored, given the size that would be unlikely and auditing it should be feasible. Reproducible builds wouldn't help here, since all binaries would be backdoored equally. They only help with situations where one build machine is compromised but another one isn't.
- koalala 4y agoDo any mathematicians around here know if this is a special case of Gödel's incompleteness theorem [1]? [1] https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_theorems https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_...
- maweki 4y agoI'd say yes insofar, that it's clearly a case of Rice's theorem. Whether such a backdoor is contained in a program is a non-trivial property and therefore generally undecidable. All the other theorems (Gödel, Church, Tarski, Turing) are basically the same if you squint your eyes hard enough.