7 ms·

“Reflections on Trusting Trust” annotated

41 points by joaobatalha 11y ago

slfnflctd 11y ago
Great read, it really does come down to ethical vs. unethical behavior at the end of it all. Someone smart doing something destructive is still behaving badly, and giving them undeserved respect (beyond their technical ability) simply for destroying things 'more intelligently' is foolish. Regardless, I'm left feeling yet again that I can't fully trust anything. Good thing I don't need to these days; I fear for those who do.
- TeMPOraL 11y ago
  > Good thing I don't need to these days; I fear for those who do Oh but you do, you trust countless of people doing their jobs right so that you can have electricity, clean water, safe food and peaceful streets. Civilization exists because of trust. And one has to realize that trustless systems come with a cost - they have stupendous overhead. Consider all the layers of bureaucracy companies (or the law) employ to protect themselves from malicious actors. Or, consider Bitcoin. It's good to have, and maintain, trust-based systems because they can get the job done much, much more efficiently.
nikisweeting 11y ago
Really great paper, glad you guys posted it!
pvg 11y ago
previously and previously and previously. https://hn.algolia.com/?query=The%20Ken%20Thompson%20Hack&sort=byDate&dateRange=all&type=story&storyText=false&prefix&page=0 https://hn.algolia.com/?query=The%20Ken%20Thompson%20Hack&so... https://hn.algolia.com/?query=reflections%20on%20trusting%20trust&sort=byDate&dateRange=all&type=story&storyText=false&prefix&page=0 https://hn.algolia.com/?query=reflections%20on%20trusting%20...
- burkaman 11y ago
  This is an annotated version of the paper, it's reasonably distinct from posting the paper on its own. Also, it looks like it's been close to a year since a post of the paper was seen by anyone. There's nothing wrong with a repost after that long.
michaelfeathers 11y ago
I've always regarded that paper as the 'Godel's Incompleteness Theorem' of Security. I'm not saying that the security field had a 'Hilbert's Program' before then, but it certainly couldn't have one afterward.
- agentxyz 11y ago
  Or how about Gödel’s Lost Theorem? https://medium.com/@yanazendo/godels-lost-theorem-ce8ee4af20b9 https://medium.com/@yanazendo/godels-lost-theorem-ce8ee4af20...
  - cwzwarich 11y ago
    That link doesn't actually state a theorem. What is it?
    - jessaustin 11y ago
      Apparently we've lost it...
- munin 11y ago
  yeah, except there's a solution to "trusting trust" : http://www.dwheeler.com/trusting-trust/ http://www.dwheeler.com/trusting-trust/
  - delinka 11y ago
    Dr. Wheeler's research and resulting paper are excellent work. It certainly gets us thinking in the right direction in an attempt to solve the problem, but I don't know that I'd call his approach a tangible solution. You still need a trusted compiler. And from whence can one obtain a trusted compiler and guarantee that it has not been compromised? Also, no two compilers (GCC vs Intel, for example) would produce the same bit-for-bit executable code. Perhaps we only lack the tools to implement his solution.
agentxyz 11y ago
Personally, I find epic massive Byzantine attacks far more interesting. For example, imagine if a gangster were so suave that he could walk into any ghetto and immediately become the local mob boss. These things are computationally possible. In fact, these attacks happen in nature. I find the Grey Goo Problem far more threatening. https://medium.com/@yanazendo/the-grey-goo-problem-bd495813c6ed https://medium.com/@yanazendo/the-grey-goo-problem-bd495813c...
- insanebits 11y ago
  Yes it's quite interesting to read, but yet in my oppinion they're far from becoming reality due to technical reasons of building AI or self replicating bots improving themselves. As for backdoored code is reality right now. IIRC intel compiler used to optimize code better for their own processors so it would score better benchmarks compared to AMD which is trojan IMO. Not to start talking about NSA conspiracy theories.
RankingMember 11y ago
Interesting read. Aside: As a compulsive text-highlighter, this site drove me nuts.
w8rbt 11y ago
At some point, everything comes down to trust.
rui314 11y ago
This self-referencing naturally occurs if you are writing a self-hosting compiler. I have read the Ken's paper, but when I was writing my C compiler in C (https://github.com/rui314/8cc/blob/master/lex.c#L306 https://github.com/rui314/8cc/blob/master/lex.c#L306), I was not aware that this was what Ken wrote in the paper. It was funny to notice that long after I wrote the code.

41 points by joaobatalha 11y ago

Great read, it really does come down to ethical vs. unethical behavior at the end of it all. Someone smart doing something destructive is still behaving badly, and giving them undeserved respect (beyond their technical ability) simply for destroying things 'more intelligently' is foolish. Regardless, I'm left feeling yet again that I can't fully trust anything. Good thing I don't need to these days; I fear for those who do.

TeMPOraL 11y ago
> Good thing I don't need to these days; I fear for those who do Oh but you do, you trust countless of people doing their jobs right so that you can have electricity, clean water, safe food and peaceful streets. Civilization exists because of trust. And one has to realize that trustless systems come with a cost - they have stupendous overhead. Consider all the layers of bureaucracy companies (or the law) employ to protect themselves from malicious actors. Or, consider Bitcoin. It's good to have, and maintain, trust-based systems because they can get the job done much, much more efficiently.

nikisweeting 11y ago

Really great paper, glad you guys posted it!

pvg 11y ago

previously and previously and previously. https://hn.algolia.com/?query=The%20Ken%20Thompson%20Hack&sort=byDate&dateRange=all&type=story&storyText=false&prefix&page=0 https://hn.algolia.com/?query=The%20Ken%20Thompson%20Hack&so... https://hn.algolia.com/?query=reflections%20on%20trusting%20trust&sort=byDate&dateRange=all&type=story&storyText=false&prefix&page=0 https://hn.algolia.com/?query=reflections%20on%20trusting%20...

burkaman 11y ago
This is an annotated version of the paper, it's reasonably distinct from posting the paper on its own. Also, it looks like it's been close to a year since a post of the paper was seen by anyone. There's nothing wrong with a repost after that long.

michaelfeathers 11y ago

I've always regarded that paper as the 'Godel's Incompleteness Theorem' of Security. I'm not saying that the security field had a 'Hilbert's Program' before then, but it certainly couldn't have one afterward.

agentxyz 11y ago
Or how about Gödel’s Lost Theorem? https://medium.com/@yanazendo/godels-lost-theorem-ce8ee4af20b9 https://medium.com/@yanazendo/godels-lost-theorem-ce8ee4af20...
- cwzwarich 11y ago
  That link doesn't actually state a theorem. What is it?
  - jessaustin 11y ago
    Apparently we've lost it...
munin 11y ago
yeah, except there's a solution to "trusting trust" : http://www.dwheeler.com/trusting-trust/ http://www.dwheeler.com/trusting-trust/
- delinka 11y ago
  Dr. Wheeler's research and resulting paper are excellent work. It certainly gets us thinking in the right direction in an attempt to solve the problem, but I don't know that I'd call his approach a tangible solution. You still need a trusted compiler. And from whence can one obtain a trusted compiler and guarantee that it has not been compromised? Also, no two compilers (GCC vs Intel, for example) would produce the same bit-for-bit executable code. Perhaps we only lack the tools to implement his solution.

agentxyz 11y ago

Personally, I find epic massive Byzantine attacks far more interesting. For example, imagine if a gangster were so suave that he could walk into any ghetto and immediately become the local mob boss. These things are computationally possible. In fact, these attacks happen in nature. I find the Grey Goo Problem far more threatening. https://medium.com/@yanazendo/the-grey-goo-problem-bd495813c6ed https://medium.com/@yanazendo/the-grey-goo-problem-bd495813c...

insanebits 11y ago
Yes it's quite interesting to read, but yet in my oppinion they're far from becoming reality due to technical reasons of building AI or self replicating bots improving themselves. As for backdoored code is reality right now. IIRC intel compiler used to optimize code better for their own processors so it would score better benchmarks compared to AMD which is trojan IMO. Not to start talking about NSA conspiracy theories.

RankingMember 11y ago

Interesting read. Aside: As a compulsive text-highlighter, this site drove me nuts.

w8rbt 11y ago

At some point, everything comes down to trust.

rui314 11y ago

This self-referencing naturally occurs if you are writing a self-hosting compiler. I have read the Ken's paper, but when I was writing my C compiler in C (https://github.com/rui314/8cc/blob/master/lex.c#L306 https://github.com/rui314/8cc/blob/master/lex.c#L306), I was not aware that this was what Ken wrote in the paper. It was funny to notice that long after I wrote the code.