9 ms·

>Such ecosystems come with incredible costs. For instance, rust cannot >even compile itself on i386 at present time because it exhausts the >address space. Is

by akerro 9y ago

>Such ecosystems come with incredible costs. For instance, rust cannot >even compile itself on i386 at present time because it exhausts the >address space. Is cargo supported on i386 platforms? Also Rust complies itself, afaik there is no way to compile Rust/Cargo but to use previous version of it. If one of the past builds of Rust is backdoored, any version between then and now is backdoored, language is safe, environment... as safe as it was never compromised. OpenBSD compiles everywhere where C code works, Rust/Cargo works where it's supported and it will takes decades to catch-up on some architectures.

hood_syntax 9y ago
I'm pretty sure that attack you describe is mentioned in literature as essentially undefeatable. I really wish I could remember exactly what the name of it was; the gist is, there has to be a first compiler somewhere. If at any point in the chain, the compiler is infected with a self-propagating virus that hides itself in the byte code of the binary, it can ensure that the exploit is in every future version of the compiler. I may be remembering the details a bit wrong, but it was a good read.
- traverseda 9y ago
  You want the "trusting trust" keyword.
- skj 9y ago
  A talk by Ken Thompson, "trusting trust".
  - mcguire 9y ago
    Technically, his Turing award lecture.
- edwincheese 9y ago
  I think you are referring to “Reflections on Trusting Trust” by Ken Thompson. https://www.ece.cmu.edu/~ganger/712.fall02/papers/p761-thompson.pdf https://www.ece.cmu.edu/~ganger/712.fall02/papers/p761-thomp...
- unhammer 9y ago
  There is a possible defense: https://www.dwheeler.com/trusting-trust/ https://www.dwheeler.com/trusting-trust/
  - steveklabnik 9y ago
    ... and the hope is that https://github.com/thepowersgang/mrustc https://github.com/thepowersgang/mrustc will let us do this for Rust.
- zdw 9y ago
  You're looking for "Reflections on Trusting Trust" by Ken Thompson, one of the original co-authors of Unix: https://dl.acm.org/citation.cfm?id=358210 https://dl.acm.org/citation.cfm?id=358210
- nickpsecurity 9y ago
  It's not undefeatable. The guy who invented it in 1970's, Paul Karger, told people the concepts on how to defeat it right afterward. Their advice for building systems in a way to catch a lot of subversion was encoded in the first standards for information security. I included most of those methods in my security framework here: https://pastebin.com/y3PufJ0V https://pastebin.com/y3PufJ0V https://en.wikipedia.org/wiki/Trusted_Computer_System_Evaluation_Criteria https://en.wikipedia.org/wiki/Trusted_Computer_System_Evalua... One compiler made to the highest standard in development assurance is CompCert. http://compcert.inria.fr/ http://compcert.inria.fr/ It has specs of everything it does, proofs it does it in tool with minimalist checker, extracts to ML that can be compared against those specs in various ways (eg visually), can optionally be compiled with a mini-ML compiler or Scheme you homebrew yourself, and passed exhaustive testing by third party with only a few bugs in specs (not code). There's another using formal specs called KCC which could be ported to something bootstrappable like META II or Scheme. The other requirement from TCSEC was that source be supplied to customer to build with their onsite, trusted tools. I looked into even having compilers done in Perl since it's already widely deployed. David A. Wheeler made brilliant suggestion of either bash or awk. I have put tools for those and more on rain1's bootstrapping page. rain1 or someone there called the concept "Ubiquitous Implementations." Note we've focused on technical content, not presentation, on that one being busy folks. Looks rough. :) http://bootstrapping.miraheze.org/ http://bootstrapping.miraheze.org/ You also need repo security to protect that source with it either cryptographically sealed and/or sent over secure transport. Link below on repo security from David A. Wheeler. Quite a few forms of transport security now. https://www.dwheeler.com/essays/scm-security.html https://www.dwheeler.com/essays/scm-security.html After Thompson wrote on Karger's attack in 1980's, it took a life of its own among people that continue to mostly ignore the prior solutions. It's a problem absolutely solved to death starting with the person who discovered it in MULTICS Security Evaluation. Far as state-of-the-art, the current path of research is exploring how to integrate solutions for many languages, models, or levels of detail in one picture of a system with no abstraction gap attacks with proof of that for all inputs. That's a bit more complex but just an imperative language to assembly delivered and bootstrapped? Straight-forward but tedious, time-consuming work the first time it's done. :) Also, expensive if you buy CompCert which is only free for GPL stuff. Two of us are eyeballed CakeML's lowest-level languages as a cheat around that for verified bootstrapping. http://cakeml.org/ http://cakeml.org/ EDIT: Btw, all that is technical discussion and argument. For fun, you might enjoy the story "Coding Machines" which is about only coding-related story I started reading and couldn't put down. Probably took an hour to read. It covers discovery of a Karger-Thompson-style attack along with how people might respond mentally and in terms of solutions. Some other stuff in that one. http://www.teamten.com/lawrence/writings/coding-machines/ http://www.teamten.com/lawrence/writings/coding-machines/
jackmott 9y ago
It seems like a barrier for Rust in many of the low level places people would like to see it, is the inability to target all platforms. An official C backend would seem to solve that problem. The other barrier is compile times, which seems to be a major challenge.
steveklabnik 9y ago
Okay, there's a lot of misunderstandings here. Theo is right about some things, and wrong about some things. And people are misunderstanding what things he's right about. The things he's wrong about are very minor. Rust absolutely works on 32-bit platforms, though we often use the i686 target rather than an i386 one. Platform support list is here: https://forge.rust-lang.org/platform-support.html https://forge.rust-lang.org/platform-support.html Theo is talking about building rustc, not compiling most Rust programs. That's the first distinction that it seems like many people are missing. However, apparently the compilation process OOMs when building on a i386 box. I don't use those platforms, but I'd believe it. The Rust compiler is large. However, I thought (and looking at our CI, this seems to be true https://travis-ci.org/rust-lang/rust/jobs/311223817 https://travis-ci.org/rust-lang/rust/jobs/311223817) we do compile with an i686-unknown-linux-gnu host (for this build), so I dunno. Maybe it was a fluke, maybe I'm misunderstanding, I'm not sure. We often provide artifacts via cross-compiling, but this is unacceptable to OpenBSD. That's totally okay. They have good reasons for doing this.
- HelloNurse 9y ago
  If compilation of rustc runs out of memory it represent an upper limit on the complexity of actually viable Rust programs, and given how much software is larger than a compiler it is a discouraging performance level.
  - steveklabnik 9y ago
    Rustc is more than just a Rust program; we compile LLVM from scratch, for example. Is the OOM in the Rust code, or in the LLVM code, or in the final linking, or what? It's not clear. The compiler is one of the largest Rust programs that exist. Last I checked, it was three quarters of a million lines of Rust, but a quick loc shows 1.5 million lines of Rust, 2.3 million lines of C++, and 900,000 lines of C (again, mostly LLVM and jemalloc). Servo is also very large, and they don't report having OOMs, though I'm not sure if they build on 32-bit or just cross-compile.
    - ratmice 9y ago

by akerro 9y ago

hood_syntax 9y ago

I'm pretty sure that attack you describe is mentioned in literature as essentially undefeatable. I really wish I could remember exactly what the name of it was; the gist is, there has to be a first compiler somewhere. If at any point in the chain, the compiler is infected with a self-propagating virus that hides itself in the byte code of the binary, it can ensure that the exploit is in every future version of the compiler. I may be remembering the details a bit wrong, but it was a good read.

traverseda 9y ago
You want the "trusting trust" keyword.
skj 9y ago
A talk by Ken Thompson, "trusting trust".
- mcguire 9y ago
  Technically, his Turing award lecture.
edwincheese 9y ago
I think you are referring to “Reflections on Trusting Trust” by Ken Thompson. https://www.ece.cmu.edu/~ganger/712.fall02/papers/p761-thompson.pdf https://www.ece.cmu.edu/~ganger/712.fall02/papers/p761-thomp...
unhammer 9y ago
There is a possible defense: https://www.dwheeler.com/trusting-trust/ https://www.dwheeler.com/trusting-trust/
- steveklabnik 9y ago
  ... and the hope is that https://github.com/thepowersgang/mrustc https://github.com/thepowersgang/mrustc will let us do this for Rust.
zdw 9y ago
You're looking for "Reflections on Trusting Trust" by Ken Thompson, one of the original co-authors of Unix: https://dl.acm.org/citation.cfm?id=358210 https://dl.acm.org/citation.cfm?id=358210
nickpsecurity 9y ago
It's not undefeatable. The guy who invented it in 1970's, Paul Karger, told people the concepts on how to defeat it right afterward. Their advice for building systems in a way to catch a lot of subversion was encoded in the first standards for information security. I included most of those methods in my security framework here: https://pastebin.com/y3PufJ0V https://pastebin.com/y3PufJ0V https://en.wikipedia.org/wiki/Trusted_Computer_System_Evaluation_Criteria https://en.wikipedia.org/wiki/Trusted_Computer_System_Evalua... One compiler made to the highest standard in development assurance is CompCert. http://compcert.inria.fr/ http://compcert.inria.fr/ It has specs of everything it does, proofs it does it in tool with minimalist checker, extracts to ML that can be compared against those specs in various ways (eg visually), can optionally be compiled with a mini-ML compiler or Scheme you homebrew yourself, and passed exhaustive testing by third party with only a few bugs in specs (not code). There's another using formal specs called KCC which could be ported to something bootstrappable like META II or Scheme. The other requirement from TCSEC was that source be supplied to customer to build with their onsite, trusted tools. I looked into even having compilers done in Perl since it's already widely deployed. David A. Wheeler made brilliant suggestion of either bash or awk. I have put tools for those and more on rain1's bootstrapping page. rain1 or someone there called the concept "Ubiquitous Implementations." Note we've focused on technical content, not presentation, on that one being busy folks. Looks rough. :) http://bootstrapping.miraheze.org/ http://bootstrapping.miraheze.org/ You also need repo security to protect that source with it either cryptographically sealed and/or sent over secure transport. Link below on repo security from David A. Wheeler. Quite a few forms of transport security now. https://www.dwheeler.com/essays/scm-security.html https://www.dwheeler.com/essays/scm-security.html After Thompson wrote on Karger's attack in 1980's, it took a life of its own among people that continue to mostly ignore the prior solutions. It's a problem absolutely solved to death starting with the person who discovered it in MULTICS Security Evaluation. Far as state-of-the-art, the current path of research is exploring how to integrate solutions for many languages, models, or levels of detail in one picture of a system with no abstraction gap attacks with proof of that for all inputs. That's a bit more complex but just an imperative language to assembly delivered and bootstrapped? Straight-forward but tedious, time-consuming work the first time it's done. :) Also, expensive if you buy CompCert which is only free for GPL stuff. Two of us are eyeballed CakeML's lowest-level languages as a cheat around that for verified bootstrapping. http://cakeml.org/ http://cakeml.org/ EDIT: Btw, all that is technical discussion and argument. For fun, you might enjoy the story "Coding Machines" which is about only coding-related story I started reading and couldn't put down. Probably took an hour to read. It covers discovery of a Karger-Thompson-style attack along with how people might respond mentally and in terms of solutions. Some other stuff in that one. http://www.teamten.com/lawrence/writings/coding-machines/ http://www.teamten.com/lawrence/writings/coding-machines/

jackmott 9y ago

It seems like a barrier for Rust in many of the low level places people would like to see it, is the inability to target all platforms. An official C backend would seem to solve that problem. The other barrier is compile times, which seems to be a major challenge.

steveklabnik 9y ago

Okay, there's a lot of misunderstandings here. Theo is right about some things, and wrong about some things. And people are misunderstanding what things he's right about. The things he's wrong about are very minor. Rust absolutely works on 32-bit platforms, though we often use the i686 target rather than an i386 one. Platform support list is here: https://forge.rust-lang.org/platform-support.html https://forge.rust-lang.org/platform-support.html Theo is talking about building rustc, not compiling most Rust programs. That's the first distinction that it seems like many people are missing. However, apparently the compilation process OOMs when building on a i386 box. I don't use those platforms, but I'd believe it. The Rust compiler is large. However, I thought (and looking at our CI, this seems to be true https://travis-ci.org/rust-lang/rust/jobs/311223817 https://travis-ci.org/rust-lang/rust/jobs/311223817) we do compile with an i686-unknown-linux-gnu host (for this build), so I dunno. Maybe it was a fluke, maybe I'm misunderstanding, I'm not sure. We often provide artifacts via cross-compiling, but this is unacceptable to OpenBSD. That's totally okay. They have good reasons for doing this.

HelloNurse 9y ago
If compilation of rustc runs out of memory it represent an upper limit on the complexity of actually viable Rust programs, and given how much software is larger than a compiler it is a discouraging performance level.
- steveklabnik 9y ago
  Rustc is more than just a Rust program; we compile LLVM from scratch, for example. Is the OOM in the Rust code, or in the LLVM code, or in the final linking, or what? It's not clear. The compiler is one of the largest Rust programs that exist. Last I checked, it was three quarters of a million lines of Rust, but a quick loc shows 1.5 million lines of Rust, 2.3 million lines of C++, and 900,000 lines of C (again, mostly LLVM and jemalloc). Servo is also very large, and they don't report having OOMs, though I'm not sure if they build on 32-bit or just cross-compile.
  - ratmice 9y ago