4 ms·

This seems like a really neat idea! I think it would be cool to use plain source code to completely bootstrap a system. But, at the same time, this sounds like

This seems like a really neat idea! I think it would be cool to use plain source code to completely bootstrap a system. But, at the same time, this sounds like a maintenance nightmare! Maintaining older or smaller versions of software just to avoid self-hosting is bound to hit all kinds of bugs - big and small. At the same time, I'm not convinced this completely solves the problem. Maybe you can trust your C compiler now, but can you trust your Bash interpreter, or your sed command? What stops them from injecting things into your C compiler while it's building. At the end of the day, you have to trust some software, why not include your C compiler in this list? Anyway, I am much more concerned about much higher places in free software bootstrapping. Lots of software has switched from Autotools to Meson without thinking about how we will bootstrap this stuff. Meson requires you to have Ninja to work, but how can I bootstrap Ninja without Python? Autotools is admittedly bad, but at least you only needed bash & make for it to work.

paulie_a 8y ago
I can't find a reference to it but there was an intentional security bug introduced to a c compiler that would pass through to the compiled code.
- fanf2 8y ago
  Reflections on trusting trust, Ken Thompson’s Turing award lecture https://www.archive.ece.cmu.edu/~ganger/712.fall02/papers/p761-thompson.pdf https://www.archive.ece.cmu.edu/~ganger/712.fall02/papers/p7...
avhon1 8y ago
> Maintaining older or smaller versions of software just to avoid self-hosting is bound to hit all kinds of bugs - big and small. Unfortunately, any compiler that can compile current versions of GCC would probably be enormous, and difficult to trust directly. C++ is a complicated language to compile. As long as the chosen versions of TCC and GCC 1. are free of bugs that affect compiling the next-more-sophisticated compiler, and 2. can be trusted, then having them as intermediate compilers isn't too terrible. Bugs in them that affect compiling the next-more-sophisticated compiler can be written and committed now, and will remain that way forever. The only place new bugs can be encountered is if the current version of GCC changes so that it can't be compiled by the last-frozen version of GCC. There are two ways to fix that: 1. Patch the last-frozen version of GCC so that it can still compile the latest GCC, or 2. add another intermediate version of GCC to the toolchain, since the last-frozen GCC could compile every version of GCC up to the most current, and the most current GCC must be compilable by at least one earlier version of GCC. This means that the compiler chain can potentially grow without bound, but it would do so slowly, and only by adding more compiler code onto the end of already-trusted compiler code. A long chain of obsolete-but-trusted compilers is not ideal, but should be a working and stable source of trust.
janneke 8y ago
> This seems like a really neat idea! I think it would be cool to use > plain source code to completely bootstrap a system. Technically that's what we are working towards. Idealogically, we are working towards moving awarenes in our communities from `neat idea' to `unbelievable that in 2018 we still used and trusted computers that had no bootstrappable audit trail!' > But, at the same time, this sounds like a maintenance nightmare! > Maintaining older or smaller versions of software just to avoid > self-hosting is bound to hit all kinds of bugs - big and small. You are not suggesting that we stop our efforts because it will require work and may contain bugs, rigth? ;-) Until now, this has been the effort of a very small team. The choice for tcc-0.9.26 and gcc-2.95.3 was a pragmatic one. We hope that when awareness of bootstrappable software rises we can make some better choices. I would love for the TinyCC and GNU GCC developers (any software developers, really) to take the lead in creating their own bootstrappable stories; we just showed it can be done. > At the same time, I'm not convinced this completely solves the > problem. Maybe you can trust your C compiler now, but can you trust > your Bash interpreter, or your sed command? Right, doing only part of the work does not solve the problem. The solution will be built from a number of such steps. One of those steps may involve hardware; I'm sure it's a long path. > What stops them from injecting things into your C compiler while > it's building. At the end of the day, you have to trust some > software, why not include your C compiler in this list? Some of us have decided that is just not good enough, that we would like our softwares not only reproducible but also bootstrappable, and are we working towards that. There is Stage0 and M2-Planet to take care of what's below Mes. We have also started work on a Bash replacement in Scheme (using GNU Guile initially) that comes with a minimal implementation of coreutils, grep, sed, tar. I have managed to build GNU make and Bash using that: https://twitter.com/janneke_gnu/status/1070434782973063168 https://twitter.com/janneke_gnu/status/1070434782973063168 > Anyway, I am much more concerned about much higher places in free > software bootstrapping. Lots of software has switched from Autotools > to Meson without thinking about how we will bootstrap this > stuff. Meson requires you to have Ninja to work, but how can I > bootstrap Ninja without Python? Surely that the technical part of that problem can be solved quite easily once the respective developers of those projects are becoming aware of bootstrapping and reproducibility and decide to give it priority? Happy hacking! janneke
im3w1l 8y ago
Well you could have an initial stage in the bootstrapping that is pure machine code, but is short and simple enough that it can be read and understood. Then that could implement a very simple assembler. You'd write a complex assembler in the simple assembly language. Then write a few steps of increasingly complicated compilers. Still need to trust the hardware though.

paulie_a 8y ago

I can't find a reference to it but there was an intentional security bug introduced to a c compiler that would pass through to the compiled code.

fanf2 8y ago
Reflections on trusting trust, Ken Thompson’s Turing award lecture https://www.archive.ece.cmu.edu/~ganger/712.fall02/papers/p761-thompson.pdf https://www.archive.ece.cmu.edu/~ganger/712.fall02/papers/p7...

avhon1 8y ago

> Maintaining older or smaller versions of software just to avoid self-hosting is bound to hit all kinds of bugs - big and small. Unfortunately, any compiler that can compile current versions of GCC would probably be enormous, and difficult to trust directly. C++ is a complicated language to compile. As long as the chosen versions of TCC and GCC 1. are free of bugs that affect compiling the next-more-sophisticated compiler, and 2. can be trusted, then having them as intermediate compilers isn't too terrible. Bugs in them that affect compiling the next-more-sophisticated compiler can be written and committed now, and will remain that way forever. The only place new bugs can be encountered is if the current version of GCC changes so that it can't be compiled by the last-frozen version of GCC. There are two ways to fix that: 1. Patch the last-frozen version of GCC so that it can still compile the latest GCC, or 2. add another intermediate version of GCC to the toolchain, since the last-frozen GCC could compile every version of GCC up to the most current, and the most current GCC must be compilable by at least one earlier version of GCC. This means that the compiler chain can potentially grow without bound, but it would do so slowly, and only by adding more compiler code onto the end of already-trusted compiler code. A long chain of obsolete-but-trusted compilers is not ideal, but should be a working and stable source of trust.

janneke 8y ago

> This seems like a really neat idea! I think it would be cool to use > plain source code to completely bootstrap a system. Technically that's what we are working towards. Idealogically, we are working towards moving awarenes in our communities from `neat idea' to `unbelievable that in 2018 we still used and trusted computers that had no bootstrappable audit trail!' > But, at the same time, this sounds like a maintenance nightmare! > Maintaining older or smaller versions of software just to avoid > self-hosting is bound to hit all kinds of bugs - big and small. You are not suggesting that we stop our efforts because it will require work and may contain bugs, rigth? ;-) Until now, this has been the effort of a very small team. The choice for tcc-0.9.26 and gcc-2.95.3 was a pragmatic one. We hope that when awareness of bootstrappable software rises we can make some better choices. I would love for the TinyCC and GNU GCC developers (any software developers, really) to take the lead in creating their own bootstrappable stories; we just showed it can be done. > At the same time, I'm not convinced this completely solves the > problem. Maybe you can trust your C compiler now, but can you trust > your Bash interpreter, or your sed command? Right, doing only part of the work does not solve the problem. The solution will be built from a number of such steps. One of those steps may involve hardware; I'm sure it's a long path. > What stops them from injecting things into your C compiler while > it's building. At the end of the day, you have to trust some > software, why not include your C compiler in this list? Some of us have decided that is just not good enough, that we would like our softwares not only reproducible but also bootstrappable, and are we working towards that. There is Stage0 and M2-Planet to take care of what's below Mes. We have also started work on a Bash replacement in Scheme (using GNU Guile initially) that comes with a minimal implementation of coreutils, grep, sed, tar. I have managed to build GNU make and Bash using that: https://twitter.com/janneke_gnu/status/1070434782973063168 https://twitter.com/janneke_gnu/status/1070434782973063168 > Anyway, I am much more concerned about much higher places in free > software bootstrapping. Lots of software has switched from Autotools > to Meson without thinking about how we will bootstrap this > stuff. Meson requires you to have Ninja to work, but how can I > bootstrap Ninja without Python? Surely that the technical part of that problem can be solved quite easily once the respective developers of those projects are becoming aware of bootstrapping and reproducibility and decide to give it priority? Happy hacking! janneke

im3w1l 8y ago

Well you could have an initial stage in the bootstrapping that is pure machine code, but is short and simple enough that it can be read and understood. Then that could implement a very simple assembler. You'd write a complex assembler in the simple assembly language. Then write a few steps of increasingly complicated compilers. Still need to trust the hardware though.