4 ms·

Bootstrapping a simple compiler from nothing (2002)

83 points by sph 3y ago

o11c 3y ago
For a more modern and complete approach see https://www.gnu.org/software/mes/manual/mes.html https://www.gnu.org/software/mes/manual/mes.html and links therefrom.
- rollcat 3y ago
  The problem with GNU is that whenever faced with a trade-off between simplicity and comprehensiveness, they chose the latter; in doing so, they made their software harder to understand and modify, and thus for their users to exercise the freedoms they were promised. mescc has one job, and one job only - why does it even have a flag for picking the optimization level? Optimize for output readability, nothing else.
tromp 3y ago
> What next? I really like how Ben Lynn went from a C like language to building a complete Haskell compiler in umpteen steps, starting with building a virtual machine for running programs written in Combinatory Logic. He also condensed this journey into a winning IOCCC entry [2]. [1] https://crypto.stanford.edu/~blynn/compiler/ https://crypto.stanford.edu/~blynn/compiler/ [2] https://www.ioccc.org/2019/lynn/hint.html https://www.ioccc.org/2019/lynn/hint.html
sylware 3y ago
What's important when bootstrapping, it is to have a set of easily audit-able code generator binaries with several alternatives (to avoid dev/vendor lock-in). That is a thing of the past. Anybody bringing on the table gcc or clang for this is only laughable matter, unfortunately.
- rekado 3y ago
  See also this for a description of the chain of small tools that get us from a binary/source hybrid to a full system: https://guix.gnu.org/en/blog/2023/the-full-source-bootstrap-building-from-source-all-the-way-down/ https://guix.gnu.org/en/blog/2023/the-full-source-bootstrap-...
  - sylware 3y ago
    I have been following the bootstrapping (aka "Trust") project for years. And everything went down the drain once gcc did switch to c++. One of the biggest mistakes in open source software (it was done on purpose probably to increase dev/vendor lock). You need to be able to compile gigantic SDKs that in order to compile the "chain" of compilers, starting from gcc 4.7.4. Everything would be clean if gcc stayed simple and plain C (max C99), as it should have always been. Hopefully, that was not something pushed at the MIT at gcc steering committee via B.Gates "donations", facilitated by the pedophile Epstein, wasn't it? And now rust, which has a syntax even more horrible than c++ is coming to linux already plagged by gcc extensions? ofmg... All that is so accutely bad, I am resisting bearely to conspiracy theories.
redleader55 3y ago
I like to imagine the following nightmare scenario: we write a compiler from scratch that it can bootstrap itself (like this one, or Stage0), then we do the same bootstrapping process for hardware - eg. write a FPGA that for a simple CPU that can run the compiler and build other stuff, we do all of this and then start building reproducible builds of everything we use today - gcc, clang, Linux, etc. Eventually we build an 100% open source machine where to run everything. After this step is done, we start running in parallel all the tools and compare the outputs between the reproducible builds on the open source stack and semi-open one. And we discover slight differences ...
- withinboredom 3y ago
  There was some children's (as in teen-targeted) book I read back in the late 90's or early 00's where the main informant was a dude who got a backdoor, via a supply chain attack, into the generic mouse drivers installed on almost every machine. Good times.
- tekknolagi 3y ago
  You might enjoy https://www.teamten.com/lawrence/writings/coding-machines/ https://www.teamten.com/lawrence/writings/coding-machines/
  - withinboredom 3y ago
    Wow. That was pretty epic...
  - bossyTeacher 3y ago
    do you mind summarising the article?
    - KineticLensman 3y ago
      It's a near-future SF, written in 2009. Sort of a riff on Ken Thompson's classic Reflections on Trusting Trust [0] (spoilers) Some programmers spot a bug that after a lot of investigation seems to be a case of a rogue machine intelligence inserting its own code into software used by humans. [0] https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_ReflectionsonTrustingTrust.pdf https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_Ref...
nickpsecurity 3y ago
People interested in this topic should check out Miraheze, esp Past Research, below: https://bootstrapping.miraheze.org/wiki/Main_Page https://bootstrapping.miraheze.org/wiki/Main_Page We dumped all kinds of bootstrapped and gradual projects on that site. Everything from assembly to type-safe languages to macros to formal verification. Take your pick!
dang 3y ago
Related: Bcompiler – Bootstrapping a simple compiler from nothing (2002) - https://news.ycombinator.com/item?id=8576237 https://news.ycombinator.com/item?id=8576237 - Nov 2014 (4 comments)
deleted 3y ago
[deleted]
zellyn 3y ago
The text refers to bcompiler.tar.gz, and indeed, changing .html to .tar.gz and keeping the rest of the archive.org URL the same, you can download http://web.archive.org/web/20121030222527/http://www.rano.org/bcompiler.tar.gz http://web.archive.org/web/20121030222527/http://www.rano.or...

83 points by sph 3y ago

o11c 3y ago

For a more modern and complete approach see https://www.gnu.org/software/mes/manual/mes.html https://www.gnu.org/software/mes/manual/mes.html and links therefrom.

rollcat 3y ago
The problem with GNU is that whenever faced with a trade-off between simplicity and comprehensiveness, they chose the latter; in doing so, they made their software harder to understand and modify, and thus for their users to exercise the freedoms they were promised. mescc has one job, and one job only - why does it even have a flag for picking the optimization level? Optimize for output readability, nothing else.

tromp 3y ago

> What next? I really like how Ben Lynn went from a C like language to building a complete Haskell compiler in umpteen steps, starting with building a virtual machine for running programs written in Combinatory Logic. He also condensed this journey into a winning IOCCC entry [2]. [1] https://crypto.stanford.edu/~blynn/compiler/ https://crypto.stanford.edu/~blynn/compiler/ [2] https://www.ioccc.org/2019/lynn/hint.html https://www.ioccc.org/2019/lynn/hint.html

sylware 3y ago

What's important when bootstrapping, it is to have a set of easily audit-able code generator binaries with several alternatives (to avoid dev/vendor lock-in). That is a thing of the past. Anybody bringing on the table gcc or clang for this is only laughable matter, unfortunately.

rekado 3y ago
See also this for a description of the chain of small tools that get us from a binary/source hybrid to a full system: https://guix.gnu.org/en/blog/2023/the-full-source-bootstrap-building-from-source-all-the-way-down/ https://guix.gnu.org/en/blog/2023/the-full-source-bootstrap-...
- sylware 3y ago
  I have been following the bootstrapping (aka "Trust") project for years. And everything went down the drain once gcc did switch to c++. One of the biggest mistakes in open source software (it was done on purpose probably to increase dev/vendor lock). You need to be able to compile gigantic SDKs that in order to compile the "chain" of compilers, starting from gcc 4.7.4. Everything would be clean if gcc stayed simple and plain C (max C99), as it should have always been. Hopefully, that was not something pushed at the MIT at gcc steering committee via B.Gates "donations", facilitated by the pedophile Epstein, wasn't it? And now rust, which has a syntax even more horrible than c++ is coming to linux already plagged by gcc extensions? ofmg... All that is so accutely bad, I am resisting bearely to conspiracy theories.

redleader55 3y ago

I like to imagine the following nightmare scenario: we write a compiler from scratch that it can bootstrap itself (like this one, or Stage0), then we do the same bootstrapping process for hardware - eg. write a FPGA that for a simple CPU that can run the compiler and build other stuff, we do all of this and then start building reproducible builds of everything we use today - gcc, clang, Linux, etc. Eventually we build an 100% open source machine where to run everything. After this step is done, we start running in parallel all the tools and compare the outputs between the reproducible builds on the open source stack and semi-open one. And we discover slight differences ...

withinboredom 3y ago
There was some children's (as in teen-targeted) book I read back in the late 90's or early 00's where the main informant was a dude who got a backdoor, via a supply chain attack, into the generic mouse drivers installed on almost every machine. Good times.
tekknolagi 3y ago
You might enjoy https://www.teamten.com/lawrence/writings/coding-machines/ https://www.teamten.com/lawrence/writings/coding-machines/
- withinboredom 3y ago
  Wow. That was pretty epic...
- bossyTeacher 3y ago
  do you mind summarising the article?
  - KineticLensman 3y ago
    It's a near-future SF, written in 2009. Sort of a riff on Ken Thompson's classic Reflections on Trusting Trust [0] (spoilers) Some programmers spot a bug that after a lot of investigation seems to be a case of a rogue machine intelligence inserting its own code into software used by humans. [0] https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_ReflectionsonTrustingTrust.pdf https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_Ref...

nickpsecurity 3y ago

People interested in this topic should check out Miraheze, esp Past Research, below: https://bootstrapping.miraheze.org/wiki/Main_Page https://bootstrapping.miraheze.org/wiki/Main_Page We dumped all kinds of bootstrapped and gradual projects on that site. Everything from assembly to type-safe languages to macros to formal verification. Take your pick!

dang 3y ago

Related: Bcompiler – Bootstrapping a simple compiler from nothing (2002) - https://news.ycombinator.com/item?id=8576237 https://news.ycombinator.com/item?id=8576237 - Nov 2014 (4 comments)

deleted 3y ago

[deleted]

zellyn 3y ago

The text refers to bcompiler.tar.gz, and indeed, changing .html to .tar.gz and keeping the rest of the archive.org URL the same, you can download http://web.archive.org/web/20121030222527/http://www.rano.org/bcompiler.tar.gz http://web.archive.org/web/20121030222527/http://www.rano.or...