7 ms·

I never understood why "self-hosting" is considered a good thing. Sure, the compiler developers can write in their favorite language and I guess it means the la

I never understood why "self-hosting" is considered a good thing. Sure, the compiler developers can write in their favorite language and I guess it means the language is stable enough to be used in a complex project; however requiring an (older) compiler to build the compiler seems a significant complication for software distributions. You either have to set up an ever increasing chain of compilers, as the complexity of the language and the features required to build the compiler grows, or rely on pre-existing binaries. Either way, it seems like a nightmare compared to keeping it in plain old C and building with any compiler of your choice. Just Imagine if any project did this, like, building Firefox now requires an existing Firefox installation.

damagednoob 4y ago
For me, compilers tend to be complex so it's a way to show that it can be done. Also, dogfooding[1]. [1]: https://en.m.wikipedia.org/wiki/Eating_your_own_dog_food https://en.m.wikipedia.org/wiki/Eating_your_own_dog_food
nindalf 4y ago
You don’t know if the product you’re building is any good unless you’re using it yourself.
- hypertele-Xii 4y ago
  Also known as dogfooding.
nicoburns 4y ago
> Either way, it seems like a nightmare compared to keeping it in plain old C and building with any compiler of your choice. Just Imagine if any project did this, like, building Firefox now requires an existing Firefox installation. But this is exactly the same for C! Building C compiler requires an existing C compiler. C compilers are admittedly more widely available at the moment, but that probably won't be the case forever. Languages like zig aim to supplant C. How are they supposed to do that when they're directly dependent on C?
hardwaregeek 4y ago
I believe Andy noted that it's way easier to find contributors for the self hosted compiler. If someone is passionate about the language, they want to write the language, not C++. And specifically for languages like Rust and Zig, the explicit orthodoxy is that the language is a better successor to C++/C. If you're spending your time telling people that they shouldn't write C++/C and should instead write your language, you probably should put your money where your mouth is and switch the compiler to your own language. Note that this is significantly less common with high level languages. Ruby, Python, even Swift are not bootstrapped. That's because none of these are claiming to take over the systems software niche and therefore don't need to prove anything by bootstrapping.
- v3ss0n 4y ago
  PyPy is written in pure python and transpiled to C++
  - ferdowsi 4y ago
    Pypy is mostly written in a restricted dialect of Python (rpython), not pure python. They're substantively different.
- pjmlp 4y ago
  Better educate yourself on Swift, not only does Apple clearly assert on their documentation that Swift is their replacement for C, C++ and Objective-C, they have started the planning efforts to bootsptrap it. Ruby and Python are scripting language, as such seldom bootstraped.
WalterBright 4y ago
Because a large motivation of developing a new language is wanting to program in that new language. Being forced to keep programming in C can become very frustrating.
- tialaramex 4y ago
  Although if you're Jonathan Blow you can stream yourself writing C++ and ranting about how terrible C++ is, while implementing your new Jai language. [I watched a few hours of Jon doing this, but annoyingly I can't find a recording of Jon actually finding and fixing the subtle Heisenbug he realises during those hours must be in his Jai code - he resolves to investigate "later" and I can't find "later". I think this would be insightful to watch because this is exactly the sort of bug Jon says he doesn't have much trouble with in his C++ and so doesn't need to prevent in Jai... so if it's five minutes to fix that shows Jon is correct, if it's hours of difficult searching or eventually was resolved by just unrelated changes to the code not so much]
  - WalterBright 4y ago
    It was quite a pleasure for me when the D compiler was converted 100% into D. I immediately set about refactoring it (a process that continues today). One consequence is that pointer bugs in the compiler have become extremely rare.
    - Gibbon1 4y ago
      One time I needed an AVR programmer for a test stand that didn't suck[1] and didn't backfeed the device under test. So I designed one using an AVR. And programmed it with the crappy one. Then I programmed the next one with the first one. Feels good. [1] Atmels cheap USB programmer would lose it's mind if the programming pins aren't all connected[2]. Tech had to reset it manually by unplugging the USB cable. If it happened a couple of times in a row, windows would disable the USB port. [2] Board house liked to put their inspection stamp on top of the programming port pads.
0x0203 4y ago
I understand that C is the same way and that there are benefits for language developers in using their own language, so I can't complain too loudly about languages that do this, but I can attest to the "nightmare" that this creates. I have personally ported java to an operating system that didn't previously have a java compiler. A colleague recently ported rust. Neither had usable cross-compilers. Both require a compiler of their own language to build their own compilers. It's a _giant_ pain in the neck. I wish that all language devs would provide a way of compiling the build tools with something that nearly everybody already has, like C or C++. Having language support baked into a commonly used compiler like gcc is also nice. I think there are good efforts to get a rust compiler into gcc, and I believe it already works for Go and D and a few others. Such things make adoption outside the standard Windows/Linux world much, much easier.
- easeout 4y ago
  Zig in particular stands out for having strong support for cross-compiling. I wonder how much simpler that makes adding a new platform to a self-hosted language, and what challenges persist?
  - 0x0203 4y ago
    I actually wouldn't mind porting Zig to our OS; right now it seems it would be pretty easy, and looks like they're going to make the bootstrapping process pretty straight forward in the future as well. I just wish other languages would do the same... Unfortunately for me, we don't have any users that need Zig right now and all my other side-projects are higher priority. I'll probably give it a try eventually though.
- defen 4y ago
  I believe the long-term goal is to implement a C backend for the compiler (compile Zig code to C). Then compile the Zig compiler (which is written in Zig) to C, so you can compile that on your target system which only has a C compiler. Then you can use that to rebuild the native Zig compiler on that system.
- LegionMammal978 4y ago
easeout 4y ago
If Zig is built on a C compiler, and (I assume) the C compiler self-hosts, then if Zig starts to self-host, it doesn't _introduce_ the dependency on a chain of previously built binaries, it just makes it a step more severe while removing a dependency. Would you prefer if Zig not only didn't self-host, but didn't have any self-hosting dependencies? For your ideal compiler, would a full build from scratch require making a platform-specific assembler from machine code, then a few stages of successively higher-level languages?
- pjmlp 4y ago
  That would be a possibility, another one is to cross-compile.
AndyKelley 4y ago
There's a third option that you missed which is what Zig does. It has a "bootstrap" compiler, written in C, which is kept in sync with the self-hosted compiler. Features are implemented twice, once in the bootstrap compiler, once in the self-hosted compiler. So the build process involves a fixed set of steps - stage1, stage2, stage3 - and the chain never grows more than this. No pre-existing binaries are required.
- deleted 4y ago
  [deleted]
- jessermeyer 4y ago
  > No pre-existing binaries are required. Except for of course a c compiler.
  - erichocean 4y ago
    Which is…self-hosted.
    - naikrovek 4y ago
      need a compiler to compile it, though.
  - pjmlp 4y ago
    Currently, alternatively they could cross-compile.
  - nyberg 4y ago
    Which is something mescc and friends solve. Zig doesn't need to solve the full bootstrap from nothing chain, just the entry.
pjmlp 4y ago
Once upon a time C was bootstrapped as well. A great outcome of bootstrapped compilers is that most contributors can use the language they known instead of two, and better it kills the myth that C is the only game in town for writing compilers.
- titzer 4y ago
  C is a wretchedly bad language for writing compilers.
  - pjmlp 4y ago
    Indeed, however since that is what many see, plenty of which place the genesis of high level languages on the birth of UNIX, the myth persists.
    - titzer 4y ago
      The fact that constant-folding is hard because the C language doesn't have portable semantics for basic things like integer addition (with overflow) is just braindead.
      - pjmlp 4y ago
        Yes, there are lots of optimizations that are hard in C and only possible thanks to UB abuse.
        titzer 4y ago
        I feel like C screwed up cross-compilation so badly that people think it is hard for some reason. Which it obviously isn't. The Virgil compiler just includes all of its backends in builds for all of its hosts. There's nothing special about the host machine.
- chrisseaton 4y ago
  > it kills the myth that C is the only game in town for writing compilers A compiler is a pretty pure, high-level operation. Makes no sense to use a low-level language for it.
titzer 4y ago
Virgil is completely self-hosted. The git repository just has stable binaries checked in that are updated periodically (15-20 times over the past 12 years or so). It bootstrapped first from an interpreter written in Java, which is long obsolete. Because of this, Virgil has no dependencies on other compilers or languages, other than for testing (bash and a little C).

damagednoob 4y ago

For me, compilers tend to be complex so it's a way to show that it can be done. Also, dogfooding[1]. [1]: https://en.m.wikipedia.org/wiki/Eating_your_own_dog_food https://en.m.wikipedia.org/wiki/Eating_your_own_dog_food

nindalf 4y ago

You don’t know if the product you’re building is any good unless you’re using it yourself.

hypertele-Xii 4y ago
Also known as dogfooding.

nicoburns 4y ago

> Either way, it seems like a nightmare compared to keeping it in plain old C and building with any compiler of your choice. Just Imagine if any project did this, like, building Firefox now requires an existing Firefox installation. But this is exactly the same for C! Building C compiler requires an existing C compiler. C compilers are admittedly more widely available at the moment, but that probably won't be the case forever. Languages like zig aim to supplant C. How are they supposed to do that when they're directly dependent on C?

hardwaregeek 4y ago

I believe Andy noted that it's way easier to find contributors for the self hosted compiler. If someone is passionate about the language, they want to write the language, not C++. And specifically for languages like Rust and Zig, the explicit orthodoxy is that the language is a better successor to C++/C. If you're spending your time telling people that they shouldn't write C++/C and should instead write your language, you probably should put your money where your mouth is and switch the compiler to your own language. Note that this is significantly less common with high level languages. Ruby, Python, even Swift are not bootstrapped. That's because none of these are claiming to take over the systems software niche and therefore don't need to prove anything by bootstrapping.

v3ss0n 4y ago
PyPy is written in pure python and transpiled to C++
- ferdowsi 4y ago
  Pypy is mostly written in a restricted dialect of Python (rpython), not pure python. They're substantively different.
pjmlp 4y ago
Better educate yourself on Swift, not only does Apple clearly assert on their documentation that Swift is their replacement for C, C++ and Objective-C, they have started the planning efforts to bootsptrap it. Ruby and Python are scripting language, as such seldom bootstraped.

WalterBright 4y ago

Because a large motivation of developing a new language is wanting to program in that new language. Being forced to keep programming in C can become very frustrating.

tialaramex 4y ago
Although if you're Jonathan Blow you can stream yourself writing C++ and ranting about how terrible C++ is, while implementing your new Jai language. [I watched a few hours of Jon doing this, but annoyingly I can't find a recording of Jon actually finding and fixing the subtle Heisenbug he realises during those hours must be in his Jai code - he resolves to investigate "later" and I can't find "later". I think this would be insightful to watch because this is exactly the sort of bug Jon says he doesn't have much trouble with in his C++ and so doesn't need to prevent in Jai... so if it's five minutes to fix that shows Jon is correct, if it's hours of difficult searching or eventually was resolved by just unrelated changes to the code not so much]
- WalterBright 4y ago
  It was quite a pleasure for me when the D compiler was converted 100% into D. I immediately set about refactoring it (a process that continues today). One consequence is that pointer bugs in the compiler have become extremely rare.
  - Gibbon1 4y ago
    One time I needed an AVR programmer for a test stand that didn't suck[1] and didn't backfeed the device under test. So I designed one using an AVR. And programmed it with the crappy one. Then I programmed the next one with the first one. Feels good. [1] Atmels cheap USB programmer would lose it's mind if the programming pins aren't all connected[2]. Tech had to reset it manually by unplugging the USB cable. If it happened a couple of times in a row, windows would disable the USB port. [2] Board house liked to put their inspection stamp on top of the programming port pads.

0x0203 4y ago

I understand that C is the same way and that there are benefits for language developers in using their own language, so I can't complain too loudly about languages that do this, but I can attest to the "nightmare" that this creates. I have personally ported java to an operating system that didn't previously have a java compiler. A colleague recently ported rust. Neither had usable cross-compilers. Both require a compiler of their own language to build their own compilers. It's a _giant_ pain in the neck. I wish that all language devs would provide a way of compiling the build tools with something that nearly everybody already has, like C or C++. Having language support baked into a commonly used compiler like gcc is also nice. I think there are good efforts to get a rust compiler into gcc, and I believe it already works for Go and D and a few others. Such things make adoption outside the standard Windows/Linux world much, much easier.

easeout 4y ago
Zig in particular stands out for having strong support for cross-compiling. I wonder how much simpler that makes adding a new platform to a self-hosted language, and what challenges persist?
- 0x0203 4y ago
  I actually wouldn't mind porting Zig to our OS; right now it seems it would be pretty easy, and looks like they're going to make the bootstrapping process pretty straight forward in the future as well. I just wish other languages would do the same... Unfortunately for me, we don't have any users that need Zig right now and all my other side-projects are higher priority. I'll probably give it a try eventually though.
defen 4y ago
I believe the long-term goal is to implement a C backend for the compiler (compile Zig code to C). Then compile the Zig compiler (which is written in Zig) to C, so you can compile that on your target system which only has a C compiler. Then you can use that to rebuild the native Zig compiler on that system.
LegionMammal978 4y ago

easeout 4y ago

If Zig is built on a C compiler, and (I assume) the C compiler self-hosts, then if Zig starts to self-host, it doesn't _introduce_ the dependency on a chain of previously built binaries, it just makes it a step more severe while removing a dependency. Would you prefer if Zig not only didn't self-host, but didn't have any self-hosting dependencies? For your ideal compiler, would a full build from scratch require making a platform-specific assembler from machine code, then a few stages of successively higher-level languages?

pjmlp 4y ago
That would be a possibility, another one is to cross-compile.

AndyKelley 4y ago

There's a third option that you missed which is what Zig does. It has a "bootstrap" compiler, written in C, which is kept in sync with the self-hosted compiler. Features are implemented twice, once in the bootstrap compiler, once in the self-hosted compiler. So the build process involves a fixed set of steps - stage1, stage2, stage3 - and the chain never grows more than this. No pre-existing binaries are required.

deleted 4y ago
[deleted]
jessermeyer 4y ago
> No pre-existing binaries are required. Except for of course a c compiler.
- erichocean 4y ago
  Which is…self-hosted.
  - naikrovek 4y ago
    need a compiler to compile it, though.
- pjmlp 4y ago
  Currently, alternatively they could cross-compile.
- nyberg 4y ago
  Which is something mescc and friends solve. Zig doesn't need to solve the full bootstrap from nothing chain, just the entry.

pjmlp 4y ago

Once upon a time C was bootstrapped as well. A great outcome of bootstrapped compilers is that most contributors can use the language they known instead of two, and better it kills the myth that C is the only game in town for writing compilers.

titzer 4y ago
C is a wretchedly bad language for writing compilers.
- pjmlp 4y ago
  Indeed, however since that is what many see, plenty of which place the genesis of high level languages on the birth of UNIX, the myth persists.
  - titzer 4y ago
    The fact that constant-folding is hard because the C language doesn't have portable semantics for basic things like integer addition (with overflow) is just braindead.
    - pjmlp 4y ago
      Yes, there are lots of optimizations that are hard in C and only possible thanks to UB abuse.
      - titzer 4y ago
        I feel like C screwed up cross-compilation so badly that people think it is hard for some reason. Which it obviously isn't. The Virgil compiler just includes all of its backends in builds for all of its hosts. There's nothing special about the host machine.
chrisseaton 4y ago
> it kills the myth that C is the only game in town for writing compilers A compiler is a pretty pure, high-level operation. Makes no sense to use a low-level language for it.

titzer 4y ago

Virgil is completely self-hosted. The git repository just has stable binaries checked in that are updated periodically (15-20 times over the past 12 years or so). It bootstrapped first from an interpreter written in Java, which is long obsolete. Because of this, Virgil has no dependencies on other compilers or languages, other than for testing (bash and a little C).