6 ms·

How Go uses Go to build itself

108 points by spahl 13y ago

DavidWanjiru 13y ago
I'm the naive one here, but is "Go using Go to build itself" much like what Paul Graham talks about re LISP?
- swdunlop 13y ago
  In a much more complex sense, yes. Go's build process starts from various cruft found in Linux, OSX, or Windows, and makes a beeline for its own (more uniform) cruft, where things are uniform enough to build Go's toolchain. It is interesting to me, primarily, because as a Go user, I never have to see it. The toolchain's abstractions are good enough that, as long as I don't meddle with the syscall package or keep my CGO interactions shallow and confined, I really don't worry about crossing between developing on OS X and testing and deploying on Linux. (Or even crossing between amd64 and ARM, now that Go 1.1 is out.) Even cross-compilation is very straightforwards, once again, assuming you stay the hell away from CGO.
- cgag 13y ago
  What things about lisp in particular? I want to say no but I'm not sure I'm clear on what you're asking.
- pjmlp 13y ago
  Most languages can be used to built themselves, in a process known as compiler bootstrap. The main reason why some compiler developers don't do it is mostly a question of using existing tooling instead of redoing all required stuff in the new language. Additionally porting to new platforms by bootstrapping usually requires cross-compilation, which for some developers might not be worth it, depending on the target audience of the language.
  - ralph 13y ago
    The Go developers have said they deliberately didn't try and write Go in Go because they've done that with other languages, possibly Alef, I forget, and a bug can arise where the fix is obvious if it wasn't that the bug exists and would be triggered by the fix. Instead, a more awkward fix has to be figured out that doesn't trigger the bug. Of course, once the bug is fixed the original straightforward fix can be substituted but it's all unwanted hassle.
    - gizmo686 13y ago
      How often does this come up? Under normal circumstances, wouldn't you be able to revert to a version of the compiler from before the bug was introduced? If your still in initial development, then you have the old, foreign, compiler to fall back on.
      - ralph 13y ago
        Seemingly often enough that it deterred them with Go. No, you may not be able to revert to before the bug was introduced as it may be the bug was there ever since that feature was added. As soon as it is self-hosted, the old compiler becomes quickly irrelevant, i.e. the code rapidly diverts from what it can compile.
        pjmlp 13y ago
        That is why bootstraping done properly is always done in stages. You have a compiler that can only compile a specific subset and use that subset to write the real compiler. There are endless book examples how to do it. Given who Go designers are, I think they don't have any issue keeping the C code around.
        ralph 13y ago
        That's not how the books say to do it, and you're right, given who created Go you'd think they'd know this stuff. :-) Many generations of the compiler are created. Let's say the compiler-in-C is worked on until it compiles subset Gosub1 which is just enough to write compiler-in-Gosub1 that duplicates compiler-in-C's behaviour. From now, compiler-in-C atrophies. G-2 features are implemented in G-1's compiler, though nothing uses them yet. The compiler's source then uses these, making it G-2 source, only compilable by a G-2-grokking compiler. Weeks later we have a G-40 where a bug is discovered, introduced in G-20. It wasn't in the compiler-in-C so that's not useful. Choices include fixing it at `head', which can sometimes be awkward as described earlier, or fixing the initial G-20 implementation and then rolling forward all changes from there assuming the fix doesn't break code that was depending on the errant behaviour.
acqq 13y ago
The last time I've looked there were a lot of C code needed to be compiled with C to build Go, and definitely it was not for the bootstrapping purposes as the part of standard libraries were written in C. Is it still the case?
- andrewreds 13y ago
  Most of the standard library is written in go. a large part of the runtime package (within the standard library) is written in c and asm. This c code compiled with one of the 5c, 6c or 8c compilers. gcc is only used for bootstrapping (I believe... tho I am still trying to get my head around what happens).

108 points by spahl 13y ago

DavidWanjiru 13y ago

I'm the naive one here, but is "Go using Go to build itself" much like what Paul Graham talks about re LISP?

swdunlop 13y ago
In a much more complex sense, yes. Go's build process starts from various cruft found in Linux, OSX, or Windows, and makes a beeline for its own (more uniform) cruft, where things are uniform enough to build Go's toolchain. It is interesting to me, primarily, because as a Go user, I never have to see it. The toolchain's abstractions are good enough that, as long as I don't meddle with the syscall package or keep my CGO interactions shallow and confined, I really don't worry about crossing between developing on OS X and testing and deploying on Linux. (Or even crossing between amd64 and ARM, now that Go 1.1 is out.) Even cross-compilation is very straightforwards, once again, assuming you stay the hell away from CGO.
cgag 13y ago
What things about lisp in particular? I want to say no but I'm not sure I'm clear on what you're asking.
pjmlp 13y ago
Most languages can be used to built themselves, in a process known as compiler bootstrap. The main reason why some compiler developers don't do it is mostly a question of using existing tooling instead of redoing all required stuff in the new language. Additionally porting to new platforms by bootstrapping usually requires cross-compilation, which for some developers might not be worth it, depending on the target audience of the language.
- ralph 13y ago
  The Go developers have said they deliberately didn't try and write Go in Go because they've done that with other languages, possibly Alef, I forget, and a bug can arise where the fix is obvious if it wasn't that the bug exists and would be triggered by the fix. Instead, a more awkward fix has to be figured out that doesn't trigger the bug. Of course, once the bug is fixed the original straightforward fix can be substituted but it's all unwanted hassle.
  - gizmo686 13y ago
    How often does this come up? Under normal circumstances, wouldn't you be able to revert to a version of the compiler from before the bug was introduced? If your still in initial development, then you have the old, foreign, compiler to fall back on.
    - ralph 13y ago
      Seemingly often enough that it deterred them with Go. No, you may not be able to revert to before the bug was introduced as it may be the bug was there ever since that feature was added. As soon as it is self-hosted, the old compiler becomes quickly irrelevant, i.e. the code rapidly diverts from what it can compile.
      - pjmlp 13y ago
        That is why bootstraping done properly is always done in stages. You have a compiler that can only compile a specific subset and use that subset to write the real compiler. There are endless book examples how to do it. Given who Go designers are, I think they don't have any issue keeping the C code around.
        ralph 13y ago
        That's not how the books say to do it, and you're right, given who created Go you'd think they'd know this stuff. :-) Many generations of the compiler are created. Let's say the compiler-in-C is worked on until it compiles subset Gosub1 which is just enough to write compiler-in-Gosub1 that duplicates compiler-in-C's behaviour. From now, compiler-in-C atrophies. G-2 features are implemented in G-1's compiler, though nothing uses them yet. The compiler's source then uses these, making it G-2 source, only compilable by a G-2-grokking compiler. Weeks later we have a G-40 where a bug is discovered, introduced in G-20. It wasn't in the compiler-in-C so that's not useful. Choices include fixing it at `head', which can sometimes be awkward as described earlier, or fixing the initial G-20 implementation and then rolling forward all changes from there assuming the fix doesn't break code that was depending on the errant behaviour.

acqq 13y ago

The last time I've looked there were a lot of C code needed to be compiled with C to build Go, and definitely it was not for the bootstrapping purposes as the part of standard libraries were written in C. Is it still the case?

andrewreds 13y ago
Most of the standard library is written in go. a large part of the runtime package (within the standard library) is written in c and asm. This c code compiled with one of the 5c, 6c or 8c compilers. gcc is only used for bootstrapping (I believe... tho I am still trying to get my head around what happens).