5 ms·

> I would have said this too a few years ago, but Go is a huge counterexample. I've been wondering about this ever since Go came out. If I recall correctly, LL

by exDM69 11y ago

> I would have said this too a few years ago, but Go is a huge counterexample. I've been wondering about this ever since Go came out. If I recall correctly, LLVM was already a viable option back then. The latest release of Go earlier this week featured a pretty big revamp of their backend with a new SSA-based middle/backend. I wonder how much work went into that and could they have achieved that faster had they been on LLVM since day 1.

amelius 11y ago
I'd like to know what is the downside of using LLVM. Why did they choose to write their own optimizer? Anyway, the most difficult part of writing the compiler was probably the garbage collector.
- markokrajnc 11y ago
  Maybe the biggest issue is optimizing register allocation while providing the information for all roots for the garbage collection... For non-GC languages you don't need to care about it and just agressively optimize register allocation. For GC languages you have to be careful, because the GC must know 1) which registers are holding references and which not 2) which of those registers are "alive" and which are not in use anymore (and therefore ready for garbage collection)...
  - pcwalton 11y ago
    LLVM has full support for precise garbage collection.
- fmstephe 11y ago
  Russ Cox gave a really good answer to this question. I have to paraphrase because I can't find the original comment. It was done for implementation speed. They had a compiler backend which they, the early Go team, were all familiar with. It was small so they could quickly change it to suit Go, rather than the other way around. Russ Cox expressed the sentiment that if they had tried to use LLVM there would have been a lot of up front work that would have taken a long time and the Go project probably would have fizzled out. Using the plan 9 tool chain they were up and running very quickly allowing them to focus on the most important work, designing the language itself. (Probably shouldn't attribute this to Russ Cox, I've probably read into his (rather briefer than what I wrote above) comments) In my opinion this was the right choice. If LLVM turns out to be the best backend for Go it's easy to use it later. However, the language itself is largely fixed after 1.0. Better to focus on the language than the compiler backend which can be swapped out later.
- pjmlp 11y ago
  > I'd like to know what is the downside of using LLVM. Depending on C++ code and keep alive the belief that you can only create compilers in C and C++? As many without experience in compiler development think. Also, and this is specially painful when compiling Rust or Swift from source, saving themselves the pain of waiting around 4h for a clean compile.
  - kibwen 11y ago
    LLVM does take a long time to compile, but four hours is an exaggeration. A mid-range computer can compile it in 30 minutes with -j4, and a good computer can compile it in 10 with -j8.
    - pjmlp 11y ago
      Core duo netbook with 4GB RAM and 5400 RPM hard disk. I can assure you it really does take 4 hours. And to bring the point home, I can link you to the Swift issue where I was discussing this matter.
      - bitJericho 11y ago
        A core duo isn't even in the realm of mid-range. That's very nearly a 10 year old processor!
        pjmlp 11y ago
        Yeah, but it works perfectly well as a travel netbook for coding on the go. Also I never had 4h builds with single core processors back when I was working full time with C++. The slowest ones were around 2h. A core duo would run circles around those computer systems, hence my dismay how much hardware LLVM requires for building.
        bitJericho 11y ago
        I would imagine compiler design has become much more complex in 10 years, and a low end i3 is something like at least 3 times faster than a core duo.
aikah 11y ago
Is LLVM that well supported on windows ? Go aims at running on as much platforms and it seems to me a lot of languages built with LLVM don't run on windows (Swift,Crystal,...).
- kryptiskt 11y ago
  LLVM works well on Windows and can even be built with Visual Studio.
akavel 11y ago
There's an "official answer" in the FAQ [1] ("We considered using LLVM for gc but we felt it was too large and slow to meet our performance goals." Note: lowercase gc refers to Go compiler, whereas uppercase GC would refer to Garbage Collector). That said, I seem to recall other additional/more detailed reasons were given on various occasions too, including: - ease of iteration and freedom to make huge breaking changes (and they did make use of it, see e.g. various generations of the GC, or the SSA work); - thorough knowledge of the internals of the compiler they used (given they were it's authors; also this is linked to the reason I listed above); - one of the explicit goals for the Go toolchain was that it must compile very fast (which I believe is also somewhat at odds with advanced optimizations, like those used in LLVM) - they even quipped something like "we designed Go while waiting for C++ to compile"; - additionally, though I'm not sure if it was ever spelled explicitly, I suppose they wouldn't want to get tied to the LLVM as a huge dependency, including having to adjust to each and every change in LLVM API, regardless if sensible or not (I seem to recall from so me articles that LLVM does introduce breaking changes in API from time to time?); [1] https://golang.org/doc/faq#What_compiler_technology_is_used_to_build_the_compilers https://golang.org/doc/faq#What_compiler_technology_is_used_...
- witty_username 11y ago
  > which I believe is also somewhat at odds with advanced optimizations, like those used in LLVM You can configure the optimization level to 0, then only constant folding is performed when the IR is built, and nothing else is done.
- exDM69 11y ago
  Yes, GC is a big issue... or was back when Go was just getting started. GC support in LLVM should be MUCH better now. Ie. it has everything you need to hook up a GC in it, but there isn't a built-in GC in it. And the fact that the first people working on Go have a good knowledge of the Plan9 toolchain probably gave them a head start. But that benefit has probably eroded long ago. And I completely disagree with the Go team's emphasis on compile times. Fast result programs are far more important than fast compile times. Fast compile times are nice to have, but not much more than that. But this is just my opinion, it's as good as anyone else's.

by exDM69 11y ago

amelius 11y ago

I'd like to know what is the downside of using LLVM. Why did they choose to write their own optimizer? Anyway, the most difficult part of writing the compiler was probably the garbage collector.

markokrajnc 11y ago
Maybe the biggest issue is optimizing register allocation while providing the information for all roots for the garbage collection... For non-GC languages you don't need to care about it and just agressively optimize register allocation. For GC languages you have to be careful, because the GC must know 1) which registers are holding references and which not 2) which of those registers are "alive" and which are not in use anymore (and therefore ready for garbage collection)...
- pcwalton 11y ago
  LLVM has full support for precise garbage collection.
fmstephe 11y ago
Russ Cox gave a really good answer to this question. I have to paraphrase because I can't find the original comment. It was done for implementation speed. They had a compiler backend which they, the early Go team, were all familiar with. It was small so they could quickly change it to suit Go, rather than the other way around. Russ Cox expressed the sentiment that if they had tried to use LLVM there would have been a lot of up front work that would have taken a long time and the Go project probably would have fizzled out. Using the plan 9 tool chain they were up and running very quickly allowing them to focus on the most important work, designing the language itself. (Probably shouldn't attribute this to Russ Cox, I've probably read into his (rather briefer than what I wrote above) comments) In my opinion this was the right choice. If LLVM turns out to be the best backend for Go it's easy to use it later. However, the language itself is largely fixed after 1.0. Better to focus on the language than the compiler backend which can be swapped out later.
pjmlp 11y ago
> I'd like to know what is the downside of using LLVM. Depending on C++ code and keep alive the belief that you can only create compilers in C and C++? As many without experience in compiler development think. Also, and this is specially painful when compiling Rust or Swift from source, saving themselves the pain of waiting around 4h for a clean compile.
- kibwen 11y ago
  LLVM does take a long time to compile, but four hours is an exaggeration. A mid-range computer can compile it in 30 minutes with -j4, and a good computer can compile it in 10 with -j8.
  - pjmlp 11y ago
    Core duo netbook with 4GB RAM and 5400 RPM hard disk. I can assure you it really does take 4 hours. And to bring the point home, I can link you to the Swift issue where I was discussing this matter.
    - bitJericho 11y ago
      A core duo isn't even in the realm of mid-range. That's very nearly a 10 year old processor!
      - pjmlp 11y ago
        Yeah, but it works perfectly well as a travel netbook for coding on the go. Also I never had 4h builds with single core processors back when I was working full time with C++. The slowest ones were around 2h. A core duo would run circles around those computer systems, hence my dismay how much hardware LLVM requires for building.
        bitJericho 11y ago
        I would imagine compiler design has become much more complex in 10 years, and a low end i3 is something like at least 3 times faster than a core duo.

aikah 11y ago

Is LLVM that well supported on windows ? Go aims at running on as much platforms and it seems to me a lot of languages built with LLVM don't run on windows (Swift,Crystal,...).

kryptiskt 11y ago
LLVM works well on Windows and can even be built with Visual Studio.

akavel 11y ago

There's an "official answer" in the FAQ [1] ("We considered using LLVM for gc but we felt it was too large and slow to meet our performance goals." Note: lowercase gc refers to Go compiler, whereas uppercase GC would refer to Garbage Collector). That said, I seem to recall other additional/more detailed reasons were given on various occasions too, including: - ease of iteration and freedom to make huge breaking changes (and they did make use of it, see e.g. various generations of the GC, or the SSA work); - thorough knowledge of the internals of the compiler they used (given they were it's authors; also this is linked to the reason I listed above); - one of the explicit goals for the Go toolchain was that it must compile very fast (which I believe is also somewhat at odds with advanced optimizations, like those used in LLVM) - they even quipped something like "we designed Go while waiting for C++ to compile"; - additionally, though I'm not sure if it was ever spelled explicitly, I suppose they wouldn't want to get tied to the LLVM as a huge dependency, including having to adjust to each and every change in LLVM API, regardless if sensible or not (I seem to recall from so me articles that LLVM does introduce breaking changes in API from time to time?); [1] https://golang.org/doc/faq#What_compiler_technology_is_used_to_build_the_compilers https://golang.org/doc/faq#What_compiler_technology_is_used_...

witty_username 11y ago
> which I believe is also somewhat at odds with advanced optimizations, like those used in LLVM You can configure the optimization level to 0, then only constant folding is performed when the IR is built, and nothing else is done.
exDM69 11y ago
Yes, GC is a big issue... or was back when Go was just getting started. GC support in LLVM should be MUCH better now. Ie. it has everything you need to hook up a GC in it, but there isn't a built-in GC in it. And the fact that the first people working on Go have a good knowledge of the Plan9 toolchain probably gave them a head start. But that benefit has probably eroded long ago. And I completely disagree with the Go team's emphasis on compile times. Fast result programs are far more important than fast compile times. Fast compile times are nice to have, but not much more than that. But this is just my opinion, it's as good as anyone else's.