7 ms·

This is sort of off topic, but the fact that every change to the source requires (on my machine) 3-4 full seconds to recompile, without optimization, is indicat

by comex 12y ago

This is sort of off topic, but the fact that every change to the source requires (on my machine) 3-4 full seconds to recompile, without optimization, is indicative of the compiler performance problems that remain my main roadblock with Rust.

ehsanu1 12y ago
The core devs are sympathetic (what with Rust being self-hosted) and apparently incremental compilation is coming soon: https://github.com/rust-lang/meeting-minutes/blob/master/weekly-meetings/2014-09-30.md#cycle-time https://github.com/rust-lang/meeting-minutes/blob/master/wee... There is also a way to parallelize compilation, which might sometimes speed things up, it's just not enabled by default yet: http://discuss.rust-lang.org/t/default-settings-for-parallel-codegen/519 http://discuss.rust-lang.org/t/default-settings-for-parallel...
Skywing 12y ago
I remember a day where I was proud that my C projects took a few seconds to compile. Made me feel like I had written something with some umph to it.
melling 12y ago
Turbo Pascal and Delphi had incredibly fast compilers. Why can't we make fast compilation a required feature today? We should never have to wait for the compiler/linker. http://prog21.dadgum.com/47.html http://prog21.dadgum.com/47.html
- deleted 12y ago
  [deleted]
- pcwalton 12y ago
  They also didn't have the zero-cost memory safety abstractions that Rust has. Nor did they do much optimization. That said, we're working on compilation speed. The focus so far has been getting the language in shape and runtime performance of the generated code, not compilation speed (although we've picked most of the low-hanging fruit in compilation speed anyway).
  - cageface 12y ago
    How optimistic are you that the compilation speed can be significantly increased? Are there a lot of easy optimizations available or is this just the price paid for extra compile time safety? I'm not really a fan of Go but waiting for the compiler can be a pretty significant productivity killer on larger projects. Their fast compilation times are a pretty big selling point for what is (IMO) an otherwise underwhelming language design.
    - pcwalton 12y ago
      > How optimistic are you that the compilation speed can be significantly increased? Are there a lot of easy optimizations available or is this just the price paid for extra compile time safety? The vast majority of compile time for optimized builds (80%-90%) is spent in code generation and optimization in LLVM. For unoptimized builds, most of the compile time is spent in the typechecker doing type unifications for method lookup. With some optimizations to quickly reject method candidates I suspect this can be greatly improved. Incremental compilation for the fast turnaround is being worked on and there has been significant progress, to address comex' complaint. > Their fast compilation times are a pretty big selling point for what is (IMO) an otherwise underwhelming language design. Go 6g/8g also doesn't do much optimization by comparison to GCC/LLVM. (Rust uses LLVM.)
      - pjmlp 12y ago
        > Go 6g/8g also doesn't do much optimization by comparison to GCC/LLVM. (Rust uses LLVM.) You should compare with gccgo, though.
    - pjmlp 12y ago
      Rust compilation times can still be made faster. Generally I think many people complain too much for what takes a few seconds. C++ builds are measured in hours, usually require distributed build systems, clever use of forward declarations and cutting class private declarations into static code or PImpl classes to bring it down to something manageable.
- lobster_johnson 12y ago
  The Borland compilers were single-pass compilers; they did not go through an AST, but generated machine code directly from parser input. Also, the original Turbo Pascal compiler was written in assembly language. (Not sure about the later compiler versions used by BP and Delphi.) Also, the language was quite simple. With large, complex programs, the most time-consuming stage was in the linking of the final binary. If you want a similarly fast compiler today, look at Go.
  - lazyjones 12y ago
    > If you want a similarly fast compiler today, look at Go. There's also the amazing and much underrated Free Pascal: http://www.freepascal.org http://www.freepascal.org
- jeffdavis 12y ago
  Why? First and foremost, a language must satisfy some need that is not met adequately by another existing language. Otherwise it's useless. There are languages already with fast compilers, and some of them are pretty fast and have pretty good semantics. Rust is really trying to offer a combination of things not available elsewhere: safety, speed, and control. But that innovation is based heavily on compiler features, which have a compile-time performance penalty. Assuming rust succeeds on delivering an innovative new language, they can secondarily try to improve compiler performance. Or find ways to avoid paying some compile-time costs in certain situations.
reubenmorais 12y ago
Have you profiled the build? Is most of it spent in the compiler? A more practical point is that the main competitor is C++, a language notorious for long compilation times, so not-ultra-fast compile times might not be the highest priority for people working on rustc.
- fspeech 12y ago
  C++ compilers are very fast (I would say amazingly fast) unfortunately atrocious header only implementations like Boost drags it down. I just did clang++ -E on a single #include of cpp-netlib which depends on Boost asio and the dumped output came to: $ wc test1 279506 897765 10174700 test1 And this is after enabling dynamic linking which cuts down a few thousand lines. Library writers are not giving compiler writers a break! I find this practice atrocious. People should be able to just import the interface instead the whole implementation of everything (and enable "headers only" feature at the end if so desired to eliminate the need of linking).
  - yoklov 12y ago
    Err, header-only libraries exist mostly because templates require them, not because people don't like linking (well, there's some of that too, but it's the minority).
  - pjmlp 12y ago
    Walter Bright, the author of D and the first real C++ native compiler (Zortech C++), explains it http://www.drdobbs.com/cpp/c-compilation-speed/228701711 http://www.drdobbs.com/cpp/c-compilation-speed/228701711 Compatibility with C tooling is also a big reason, as templates can only be header only to be consumed by other translation units. Unfortunately "export template" was a failed experiment. Now C++ developers need to wait until C++17 for modules, if they ever get into the standard. And if they do, most likely it will take until around 2020 for all major C++ compilers across embedded, desktop and server systems offer support for it. Now the question is, if one is willing to wait that long or rather use a language that can use modules today.
  - lultimouomo 12y ago
smosher_ 12y ago
(Even more OT and not a criticism of you comment.) I wish someone could send your comment back in time so I could have read it when Doom was first released. I think it took longer just to get the game started up back then.
pcwalton 12y ago
Sounds like that's mostly the lack of incremental compilation, which is being worked on.
adamnemecek 12y ago
It's probably better that before 1.0, the development effort concentrates on getting the language right rather than compilation speed which can be fixed later.
- steveklabnik 12y ago
  While this is true, it's also important that things aren't introduced which make these kinds of optimizations impossible. I think we've done a good job of that, but it is something to keep in mind.
spullara 12y ago
It is in the top 3 issues with Scala as well. The amount of drag it causes in a project just isn't worth it.
- eropple 12y ago
  FWIW, I don't find it to be a big issue in Scala anymore with SBT's incremental compilation.
  - spullara 12y ago
    Depends on how many people are developing on the code base. If every morning is a full build with a few spread throughout the day for integration it is pretty awful. Additionally, I think submit queues are the way to go for committing code to master and if that takes an hour before you know your change is good I'm pretty unhappy. As far as incremental compiles go, just developing in IDEA does a pretty good job.
- pcwalton 12y ago
  The Rust compiler is much faster than Scala's compiler, from everything I have heard.
  - Alupis 12y ago
    really doesn't matter how long the compiler takes, so long as the result is fast.
    - pcwalton 12y ago
      That's not true; for quick turnaround it's important to be able to get results out of the compiler quickly.
      - Alupis 12y ago
        In what scenario does executable correctness and performance take a backseat to compiler runtime? Seems backwards to me.
pjc50 12y ago
3-4 seconds is extremely fast if you're coming from the C/C++ world. I can't see how this could possibly be a roadblock.
- comex 12y ago
  Not for incremental compilation when a single file changed, without optimization, it's not - depending on the type of change, of course, and how heavy the C++ code in question is, and I guess on whether you're using a broken IDE/build system (my standard is the command line, using make for small projects and ninja for larger ones). And incremental compilation can scale up to much larger projects without increasing the time much, whereas in Rust that would require splitting up into crates and even then would frequently require a full rebuild due to the coarse dependency tracking of "anything in this crate changed -> rebuild all dependents". That said, I expect incremental compilation built into rustc to greatly improve the situation - if done right, it should easily beat C++, although I don't know the details of the plan - so I'm happy someone is working on it. (In the past I wanted to work on it myself, but I was in a big generally unproductive slump..)
- simias 12y ago
  C++ definitely if you're using heavily templated code, C not so much though. Although I agree with your point, I've been using Rust quite a lot lately and compilation time was never an issue so far.
- phaylon 12y ago
  Another thing which I don't see often mentioned, but which alleviate most of the compile-time waiting-pain for me (and I'm actually a dynamic language guy) is the fact that since optimizations are usually what takes longest, the compiler is quite quick about telling me about errors. But maybe my projects just haven't been big enough :)
angersock 12y ago
Oh, Gods, 3-4 full seconds, perish the thought. That's really not that long a time at all.

by comex 12y ago

ehsanu1 12y ago

The core devs are sympathetic (what with Rust being self-hosted) and apparently incremental compilation is coming soon: https://github.com/rust-lang/meeting-minutes/blob/master/weekly-meetings/2014-09-30.md#cycle-time https://github.com/rust-lang/meeting-minutes/blob/master/wee... There is also a way to parallelize compilation, which might sometimes speed things up, it's just not enabled by default yet: http://discuss.rust-lang.org/t/default-settings-for-parallel-codegen/519 http://discuss.rust-lang.org/t/default-settings-for-parallel...

Skywing 12y ago

I remember a day where I was proud that my C projects took a few seconds to compile. Made me feel like I had written something with some umph to it.

melling 12y ago

Turbo Pascal and Delphi had incredibly fast compilers. Why can't we make fast compilation a required feature today? We should never have to wait for the compiler/linker. http://prog21.dadgum.com/47.html http://prog21.dadgum.com/47.html

deleted 12y ago
[deleted]
pcwalton 12y ago
They also didn't have the zero-cost memory safety abstractions that Rust has. Nor did they do much optimization. That said, we're working on compilation speed. The focus so far has been getting the language in shape and runtime performance of the generated code, not compilation speed (although we've picked most of the low-hanging fruit in compilation speed anyway).
- cageface 12y ago
  How optimistic are you that the compilation speed can be significantly increased? Are there a lot of easy optimizations available or is this just the price paid for extra compile time safety? I'm not really a fan of Go but waiting for the compiler can be a pretty significant productivity killer on larger projects. Their fast compilation times are a pretty big selling point for what is (IMO) an otherwise underwhelming language design.
  - pcwalton 12y ago
    > How optimistic are you that the compilation speed can be significantly increased? Are there a lot of easy optimizations available or is this just the price paid for extra compile time safety? The vast majority of compile time for optimized builds (80%-90%) is spent in code generation and optimization in LLVM. For unoptimized builds, most of the compile time is spent in the typechecker doing type unifications for method lookup. With some optimizations to quickly reject method candidates I suspect this can be greatly improved. Incremental compilation for the fast turnaround is being worked on and there has been significant progress, to address comex' complaint. > Their fast compilation times are a pretty big selling point for what is (IMO) an otherwise underwhelming language design. Go 6g/8g also doesn't do much optimization by comparison to GCC/LLVM. (Rust uses LLVM.)
    - pjmlp 12y ago
      > Go 6g/8g also doesn't do much optimization by comparison to GCC/LLVM. (Rust uses LLVM.) You should compare with gccgo, though.
  - pjmlp 12y ago
    Rust compilation times can still be made faster. Generally I think many people complain too much for what takes a few seconds. C++ builds are measured in hours, usually require distributed build systems, clever use of forward declarations and cutting class private declarations into static code or PImpl classes to bring it down to something manageable.
lobster_johnson 12y ago
The Borland compilers were single-pass compilers; they did not go through an AST, but generated machine code directly from parser input. Also, the original Turbo Pascal compiler was written in assembly language. (Not sure about the later compiler versions used by BP and Delphi.) Also, the language was quite simple. With large, complex programs, the most time-consuming stage was in the linking of the final binary. If you want a similarly fast compiler today, look at Go.
- lazyjones 12y ago
  > If you want a similarly fast compiler today, look at Go. There's also the amazing and much underrated Free Pascal: http://www.freepascal.org http://www.freepascal.org
jeffdavis 12y ago
Why? First and foremost, a language must satisfy some need that is not met adequately by another existing language. Otherwise it's useless. There are languages already with fast compilers, and some of them are pretty fast and have pretty good semantics. Rust is really trying to offer a combination of things not available elsewhere: safety, speed, and control. But that innovation is based heavily on compiler features, which have a compile-time performance penalty. Assuming rust succeeds on delivering an innovative new language, they can secondarily try to improve compiler performance. Or find ways to avoid paying some compile-time costs in certain situations.

reubenmorais 12y ago

Have you profiled the build? Is most of it spent in the compiler? A more practical point is that the main competitor is C++, a language notorious for long compilation times, so not-ultra-fast compile times might not be the highest priority for people working on rustc.

fspeech 12y ago
C++ compilers are very fast (I would say amazingly fast) unfortunately atrocious header only implementations like Boost drags it down. I just did clang++ -E on a single #include of cpp-netlib which depends on Boost asio and the dumped output came to: $ wc test1 279506 897765 10174700 test1 And this is after enabling dynamic linking which cuts down a few thousand lines. Library writers are not giving compiler writers a break! I find this practice atrocious. People should be able to just import the interface instead the whole implementation of everything (and enable "headers only" feature at the end if so desired to eliminate the need of linking).
- yoklov 12y ago
  Err, header-only libraries exist mostly because templates require them, not because people don't like linking (well, there's some of that too, but it's the minority).
- pjmlp 12y ago
  Walter Bright, the author of D and the first real C++ native compiler (Zortech C++), explains it http://www.drdobbs.com/cpp/c-compilation-speed/228701711 http://www.drdobbs.com/cpp/c-compilation-speed/228701711 Compatibility with C tooling is also a big reason, as templates can only be header only to be consumed by other translation units. Unfortunately "export template" was a failed experiment. Now C++ developers need to wait until C++17 for modules, if they ever get into the standard. And if they do, most likely it will take until around 2020 for all major C++ compilers across embedded, desktop and server systems offer support for it. Now the question is, if one is willing to wait that long or rather use a language that can use modules today.
- lultimouomo 12y ago

smosher_ 12y ago

(Even more OT and not a criticism of you comment.) I wish someone could send your comment back in time so I could have read it when Doom was first released. I think it took longer just to get the game started up back then.

pcwalton 12y ago

Sounds like that's mostly the lack of incremental compilation, which is being worked on.

adamnemecek 12y ago

It's probably better that before 1.0, the development effort concentrates on getting the language right rather than compilation speed which can be fixed later.

steveklabnik 12y ago
While this is true, it's also important that things aren't introduced which make these kinds of optimizations impossible. I think we've done a good job of that, but it is something to keep in mind.

spullara 12y ago

It is in the top 3 issues with Scala as well. The amount of drag it causes in a project just isn't worth it.

eropple 12y ago
FWIW, I don't find it to be a big issue in Scala anymore with SBT's incremental compilation.
- spullara 12y ago
  Depends on how many people are developing on the code base. If every morning is a full build with a few spread throughout the day for integration it is pretty awful. Additionally, I think submit queues are the way to go for committing code to master and if that takes an hour before you know your change is good I'm pretty unhappy. As far as incremental compiles go, just developing in IDEA does a pretty good job.
pcwalton 12y ago
The Rust compiler is much faster than Scala's compiler, from everything I have heard.
- Alupis 12y ago
  really doesn't matter how long the compiler takes, so long as the result is fast.
  - pcwalton 12y ago
    That's not true; for quick turnaround it's important to be able to get results out of the compiler quickly.
    - Alupis 12y ago
      In what scenario does executable correctness and performance take a backseat to compiler runtime? Seems backwards to me.

pjc50 12y ago

3-4 seconds is extremely fast if you're coming from the C/C++ world. I can't see how this could possibly be a roadblock.

comex 12y ago
Not for incremental compilation when a single file changed, without optimization, it's not - depending on the type of change, of course, and how heavy the C++ code in question is, and I guess on whether you're using a broken IDE/build system (my standard is the command line, using make for small projects and ninja for larger ones). And incremental compilation can scale up to much larger projects without increasing the time much, whereas in Rust that would require splitting up into crates and even then would frequently require a full rebuild due to the coarse dependency tracking of "anything in this crate changed -> rebuild all dependents". That said, I expect incremental compilation built into rustc to greatly improve the situation - if done right, it should easily beat C++, although I don't know the details of the plan - so I'm happy someone is working on it. (In the past I wanted to work on it myself, but I was in a big generally unproductive slump..)
simias 12y ago
C++ definitely if you're using heavily templated code, C not so much though. Although I agree with your point, I've been using Rust quite a lot lately and compilation time was never an issue so far.
phaylon 12y ago
Another thing which I don't see often mentioned, but which alleviate most of the compile-time waiting-pain for me (and I'm actually a dynamic language guy) is the fact that since optimizations are usually what takes longest, the compiler is quite quick about telling me about errors. But maybe my projects just haven't been big enough :)

angersock 12y ago

Oh, Gods, 3-4 full seconds, perish the thought. That's really not that long a time at all.