3 ms·

If compilation of rustc runs out of memory it represent an upper limit on the complexity of actually viable Rust programs, and given how much software is larger

If compilation of rustc runs out of memory it represent an upper limit on the complexity of actually viable Rust programs, and given how much software is larger than a compiler it is a discouraging performance level.

steveklabnik 9y ago
Rustc is more than just a Rust program; we compile LLVM from scratch, for example. Is the OOM in the Rust code, or in the LLVM code, or in the final linking, or what? It's not clear. The compiler is one of the largest Rust programs that exist. Last I checked, it was three quarters of a million lines of Rust, but a quick loc shows 1.5 million lines of Rust, 2.3 million lines of C++, and 900,000 lines of C (again, mostly LLVM and jemalloc). Servo is also very large, and they don't report having OOMs, though I'm not sure if they build on 32-bit or just cross-compile.
- ratmice 9y ago
  FWIW, i've compiled llvm/rust with 2G, I never ran out of ram compiling rust, but in llvm gnu-ld would run out of memory, using the gold linker fixes that, as well as the configuration flag for enabling separate debug info -DLLVM_USE_SPLIT_DWARF=ON
oblio 9y ago
True, but we're talking about 32 bit systems. Chrome stopped being able to compile on 32 bit systems years ago, as a C++ programs. I think Firefox is in the same situations. It does sound kind of bad for Rust, but the competition isn't doing much better :p (weak excuse, I know)
- zbentley 9y ago
  There is a really, really substantial difference in the "required for basic credibility/usage" qualifications of a web browser versus an operating system. Operating system instances can run for decades, executing arbitrary tasks, without needing to be restarted. Web browsers need to be updated incredibly frequently just to remain functional for parts of the internet. That's not to say either one is "worse" or "better", but comparing the two on an axis like platform support is like comparing tomahawk missiles and tall ships. Totally different requirements and use cases.
MageSlayer 9y ago
Agree. Moreover, Rust compiler contains some dark areas which nobody wants to deal with. See https://github.com/rust-lang/rust/issues/38528 https://github.com/rust-lang/rust/issues/38528 for example. Basically it means that Rust compiler can suddenly take exponential time and space for compilation. That bug really bites hard any code heavy on iterators (Rust often praised feature!). It has reliable reproduce test-case, but still it's already year old and was down-prioritized! Hard to believe anybody uses Rust for real large project given so little attention to crucial details.
- steveklabnik 9y ago
  I mean, that thread has a comment less than a day ago, and Niko says: > I'm going to lower this from P-high to reflect reality. I'm still eager to investigate but haven't had time, and making it P-high is not helping =) P-high means someone is actively assigned and working on it, so yeah in some sense this is a down-prioritization, but only from "Someone is working on this, so put your work somewhere else" to "this is open to work on"; the de-prioritization may lead to it getting fixed sooner, as Niko is a busy guy. So, "nobody wants to deal with" feels like a mischaracterization to me here.
  - MageSlayer 9y ago
    Well, yes. The last comment says the issue is still there :) I mean this bug alone in fact nullify the entire incremental compilation effort. It's kind of weird. > The de-prioritization may lead to it getting fixed sooner, as Niko is a busy guy And Niko put "medium" priority month ago :)

steveklabnik 9y ago

Rustc is more than just a Rust program; we compile LLVM from scratch, for example. Is the OOM in the Rust code, or in the LLVM code, or in the final linking, or what? It's not clear. The compiler is one of the largest Rust programs that exist. Last I checked, it was three quarters of a million lines of Rust, but a quick loc shows 1.5 million lines of Rust, 2.3 million lines of C++, and 900,000 lines of C (again, mostly LLVM and jemalloc). Servo is also very large, and they don't report having OOMs, though I'm not sure if they build on 32-bit or just cross-compile.

ratmice 9y ago
FWIW, i've compiled llvm/rust with 2G, I never ran out of ram compiling rust, but in llvm gnu-ld would run out of memory, using the gold linker fixes that, as well as the configuration flag for enabling separate debug info -DLLVM_USE_SPLIT_DWARF=ON

oblio 9y ago

True, but we're talking about 32 bit systems. Chrome stopped being able to compile on 32 bit systems years ago, as a C++ programs. I think Firefox is in the same situations. It does sound kind of bad for Rust, but the competition isn't doing much better :p (weak excuse, I know)

zbentley 9y ago
There is a really, really substantial difference in the "required for basic credibility/usage" qualifications of a web browser versus an operating system. Operating system instances can run for decades, executing arbitrary tasks, without needing to be restarted. Web browsers need to be updated incredibly frequently just to remain functional for parts of the internet. That's not to say either one is "worse" or "better", but comparing the two on an axis like platform support is like comparing tomahawk missiles and tall ships. Totally different requirements and use cases.

MageSlayer 9y ago

Agree. Moreover, Rust compiler contains some dark areas which nobody wants to deal with. See https://github.com/rust-lang/rust/issues/38528 https://github.com/rust-lang/rust/issues/38528 for example. Basically it means that Rust compiler can suddenly take exponential time and space for compilation. That bug really bites hard any code heavy on iterators (Rust often praised feature!). It has reliable reproduce test-case, but still it's already year old and was down-prioritized! Hard to believe anybody uses Rust for real large project given so little attention to crucial details.

steveklabnik 9y ago
I mean, that thread has a comment less than a day ago, and Niko says: > I'm going to lower this from P-high to reflect reality. I'm still eager to investigate but haven't had time, and making it P-high is not helping =) P-high means someone is actively assigned and working on it, so yeah in some sense this is a down-prioritization, but only from "Someone is working on this, so put your work somewhere else" to "this is open to work on"; the de-prioritization may lead to it getting fixed sooner, as Niko is a busy guy. So, "nobody wants to deal with" feels like a mischaracterization to me here.
- MageSlayer 9y ago
  Well, yes. The last comment says the issue is still there :) I mean this bug alone in fact nullify the entire incremental compilation effort. It's kind of weird. > The de-prioritization may lead to it getting fixed sooner, as Niko is a busy guy And Niko put "medium" priority month ago :)