4 ms·

> That's the fundamental limitation of modules systems that supposedly prevents this scaling? Not the person you're replying to but I can see a problem with so

by TuxSH 1y ago

> That's the fundamental limitation of modules systems that supposedly prevents this scaling? Not the person you're replying to but I can see a problem with some dependency chains. Let's say you have: stdlib <- A.hpp <- B.hpp (small header) <- C.hpp (likewise) <- many .cpp files. If you only precompile A.hpp (as is commonly done), the many .cpp files can be compiled in parallel once A.hpp is precompiled, and you get a nice speedup. If on the other hand you need to precompile everything, then all these cpp files must wait on A, then B, then C to be precompiled

aw1621107 1y ago
I'm not entirely sure modules systems must face that limitation. C++'s module system, for example, permits separation of module interfaces and module implementations, much like the existing header/implementation system. IIRC OCaml's module system does something similar, though I'm not familiar enough with it to say whether it qualifies as a module system beyond the name. Speaking more abstractly even if there isn't an explicit interface/implementation separation perhaps compilers could pick out and make available interface information "ahead of time" to alleviate/possibly eliminate the effect of otherwise problematic dependency chains? For example, in Rust you "just" need to look for signatures to figure out the interface and you have the corresponding keywords to make searching for those easy without having to fully parse the entire file. There's also the question of whether super large projects must have problematic dependency chains - hypothetically, if a super large project were written in a language with a module system some work would be done to try to structure it to minimize build time/system pain points. I don't think I can confidently say that that is always (im)possible.
- TuxSH 1y ago
  > Speaking more abstractly even if there isn't an explicit interface/implementation separation perhaps compilers could pick out and make available interface information "ahead of time" to alleviate/possibly eliminate the effect of otherwise problematic dependency chains? I'm not sure how well this would work for non-instantiated templates > There's also the question of whether super large projects must have problematic dependency chains Any header precompilation dependency chain is a dependency chain and may end up worse than fully parallel TU compilation if the time to parse said headers is faster than the time to compile them in a serial way. I can see modules being used, but relegated to, "import std; import fmt; import vulkan (etc)", typically use cases one should already use PCH for.
  - aw1621107 1y ago
    > I'm not sure how well this would work for non-instantiated templates I don't know either, but I was thinking about module systems in general rather than C++'s module system specifically, since the original comment I was responding to seemed to be speaking in generalities as well for that particular topic. > Any header precompilation dependency chain is a dependency chain and may end up worse than fully parallel TU compilation if the time to parse said headers is faster than the time to compile them in a serial way. Right, but it comes down to whether it's literally impossible to structure super large projects in a practical manner. Sure, maybe you eat some slowdown, maybe you get some speedup, but I'm a bit skeptical that modules must result in slowdowns of such a magnitude that super large projects are infeasible.

by TuxSH 1y ago

aw1621107 1y ago

I'm not entirely sure modules systems must face that limitation. C++'s module system, for example, permits separation of module interfaces and module implementations, much like the existing header/implementation system. IIRC OCaml's module system does something similar, though I'm not familiar enough with it to say whether it qualifies as a module system beyond the name. Speaking more abstractly even if there isn't an explicit interface/implementation separation perhaps compilers could pick out and make available interface information "ahead of time" to alleviate/possibly eliminate the effect of otherwise problematic dependency chains? For example, in Rust you "just" need to look for signatures to figure out the interface and you have the corresponding keywords to make searching for those easy without having to fully parse the entire file. There's also the question of whether super large projects must have problematic dependency chains - hypothetically, if a super large project were written in a language with a module system some work would be done to try to structure it to minimize build time/system pain points. I don't think I can confidently say that that is always (im)possible.

TuxSH 1y ago
> Speaking more abstractly even if there isn't an explicit interface/implementation separation perhaps compilers could pick out and make available interface information "ahead of time" to alleviate/possibly eliminate the effect of otherwise problematic dependency chains? I'm not sure how well this would work for non-instantiated templates > There's also the question of whether super large projects must have problematic dependency chains Any header precompilation dependency chain is a dependency chain and may end up worse than fully parallel TU compilation if the time to parse said headers is faster than the time to compile them in a serial way. I can see modules being used, but relegated to, "import std; import fmt; import vulkan (etc)", typically use cases one should already use PCH for.
- aw1621107 1y ago
  > I'm not sure how well this would work for non-instantiated templates I don't know either, but I was thinking about module systems in general rather than C++'s module system specifically, since the original comment I was responding to seemed to be speaking in generalities as well for that particular topic. > Any header precompilation dependency chain is a dependency chain and may end up worse than fully parallel TU compilation if the time to parse said headers is faster than the time to compile them in a serial way. Right, but it comes down to whether it's literally impossible to structure super large projects in a practical manner. Sure, maybe you eat some slowdown, maybe you get some speedup, but I'm a bit skeptical that modules must result in slowdowns of such a magnitude that super large projects are infeasible.