7 ms·

Not deeply familiar with C++ modules, but I've built and maintained fairly large build systems for other languages (and written my fair share of C++), and from

Not deeply familiar with C++ modules, but I've built and maintained fairly large build systems for other languages (and written my fair share of C++), and from this article I'm not quite sure where the intractable problems lie. It seems like the .bmi files are effectively an optimization that allows for fast incremental compilation, but a compiler doesn't actually need them to run compilation from scratch: it knows how to generate them, so if they're missing it can fall back to the old, slower #include-style compile-every-file behavior, generating the .bmi files as it goes. It doesn't seem like they add new slow paths that you can't already construct today with macros and #include, so it's hard to see why they'd be DOA: first time compilation should be no slower, but incremental compilation should be much faster thanks to interface stability. Maybe I'm missing something? It's not like C++ modules were designed by random nobodies, though; this has been worked on by build infra engineers at major companies with enormous C++ codebases like Facebook, and compiler maintainers e.g. the Clang maintainers. It's possible they completely forgot to think about parallel builds, but that seems at least a little unlikely.

dodobirdlord 8y ago
What I gathered is that module compilation is intended to be safe from preprocessor actions defined outside of the module. So the code you would generate with #include-style compilation and the code you would generate by compiling modules in the intended fashion aren't guaranteed to be the same. It seems as though this would mean that projects involving modules simply couldn't be compiled in the previous fashion.
andreareina 8y ago
First compilation time will be slower if before you could compile 8 files at a time, but now you can only do 3 at first because the others depend on those 3. Then maybe you can compile 6 at the same time, because all the rest of the code depends on those 6 modules, etc.
jsnell 8y ago
But you can't just compile-every-file. Each file can depend on the outputs of compiling some unknown set of other files. The compiler needs to become a build system, or the build system needs to become a compiler. The clang modules proposal had the concept of mapping files, mapping module names to file names. Companies like Facebook will presumably use proper build systems that already encode the dependency information in the build files rather than try to autodetect it. In that kind of an environment this proposal probably isn't particularly painful.
- coliveira 8y ago
  The compiler will not become a build system because this is out of scope for C++. With or without modules, C++ will continue to rely on an external dependency management tool, such as a Makefile. The introduction of modules will not change anything in this respect.
  - jsnell 8y ago
    Indeed. You've now taken one solution off the table. The other one is for the build system to become a compiler, which is equally unacceptable. That leaves you with manually encoding all dependency information in the build files. Which most people aren't doing (the exception being Bazel-like build systems which enforce that). That seems to leave us with just one conclusion: the article is right, and most of the ecosystem will never migrate to modules, leaving us with the worst of both worlds.
    - ginko 8y ago
      The build system doesn't need to become a compiler. It just needs to parse the source files for import statements in order to create a DAG.
      - ZirconiumX 8y ago
        But you also need compatibility with #include, so your build system also needs to become a preprocessor.
        ginko 8y ago
        The preprocessor is a self-contained binary. The build system can just pipe through it.
        gchpaco 8y ago
        The C and I presume C++ standard has been very carefully avoiding the idea of a preprocessor being separate at all. The standard was very carefully worded to prevent that being necessary, because most C compilers do not have a separate preprocessor. It is only Unix heritage compilers that really have one, and even they're not consistent about it.

dodobirdlord 8y ago

What I gathered is that module compilation is intended to be safe from preprocessor actions defined outside of the module. So the code you would generate with #include-style compilation and the code you would generate by compiling modules in the intended fashion aren't guaranteed to be the same. It seems as though this would mean that projects involving modules simply couldn't be compiled in the previous fashion.

andreareina 8y ago

First compilation time will be slower if before you could compile 8 files at a time, but now you can only do 3 at first because the others depend on those 3. Then maybe you can compile 6 at the same time, because all the rest of the code depends on those 6 modules, etc.

jsnell 8y ago

But you can't just compile-every-file. Each file can depend on the outputs of compiling some unknown set of other files. The compiler needs to become a build system, or the build system needs to become a compiler. The clang modules proposal had the concept of mapping files, mapping module names to file names. Companies like Facebook will presumably use proper build systems that already encode the dependency information in the build files rather than try to autodetect it. In that kind of an environment this proposal probably isn't particularly painful.

coliveira 8y ago
The compiler will not become a build system because this is out of scope for C++. With or without modules, C++ will continue to rely on an external dependency management tool, such as a Makefile. The introduction of modules will not change anything in this respect.
- jsnell 8y ago
  Indeed. You've now taken one solution off the table. The other one is for the build system to become a compiler, which is equally unacceptable. That leaves you with manually encoding all dependency information in the build files. Which most people aren't doing (the exception being Bazel-like build systems which enforce that). That seems to leave us with just one conclusion: the article is right, and most of the ecosystem will never migrate to modules, leaving us with the worst of both worlds.
  - ginko 8y ago
    The build system doesn't need to become a compiler. It just needs to parse the source files for import statements in order to create a DAG.
    - ZirconiumX 8y ago
      But you also need compatibility with #include, so your build system also needs to become a preprocessor.
      - ginko 8y ago
        The preprocessor is a self-contained binary. The build system can just pipe through it.
        gchpaco 8y ago
        The C and I presume C++ standard has been very carefully avoiding the idea of a preprocessor being separate at all. The standard was very carefully worded to prevent that being necessary, because most C compilers do not have a separate preprocessor. It is only Unix heritage compilers that really have one, and even they're not consistent about it.