4 ms·

I do not understand this comment, as I have done separate compilation every day with C++ for over two two decades: the model inherently only allows this, as eve

by saurik 7y ago

I do not understand this comment, as I have done separate compilation every day with C++ for over two two decades: the model inherently only allows this, as every translation unit absolutely must be compiled separately and then linked... you don't even have a choice in the matter. I have also used both off-the-shelf and custom distributed build systems (most recently, I have deployed one which does insanely-parallel builds of a C++2a codebase using AWS Lambda), which pair extremely well with the explicit file-oriented dependencies that can be reported by each translation unit and used to determine which parts of the build are now stale given changes to the filesystem. I have never seen languages with module systems support even basic separate compilation, and it terrifies me; meanwhile, the flags I have seen being added to clang for attempting to preserve separate compilation behavior with the modules spec doesn't leave me feeling good about the endeavor. It could be my fears are unjustified, but it is extremely strange and somewhat annoying that the advocates for it don't ever seem to appreciate the state of the art for the status quo.

naasking 7y ago
> I do not understand this comment, as I have done separate compilation every day with C++ for over two two decades: the model inherently only allows this, as every translation unit absolutely must be compiled separately and then linked... you don't even have a choice in the matter. Except if your headers include macros or templates, which must be evaluated every time. This is not separate compilation. What C/C++ developers have been doing for 20 years is jumping through hoops to incrementally reuse previously compiled results because C/C++ violate basic modularity principles [1]. They call it "separate compilation", but it's a pale shadow of true separate compilation. Proper modules would enforce abstraction boundaries so true separate compilation becomes possible. I can't speak to the specific C++ proposal here, but enabling separate compilation is exactly what modules are for. [1] http://llvm.org/devmtg/2012-11/Gregor-Modules.pdf http://llvm.org/devmtg/2012-11/Gregor-Modules.pdf
- gpderetta 7y ago
  Normally use of templates do break separate compilation but it can be recovered, and it is routinely done on large codebases, via explicit template instantiation. The ergonomics are not great of course and hopefully modules will improve things.
dwaite 7y ago
With C++, you have a lot of limitations on cross-machine, cross-library compilation. The library is not defined just by the platform, but by the compilation directives and macros. Since headers can redefine macros, that also means that even including headers in a different order can change the meaning. Older languages like C++ with headers are simply source files with a different extension, so your individual source file compilation is actually loading in and parsing cross-sectional subsets of your full project. Modern languages typically define the interface inline with the code, and export that interface to other modules which would consume it. The reason that you don't have intra-module distributed build is because the intra-module code dependencies are implicit - there is no developer-defined partial cross section of the code, all of the code needs to be pulled in and semantically evaluated as a unit. You could hypothetically still do distributed optimization of the code after it is semantically evaluated, but TBH there is a lot of time wasted in evaluating headers for each file. Modern compiled languages usually have compilation time as a restriction on the implementation and design, and elimination of the need to repeatedly parse and semantically evaluate files is part of their design.
- saurik 7y ago
  > The reason that you don't have intra-module distributed build is because the intra-module code dependencies are implicit - there is no developer-defined partial cross section of the code, all of the code needs to be pulled in and semantically evaluated as a unit. And this seriously somehow doesn't seem like a problem to you? My massively distributed (to AWS Lambda) C++ build compiles at the speed of the slowest single translation unit. You seem to be optimizing for serial compile performance, but if you truly want speed you have to go parallel. These explicitly-managed "cross sectional sunsets" are what makes this so easy to do with C++.
  - swift 7y ago
    There are definite tradeoffs here, but one thing to consider is that dev tooling generally can't benefit from that kind of massively distributed parallelism. If you make a change to a widely-included header file, code intelligence tools suddenly have to do a huge amount of work to reevaluate their internal model of all affected translation units. This can really result in an unpleasant and laggy editing experience, even on medium-sized projects. That's the kind of thing that modules can help with; I certainly wished fervently that they were already a thing when I used to work on C++ static analysis tools years ago.

by saurik 7y ago

naasking 7y ago

> I do not understand this comment, as I have done separate compilation every day with C++ for over two two decades: the model inherently only allows this, as every translation unit absolutely must be compiled separately and then linked... you don't even have a choice in the matter. Except if your headers include macros or templates, which must be evaluated every time. This is not separate compilation. What C/C++ developers have been doing for 20 years is jumping through hoops to incrementally reuse previously compiled results because C/C++ violate basic modularity principles [1]. They call it "separate compilation", but it's a pale shadow of true separate compilation. Proper modules would enforce abstraction boundaries so true separate compilation becomes possible. I can't speak to the specific C++ proposal here, but enabling separate compilation is exactly what modules are for. [1] http://llvm.org/devmtg/2012-11/Gregor-Modules.pdf http://llvm.org/devmtg/2012-11/Gregor-Modules.pdf

gpderetta 7y ago
Normally use of templates do break separate compilation but it can be recovered, and it is routinely done on large codebases, via explicit template instantiation. The ergonomics are not great of course and hopefully modules will improve things.

dwaite 7y ago

With C++, you have a lot of limitations on cross-machine, cross-library compilation. The library is not defined just by the platform, but by the compilation directives and macros. Since headers can redefine macros, that also means that even including headers in a different order can change the meaning. Older languages like C++ with headers are simply source files with a different extension, so your individual source file compilation is actually loading in and parsing cross-sectional subsets of your full project. Modern languages typically define the interface inline with the code, and export that interface to other modules which would consume it. The reason that you don't have intra-module distributed build is because the intra-module code dependencies are implicit - there is no developer-defined partial cross section of the code, all of the code needs to be pulled in and semantically evaluated as a unit. You could hypothetically still do distributed optimization of the code after it is semantically evaluated, but TBH there is a lot of time wasted in evaluating headers for each file. Modern compiled languages usually have compilation time as a restriction on the implementation and design, and elimination of the need to repeatedly parse and semantically evaluate files is part of their design.

saurik 7y ago
> The reason that you don't have intra-module distributed build is because the intra-module code dependencies are implicit - there is no developer-defined partial cross section of the code, all of the code needs to be pulled in and semantically evaluated as a unit. And this seriously somehow doesn't seem like a problem to you? My massively distributed (to AWS Lambda) C++ build compiles at the speed of the slowest single translation unit. You seem to be optimizing for serial compile performance, but if you truly want speed you have to go parallel. These explicitly-managed "cross sectional sunsets" are what makes this so easy to do with C++.
- swift 7y ago
  There are definite tradeoffs here, but one thing to consider is that dev tooling generally can't benefit from that kind of massively distributed parallelism. If you make a change to a widely-included header file, code intelligence tools suddenly have to do a huge amount of work to reevaluate their internal model of all affected translation units. This can really result in an unpleasant and laggy editing experience, even on medium-sized projects. That's the kind of thing that modules can help with; I certainly wished fervently that they were already a thing when I used to work on C++ static analysis tools years ago.