5 ms·

The road to hell is paved with good intentions and C++ modules (2023)

85 points by fanf2 2y ago

db48x 2y ago
I stopped paying attention to C++ stuff years ago. Did any of these problems ever get fixed?
- kevindamm 2y ago
  If by fixed you mean "elaborate measures for specificity and saliency but still the same order of undefined behavior & bugbears" then yeah? It's gotten a lot more... more.
  - db48x 2y ago
    Oh good, I’d hate for the industry to have stagnated :P
Bostonian 2y ago
The author says Fortran modules are easier for build systems to handle: 'A Ninja file is static, thus we need to know a) the compilation arguments to use and b) interdependencies between files. The latter can't be done for things like Fortran or C++ modules. This is why Ninja has a functionality called dyndeps. It is a way to call an external program that scans the sources to be built and then writes a simple Makefile-esque snippet describing which order things should be built in. This is a simple and understandable step that works nicely for Fortran modules but is by itself insufficient for C++ modules.' ... 'Let's start by listing the requirements [for modules]: All command lines used must be knowable a priori, that is, without scanning the contents of source files Any compiler may choose to name its module files however it wants, but said mapping must be knowable just from the compiler name and version, i.o.w. it has to be documented Changing source files must not cause, by itself, a reconfiguration, only a rescan for deps followed by a build of the changed files The developer must not need to tell which sources are which module types in the build system, it is to be deduced automatically without needing to scan the contents of source files (i.e. by using the proper file extension) Module files are per-compiler and per-version and only make sense within the build tree of a single project If module files are to be installed, those are defined in a way that does not affect source files with modules that do not need installing. Fortran modules already satisfy all of these.'
- kergonath 2y ago
  Fortran modules are great for developers but a bit of a pain when setting up a build system (even though it’s quite straightforward now, it still requires a first pass with something like makedepf90 or one of its many equivalents). I find it quite a feat that C++ modules, while benefitting from the experience of many languages, ended up so broken. Also, I never really liked cmake but this looks really bad.
  - kazinator 2y ago
    If your language has modules, like in the syntax and semantics, and you're still requiring external tools to hunt down dependencies, your module system must somehow be a failure. The idea behind modules is that since they are part of the language, the compiler for the language understands modules and follows the dependencies. All module-related tooling, like incremental builds, is hooked through the compiler. You don't use make or external dependency generators. If someone does need a dependency generator for whatever reason, the compiler has to provide that function. You give the compiler the proper option to spit out dependencies, point it at the root module of the program, and let it do its thing.
    - kergonath 2y ago
      > If your language has modules, like in the syntax and semantics, and you're still requiring external tools to hunt down dependencies, your module system must somehow be a failure. No argument from me :) > The idea behind modules is that since they are part of the language, the compiler for the language understands modules and follows the dependencies. All module-related tooling, like incremental builds, is hooked through the compiler. I am not sure I follow. The compiler still compiles one file each time it is run, so we need a first pass to determine the dependency tree and what can be compiled in which order, right? Even if that first pass uses the compiler itself (doing only enough parsing to see what is needed). > You give the compiler the proper option to spit out dependencies, point it at the root module of the program, and let it do its thing. Nowadays, gfortran can generate Makefile fragments with the dependencies, but it was not always the case. This is why there are a handful of small tools to do it and makedepf90 is one of them (it’s a small Fortran program with a rudimentary parser). There are also other compilers that don’t do it properly.
awinter-py 2y ago
ughghgh every proposal for C/C++ modules feels like it's by someone who has not used a working module system and doesn't believe it's truly possible to create
- IshKebab 2y ago
  It's because they're trying to make a module system that is compatible with decades of development based around `#include`. It's nowhere near as simple as making a new language with modules. I almost feel like they should give up. IMO Rust and Zig have made C/C++ a legacy language. Except in some specific areas where Rust doesn't have mature libraries (GUIs & games mostly) you'd be silly to start a new project in C++. Migrating existing C++ projects to modules likely isn't worth the enormous effort, and the days of starting new C++ projects are numbered... so why bother?
  - deleted 2y ago
    [deleted]
  - _hl_ 2y ago
    There is still an unfathomably huge landscape of problems where C++ is the more mature, “right” choice over rust for new projects. Just from my (extremely limited and narrow) personal experience: - scientific/numerical computing (super mature) - domain-specific high performance computing (C++ is for better or worse extremely flexible) - weird CUDA use cases (feels very native in C++) - game dev as you mention - probably much much more C++ to me feels more like a meta-language where you go off to build your own kind of high performance DSL. Rust is much more rigid, which is probably a good thing for many “standard” use cases. But I’m sure there will always be a very very long tail of these niche projects where the flexibility comes in handy.
    - zrules 2y ago
      Opencv while not niche is also one of them. It’s definitely preventing me from going full in on rust. Probably for a while. This Reddit comment captures my reasons pretty well: https://www.reddit.com/r/rust/comments/1arys3z/comment/kqoak17/ https://www.reddit.com/r/rust/comments/1arys3z/comment/kqoak...
Conscat 2y ago
How is it that D and Objective-C++ came up with modules that work and can be used many years ago, yet C++ with much greater stakes fumbled it on almost every level?
- marcosdumay 2y ago
  Because the C family is guided by a committee. Even worse, an important and respected committee.
  - wudangmonk 2y ago
    The issue seems to be the C++ committee, never heard anyone complain about C's committee.
    - nicoburns 2y ago
      The C committee's big problem is being too conservative. But you don't hear too many complaints because people just use C++.
      - flohofwoe 2y ago
        I think the reason is more that the goals of the C committee align better with the goals of C programmers (which is a language that's a tool and not a playground for language designers). Unlike C++, C also remained a simple language (which is definitely a side effect of the "conservatism" of the C committee).
        edflsafoiewq 2y ago
        That's what he means; like the puddle which marvelously fits its own hole, anyone for whom C in all of its aspects is not very close to optimal has long ago moved to another language.
      - rightbyte 2y ago
        Wait what? C's greatest strength IMO is that it hardly change at all. The mental overload of some small things added every 10th year and in use 20 years later is how I like it.
vlovich123 2y ago
What’s upsetting is that the feedback has been provided before that the standards body and compiler devs aren’t communicating with build systems folks when modules were first being standardized. Looks like not a lot has changed.
- Galanwe 2y ago
  I don't know of anyone happy with the C++ standard committee. Some parts of the new std features were copied from Python, which is good, a semantic and naming a lot of people already know. Yet some weirdo decided to half copy the naming scheme of APL and we end up with functions named "iota". Seriously. What. The. Don't get me started on features added in C++17 and deprecated 2 versions later. Don't ask also why they decided to rename universal references by forwarding references while the former makes sense and the latter none, just because they were pissed to not have named it first. All I want now, for the future of C++, is a Circle-like approach. I want to be able to selectively disable and replace legacy behavior and syntax, per file, so that at last we can have the language evolve.
  - pjmlp 2y ago
    > Don't get me started on features added in C++17 and deprecated 2 versions later They are currently suffering from not having proper field experience on many features before setting them into the standard.
    - vlovich123 2y ago
      And it’s not like they’re unfamiliar with how rust is doing things and yet they still choose to cling to clearly broken processes. ISO is a mess.
      - pjmlp 2y ago
        The problem is that isn't that easy to get out of ISO, and only a vocal minority cares about this.
dgellow 2y ago
(2023)
josefx 2y ago
> At first you run into all the usual tool roughness that you'd expect from new experimental features. For example, GCC generates dependency files that don't work with Ninja. To sum the link behind that statement up: The default output format used by the GNU Compiler is for GNU Make, a ninja compatible format can be generated but is not the default. The blog author considers this default broken, GNU developers consider this behavior correct.
- IshKebab 2y ago
  Typical GNU attitude. I'd say the author is correct unless there is a good reason to make it incompatible with Ninja.
  - dlundqvist 2y ago
    If it can detect type of compiler, I would think it can pass the correct flags to it. If that is all it is.
    - deleted 2y ago
      [deleted]
  - bonzini 2y ago
    Ninja is supposed to read dependency files "in Makefile syntax", so I would at least try to understand the issue before placing blame.
    - IshKebab 2y ago
      I can see you google it, found this link: https://ninja-build.org/manual.html#_depfile https://ninja-build.org/manual.html#_depfile and then got as far as reading the first sentence: > gcc (and other compilers like clang) support emitting dependency information in the syntax of a Makefile. But didn't quite make it to the second paragraph: > (Ninja only supports the limited subset of the Makefile syntax emitted by compilers.) Obviously Ninja can't parse and execute full Makefile syntax because then it would just be Make. But the actual information in depfiles doesn't require the full Makefile syntax; only a subset of it. And compilers output that subset, so Ninja can read it. The issue is it's a de facto subset - a "depfile syntax" that exists by convention but the GNU authors have broken this format. As I said, unless there's a good reason to break it, that's just shitty behaviour.
codelikeawolf 2y ago
> In other words in order to be able to compile this source file you first need to parse the source file and all included sources until you hit the export declaration and then throw away the result. I might be completely wrong here, but reading this reminded me of barrel files [1] in JS/TS projects, which are an absolute nightmare. I put together a little visualization a while back that demonstrated how much extra work is required to resolve the module graph if you forward exports, and it is orders of magnitude more than just importing something directly from the file where it is exported. If this is also the case for C++ modules, I'd like to buy the author a beer or six. [1] https://basarat.gitbook.io/typescript/main-1/barrel https://basarat.gitbook.io/typescript/main-1/barrel
- evanpurkhiser 2y ago
  Would love to see the visualization if you have it on hand!
  - codelikeawolf 2y ago
    Unfortunately, there might be too much IP in the code (it was for a team presentation at a company I contract for). I could probably scrub that out and put it up on GitHub, though.
dang 2y ago
Discussed (a bit) at the time: The road to hell is paved with good intentions and C++ modules - https://news.ycombinator.com/item?id=37891898 https://news.ycombinator.com/item?id=37891898 - Oct 2023 (2 comments)
deleted 2y ago
[deleted]
vitus 2y ago
Do we even have a second C++20-standard-compliant module implementation? Last I checked, it was just MSVC -- GCC and Clang were like 90% but missing a few features. Looks like still no, although Intel provides yet another partial implementation: https://en.cppreference.com/w/cpp/compiler_support/20 https://en.cppreference.com/w/cpp/compiler_support/20

85 points by fanf2 2y ago

db48x 2y ago

I stopped paying attention to C++ stuff years ago. Did any of these problems ever get fixed?

kevindamm 2y ago
If by fixed you mean "elaborate measures for specificity and saliency but still the same order of undefined behavior & bugbears" then yeah? It's gotten a lot more... more.
- db48x 2y ago
  Oh good, I’d hate for the industry to have stagnated :P

Bostonian 2y ago

The author says Fortran modules are easier for build systems to handle: 'A Ninja file is static, thus we need to know a) the compilation arguments to use and b) interdependencies between files. The latter can't be done for things like Fortran or C++ modules. This is why Ninja has a functionality called dyndeps. It is a way to call an external program that scans the sources to be built and then writes a simple Makefile-esque snippet describing which order things should be built in. This is a simple and understandable step that works nicely for Fortran modules but is by itself insufficient for C++ modules.' ... 'Let's start by listing the requirements [for modules]: All command lines used must be knowable a priori, that is, without scanning the contents of source files Any compiler may choose to name its module files however it wants, but said mapping must be knowable just from the compiler name and version, i.o.w. it has to be documented Changing source files must not cause, by itself, a reconfiguration, only a rescan for deps followed by a build of the changed files The developer must not need to tell which sources are which module types in the build system, it is to be deduced automatically without needing to scan the contents of source files (i.e. by using the proper file extension) Module files are per-compiler and per-version and only make sense within the build tree of a single project If module files are to be installed, those are defined in a way that does not affect source files with modules that do not need installing. Fortran modules already satisfy all of these.'

kergonath 2y ago
Fortran modules are great for developers but a bit of a pain when setting up a build system (even though it’s quite straightforward now, it still requires a first pass with something like makedepf90 or one of its many equivalents). I find it quite a feat that C++ modules, while benefitting from the experience of many languages, ended up so broken. Also, I never really liked cmake but this looks really bad.
- kazinator 2y ago
  If your language has modules, like in the syntax and semantics, and you're still requiring external tools to hunt down dependencies, your module system must somehow be a failure. The idea behind modules is that since they are part of the language, the compiler for the language understands modules and follows the dependencies. All module-related tooling, like incremental builds, is hooked through the compiler. You don't use make or external dependency generators. If someone does need a dependency generator for whatever reason, the compiler has to provide that function. You give the compiler the proper option to spit out dependencies, point it at the root module of the program, and let it do its thing.
  - kergonath 2y ago
    > If your language has modules, like in the syntax and semantics, and you're still requiring external tools to hunt down dependencies, your module system must somehow be a failure. No argument from me :) > The idea behind modules is that since they are part of the language, the compiler for the language understands modules and follows the dependencies. All module-related tooling, like incremental builds, is hooked through the compiler. I am not sure I follow. The compiler still compiles one file each time it is run, so we need a first pass to determine the dependency tree and what can be compiled in which order, right? Even if that first pass uses the compiler itself (doing only enough parsing to see what is needed). > You give the compiler the proper option to spit out dependencies, point it at the root module of the program, and let it do its thing. Nowadays, gfortran can generate Makefile fragments with the dependencies, but it was not always the case. This is why there are a handful of small tools to do it and makedepf90 is one of them (it’s a small Fortran program with a rudimentary parser). There are also other compilers that don’t do it properly.

awinter-py 2y ago

ughghgh every proposal for C/C++ modules feels like it's by someone who has not used a working module system and doesn't believe it's truly possible to create

IshKebab 2y ago
It's because they're trying to make a module system that is compatible with decades of development based around `#include`. It's nowhere near as simple as making a new language with modules. I almost feel like they should give up. IMO Rust and Zig have made C/C++ a legacy language. Except in some specific areas where Rust doesn't have mature libraries (GUIs & games mostly) you'd be silly to start a new project in C++. Migrating existing C++ projects to modules likely isn't worth the enormous effort, and the days of starting new C++ projects are numbered... so why bother?
- deleted 2y ago
  [deleted]
- _hl_ 2y ago
  There is still an unfathomably huge landscape of problems where C++ is the more mature, “right” choice over rust for new projects. Just from my (extremely limited and narrow) personal experience: - scientific/numerical computing (super mature) - domain-specific high performance computing (C++ is for better or worse extremely flexible) - weird CUDA use cases (feels very native in C++) - game dev as you mention - probably much much more C++ to me feels more like a meta-language where you go off to build your own kind of high performance DSL. Rust is much more rigid, which is probably a good thing for many “standard” use cases. But I’m sure there will always be a very very long tail of these niche projects where the flexibility comes in handy.
  - zrules 2y ago
    Opencv while not niche is also one of them. It’s definitely preventing me from going full in on rust. Probably for a while. This Reddit comment captures my reasons pretty well: https://www.reddit.com/r/rust/comments/1arys3z/comment/kqoak17/ https://www.reddit.com/r/rust/comments/1arys3z/comment/kqoak...

Conscat 2y ago

How is it that D and Objective-C++ came up with modules that work and can be used many years ago, yet C++ with much greater stakes fumbled it on almost every level?

marcosdumay 2y ago
Because the C family is guided by a committee. Even worse, an important and respected committee.
- wudangmonk 2y ago
  The issue seems to be the C++ committee, never heard anyone complain about C's committee.
  - nicoburns 2y ago
    The C committee's big problem is being too conservative. But you don't hear too many complaints because people just use C++.
    - flohofwoe 2y ago
      I think the reason is more that the goals of the C committee align better with the goals of C programmers (which is a language that's a tool and not a playground for language designers). Unlike C++, C also remained a simple language (which is definitely a side effect of the "conservatism" of the C committee).
      - edflsafoiewq 2y ago
        That's what he means; like the puddle which marvelously fits its own hole, anyone for whom C in all of its aspects is not very close to optimal has long ago moved to another language.
    - rightbyte 2y ago
      Wait what? C's greatest strength IMO is that it hardly change at all. The mental overload of some small things added every 10th year and in use 20 years later is how I like it.

vlovich123 2y ago

What’s upsetting is that the feedback has been provided before that the standards body and compiler devs aren’t communicating with build systems folks when modules were first being standardized. Looks like not a lot has changed.

Galanwe 2y ago
I don't know of anyone happy with the C++ standard committee. Some parts of the new std features were copied from Python, which is good, a semantic and naming a lot of people already know. Yet some weirdo decided to half copy the naming scheme of APL and we end up with functions named "iota". Seriously. What. The. Don't get me started on features added in C++17 and deprecated 2 versions later. Don't ask also why they decided to rename universal references by forwarding references while the former makes sense and the latter none, just because they were pissed to not have named it first. All I want now, for the future of C++, is a Circle-like approach. I want to be able to selectively disable and replace legacy behavior and syntax, per file, so that at last we can have the language evolve.
- pjmlp 2y ago
  > Don't get me started on features added in C++17 and deprecated 2 versions later They are currently suffering from not having proper field experience on many features before setting them into the standard.
  - vlovich123 2y ago
    And it’s not like they’re unfamiliar with how rust is doing things and yet they still choose to cling to clearly broken processes. ISO is a mess.
    - pjmlp 2y ago
      The problem is that isn't that easy to get out of ISO, and only a vocal minority cares about this.

dgellow 2y ago

(2023)

josefx 2y ago

> At first you run into all the usual tool roughness that you'd expect from new experimental features. For example, GCC generates dependency files that don't work with Ninja. To sum the link behind that statement up: The default output format used by the GNU Compiler is for GNU Make, a ninja compatible format can be generated but is not the default. The blog author considers this default broken, GNU developers consider this behavior correct.

IshKebab 2y ago
Typical GNU attitude. I'd say the author is correct unless there is a good reason to make it incompatible with Ninja.
- dlundqvist 2y ago
  If it can detect type of compiler, I would think it can pass the correct flags to it. If that is all it is.
  - deleted 2y ago
    [deleted]
- bonzini 2y ago
  Ninja is supposed to read dependency files "in Makefile syntax", so I would at least try to understand the issue before placing blame.
  - IshKebab 2y ago
    I can see you google it, found this link: https://ninja-build.org/manual.html#_depfile https://ninja-build.org/manual.html#_depfile and then got as far as reading the first sentence: > gcc (and other compilers like clang) support emitting dependency information in the syntax of a Makefile. But didn't quite make it to the second paragraph: > (Ninja only supports the limited subset of the Makefile syntax emitted by compilers.) Obviously Ninja can't parse and execute full Makefile syntax because then it would just be Make. But the actual information in depfiles doesn't require the full Makefile syntax; only a subset of it. And compilers output that subset, so Ninja can read it. The issue is it's a de facto subset - a "depfile syntax" that exists by convention but the GNU authors have broken this format. As I said, unless there's a good reason to break it, that's just shitty behaviour.

codelikeawolf 2y ago

> In other words in order to be able to compile this source file you first need to parse the source file and all included sources until you hit the export declaration and then throw away the result. I might be completely wrong here, but reading this reminded me of barrel files [1] in JS/TS projects, which are an absolute nightmare. I put together a little visualization a while back that demonstrated how much extra work is required to resolve the module graph if you forward exports, and it is orders of magnitude more than just importing something directly from the file where it is exported. If this is also the case for C++ modules, I'd like to buy the author a beer or six. [1] https://basarat.gitbook.io/typescript/main-1/barrel https://basarat.gitbook.io/typescript/main-1/barrel

evanpurkhiser 2y ago
Would love to see the visualization if you have it on hand!
- codelikeawolf 2y ago
  Unfortunately, there might be too much IP in the code (it was for a team presentation at a company I contract for). I could probably scrub that out and put it up on GitHub, though.

dang 2y ago

Discussed (a bit) at the time: The road to hell is paved with good intentions and C++ modules - https://news.ycombinator.com/item?id=37891898 https://news.ycombinator.com/item?id=37891898 - Oct 2023 (2 comments)

deleted 2y ago

[deleted]

vitus 2y ago

Do we even have a second C++20-standard-compliant module implementation? Last I checked, it was just MSVC -- GCC and Clang were like 90% but missing a few features. Looks like still no, although Intel provides yet another partial implementation: https://en.cppreference.com/w/cpp/compiler_support/20 https://en.cppreference.com/w/cpp/compiler_support/20