16 ms·

Killing the C++ Modules Technical Specification

128 points by jahnu 9y ago

twoodfin 9y ago
I love strongly worded technical arguments. I am not deeply familiar with the Modules TS, but couldn't each environment solve the module name <-> translation unit mapping problem in its own way? For systems that can't keep reliable track of changes to the underlying representation (i.e., files) that would mean a fairly strict naming scheme. The mapping doesn't have to be maintained by the compiler. C/C++ already depend on a suite of more-or-less independent tools to produce runnable software. I don't see how this problem conceptually is much worse than what a linker has to do.
- humanrebar 9y ago
  > I don't see how this problem conceptually is much worse... I'm asking the same question to myself. There's a bit in the article about it being slower than the status quo, but I'd like to see actual benchmarks before making pronouncements. If that hypothesis is true, then surely the TS will fail before it makes it into C++20 proper.
humanrebar 9y ago
> Granted, the Rust community is typically wrong about nearly everything when comparing their language to C++... What percentage of the Rust community is a professional C++ developer, either now or recently in the past? I was under the impression that "nearly all" was a fair estimate. If that's true, does that mean that the C++ community itself is "typically wrong" about their language?
- gpderetta 9y ago
  It does not follow. It could be that the subset of C++ programmers that are "typically wrong" about C++ are more likely to use rust.
  - humanrebar 9y ago
    I think the rest of the article elaborates on why I asked that. If only a subset of the C++ community is "typically wrong", that subset involves quite a lot of people, including those on the standards committee, at CppCon, and on /r/cpp.
- flohofwoe 9y ago
  It seems most Rust programmers are switching over from Python (but C and C++ are next, followed by Java and Javascript, look near the end): https://blog.rust-lang.org/2016/06/30/State-of-Rust-Survey-2016.html https://blog.rust-lang.org/2016/06/30/State-of-Rust-Survey-2... There is no "right" or "wrong" C++, there's also no single "C++ community". I'd say that C++ isn't even a programming language, it's more like a meta-language to build your own language. Whether that's good or bad is up for discussion, but (a) it gives a lot of freedom, (b) it invites tinkering, (c) it wastes a massive amount of time when trying to communicate with C++ coders from other confessions, and (c) it makes it hard to integrate C++ libraries into C++ projects. edit: confession => 'denomination' seems to be the correct English term
  - Koshkin 9y ago
    > a meta-language to build your own language All (good) programming languages are like that. In general, any API, any library that you develop is a language. I agree that C++ provides powerful tools of abstraction that make these languages easier to use (compared to what can be accomplished in C or Go, for example).
    - rtpg 9y ago
      My feeling has been that C++ has a special degree of complexity beyond most languages. Many terms come up in the "day-to-day" that wouldn't come up in other languages (lvalue/rvalue, move semantics), and so much can be overwritten through operators. I think the only other language with a similar feel is Scala. It's the programming language equivalent to Magic The Gathering: half the fun is just in the mechanics of it all
      - deleted 9y ago
        [deleted]
      - Koshkin 9y ago
        > complexity beyond most languages I think complexity of C++ is merely a (partial) reflection of complexity of programming. For example, move semantics that you have mentioned is not specific to C++, and I find it nice when a language offers an explicit formalism for it.
- _pmf_ 9y ago
  > Granted, the Rust community is typically wrong about nearly everything when comparing their language to C++... Well, where they're right is in having a modest and non-condescending attitude, which is something that C++ pundits should consider adopting.
  - ovao 9y ago
    You're generalizing about a group (C++ pundits) in the same way that the author generalizes about another group (the Rust community). There are C++ experts who display modesty, just as there are members of the Rust community who display modesty. There is no need to try to retaliate against one person's words by hurling generalized insults against an entire group.
- Manishearth 9y ago
  More like half. A large fraction of the Rust community is folks coming from non-systems languages who saw Rust as a gateway to systems. But we have both, and most of the folks doing comparisons with C++ do tend to be actual C++ devs (because if you're coming from Python or Ruby and haven't done C++ you really ... can't? compare with C++?). At least within the community folks pretty strongly care about misrepresenting other languages so if stuff is inaccurate it gets called out; and for most "rust vs foo" blog posts I've only rarely seen glaring mistakes a few times. Small mistakes are common but those are common to any kind of post. I took the comment in the post as a bit of hyperbole and probably referring to subjective bits that Rust+C++ & only-C++ programmers tend to disagree on. In general most of the vocal Rust folks who talk about C++ talk about C++ because they have lived it. shrug
  - steveklabnik 9y ago
    It can get more subtle than that too, though. For example, see this recent discussion I had on this topic: https://www.reddit.com/r/programming/comments/72p472/why_did_we_choose_rust_over_golang_or_cc_to/dnlp9m7/ https://www.reddit.com/r/programming/comments/72p472/why_did...
    - SAHChandler 9y ago
      That post in particular was a subject of discussion at CppCon for a hot second and is partially why I mentioned the Rust thing. There are other things as well, but I don't want to rake people if it isn't necessary. I pay attention to the Rust subreddit and I see enough misconceptions about C++ and how it works in various comments that it helped reinforce my choice to include that snarky statement. I do understand why so many C++ programmers at CppCon thought that Rust can be dismissed as a fad because of its community's understanding of C++. That said, this sort of parroting about how C++ works and why it is "bad" is everywhere and you can even see it in the comments on this very submission (such as mention of std::variant and implicit conversion to bool. That specific issue is easily remedied but wasn't part of the specification. It's been noted as a defect and will be fixed "soon"?) I still chuckle when randos make arguments like "C++ can't be parsed with an LALR parser", or "virtual functions are more expensive than function pointers", as if the former is the end of the world, and the latter is a checkmate that will suddenly make my knees week, arms heavy. There's code in my editor, and it's just spaghetti.
- sunfish 9y ago
  One cause of this phenomenon is both sides sometimes oversimplifying a complex situation in different directions. When people talk about "C++", they may mean: - "C++ as it theoretically could be in when C++20 or so is used pervasively" - "C++ as it can be when certain C++17 features are used pervasively" - "C++ as it is often used in practice today" (and this is still oversimplifying more nuanced realities). It's very common for people to make observations about "C++" which are wrong on one sense and right in another.
- epage 9y ago
  This abrasive comment from the author really put me off from the piece. Doesn't help that I first read berium's reddit post which disagrees [1]. He seems to be involved in build2 which has implemented support for Modules which garners more of my trust in this conversation. Coming from the C++ world and being fairly knowledgeable about the language (not a full language lawyer but someone people usually turn to) and learning Rust, I find it rare for there to be bad comments from the Rust folks. I don't understand where this is coming from. Now for my own judgemental comment. I see a trend of people in /r/cpp and other places ignoring the differences (like safety) or misrepresent Rust. To be clear, I fully accept people having differences in priority (how much safety needs to be compiler-enforced). Its people who don't recognize the difference that I'm bothered by. Rust is also foreign enough that a trivial glance can be misleading like assuming exceptions are the equivalent of panics because someone got caught up on them having a similar implementation (stack unwinding by default) when in reality, Result is more what people should look at (transport information, ?-operator for unwinding, etc). [1] https://www.reddit.com/r/cpp/comments/75eqal/millennials_are_killing_the_modules_ts/do5r9m3/?st=j8navyeo&sh=f7b886bd https://www.reddit.com/r/cpp/comments/75eqal/millennials_are...
gpderetta 9y ago
"The compiler must not become a build system" does not prevent the compiler generating local dependency info. In fact this is already possible today as many compilers (preprocessors really) are capable of generating header dependency info. You will need a similar mode were the compiler does only minimal parsing of the source file and generate a dep file containing the list of modules defined in this file and the list of modules imported by this file. The build system can then collet all these dep files and reconstruct the full dependency graph. Given the right format, you could probably feed them directly to make.
- twoodfin 9y ago
  This reminds me a little bit of the C++ compilers (HP's?) that would build a database of instantiated STL types to avoid rebuilding std::basic_string<char> for every translation unit.
  - colanderman 9y ago
    In fact, starting with C++11, you can do this in your projects. Make a common header file with contents like this: extern template class std::basic_string<char>; extern template class std::vector<int>; // etc... and a corresponding .cc file like this: template class std::basic_string<char>; template class std::vector<int>; // etc... The template instantiations you list will be compiled in that .cc's object file. Anyone who includes the header will assume those templates were instantiated elsewhere. Net effect is the listed templates are only compiled once. (Of course this doesn't alleviate the problem of parsing the template header files over and over.) See the section on explicit instantiation here: http://en.cppreference.com/w/cpp/language/class_template http://en.cppreference.com/w/cpp/language/class_template
    - corysama 9y ago
      Now I'm wondering what is a good way to measure my project's top 100 redundantly instantiated templates.
wyldfire 9y ago
> Effectively, without support for build tools, modules are effectively dead in the water. ... > "We'll make the build systems understand". This seems to be a critical point that I agree with: you can't divorce these two, their fates must be bound to one another. Hopefully the build systems can be easily modified to accommodate. Rust community folks might be wrong about c++ but they're right about Rust. Rust had the luxury of not being bound to any legacy specifications and it wasn't unreasonable for them to write their own build+dependency/package management tool. The net effect of them having done that means that independent of all the other great things Rust brings, much of the tedium of developing with C/C++ is gone. Make no mistake, C++ standard committee: the bar is high and Rust has set the bar.
- pjmlp 9y ago
  > the bar is high and Rust has set the bar. Modules were introduced in the 70's as concept, Mesa being one of the first languages to use them, followed by CLU and ML.... The issue with lack of modules in C++, was that Bjarne had the compromise C++ should fit into C toolchains used by AT&T, and later all C shops. EDIT: I get Rust is quite cool, but maybe a bit of programming language research is also welcomed.
  - humanrebar 9y ago
    I think "the bar" is talking about languages a developer might write her next application in. Mesa and CLU aren't on the list. OCaml might be, depending.
    - pjmlp 9y ago
      Except Rust modules (which are the subject here) are anything but innovative, there are plenty of languages since Mesa, CLU and ML came to the world with equally or better module systems.
      - wyldfire 9y ago
        Note that I didn't mention innovation or novelty or originality. I just said that Rust "set the bar." To respond to your objection I'd just clarify by qualifying it better and refer to the "systems programming" or whatever domain C/C++ and Rust both belong to.
cousin_it 9y ago
Forget C++ for a moment. In your dream language, if one source file uses functionality from another, where should that fact be specified? 1) In the source file 2) In the makefile 3) In both It seems to me that the only sane option is (1) with a project-wide or system-wide set of "roots". Why is anyone in favor of (2) or (3)?
- mpweiher 9y ago
  4.) In the source file where the two are composed. In general, modules should not import their dependencies, but have them provided to them by their common parent. See Newspeak's module system as an example.
  - cousin_it 9y ago
    Radical! But doesn't that mean the first module can't be typechecked in isolation? I see Newspeak is dynamically typed so maybe it's not a problem there, but what's the right solution for statically typed languages then?
    - qznc 9y ago
      Specify an interface. Alternatively, infer the interface and type check that in the composing module.
    - mpweiher 9y ago
      Newspeak is intended to be statically typed, presumably optionally/gradually. I think the implementation is missing right now, though the syntax already allows type annotations. As the other commenter wrote, the answer is that modules can never "import" concrete modules, only interfaces. And that means some extra effort, though having type-inference assist with the construction of that module should help. Definitely something that needs figuring out. IMHO, explicit composition is something we really need to figure out in general, and it should solve the weird mix of linguistic and extra-linguistic mechanisms we have now.
- jmh530 9y ago
opk 9y ago
I'm glad to see this point being made because it covers exactly what worried me when I first saw this C++ modules specification. At work, I maintain the build system for some mixed C++ and Fortran 90 code and prepocessor includes really are a case of worse is better in my view. Generating build dependencies for Fortran 90 modules is a horrible mess and can easily get broken. And I can't really see that modules solve many real problems.
slezyr 9y ago
> Build tools will parse source files over time and keep track of which modules are where and which translation unit needs what. Isn't this is what cmake does? IIRC it already parses the headers.
- humanrebar 9y ago
  In vanilla cmake, you declare an ordered list of include directories. But the point about "how does it work now?" is fair. I'm confused why a module search path is any worse than an include search path.
  - lultimouomo 9y ago
    This is not related to the search path, but to determining which files need to be rebuilt when a file changes. The idea is that when you change a module, you might need to recompile all the modules that include it, since its interface might have changed. What I don't get in this criticism is that you have this exact problem right now with headers, and it has been solved by the preprocessor outputting dependency information (based on #include directives) for the build system to read. Why can't the same be done with modules (of course it might not be the preprocessor that does it, but the compiler or a specific tool)?
    - humanrebar 9y ago
      I think I'm catching on. With a '#include', you give a file name. With an 'import', you give a module name, which doesn't necessarily map to a file name. So the problem could be solved by saying 'import foobar;' maps to a file (on a search path, maybe) named exactly "foobar". But then we need to work out how subdirectories and module names map to each other. How do you import "foo/bar.ixx"? "foo.bar"? "<foo/bar>"? Though all that seems technically solvable to me. Maybe there's just difficulty in designing this in the context of a committee.
  - mikepurvis 9y ago
    I think it's worse because even with a list of directories to look in, you still don't know what _file_ contains the module you're looking for. Literally, you need to open up each file in each directory of the specified search path and possibly even _run it through the preprocessor_ in order to determine if it's the module you're looking for. And assuming you don't want to do all this every time, then you need to maintain a cache somewhere, and all of this logic has to be part of the compiler rather than the build system. Anyway, Java dealt with a bunch of this by enforcing that the file naming and directory hierarchy would match the class hierarchy.
_pmf_ 9y ago
Why do C++ developers make it so, so, so, so hard on themselves and the unfortunate poor souls like me who have to interact with their products?
- Koshkin 9y ago
  Perhaps, they just want to share with everyone their experience interacting with the language of their choice? (I am joking, but the complexity of the language may, in fact, reflect in the relative complexity of the APIs, say.)
simias 9y ago
I wonder if C++ is headed for COBOL-ification. This article is very interesting but doesn't really give any hope for success in implementing a modern module system in C++. I feel like the only way to have truly useful, simple and intuitive modules in C++ would involve breaking a certain amount of backward compatibility and force a few constraints (project layout, file naming scheme, build system, ...) on the user. But clearly that's not really in C++'s DNA and you risk ending up with something that's not quite C++ while at the same time not having the simplicity and elegance of modern languages that have been designed from scratch without all the baggage C++ carries. C++ won't go away any time soon, that's for sure. But when I read articles like TFA or for instance https://bitbashing.io/std-visit.html https://bitbashing.io/std-visit.html my gut reaction is always that maybe if you get frustrated with C++'s extreme complexity, historical baggage and user-unfriendliness you should consider moving to something else instead of trying to bolt on even more exotic features on a language which is already crumbling under the weight of all the features and programing styles it already supports. I know I did. As an example I stumbled upon this std::visit link by perusing the top posts of reddit's /c/cpp (linked by TFA), here's the discussion: https://www.reddit.com/r/cpp/comments/703k9k/stdvisit_is_everything_wrong_with_modern_c/ https://www.reddit.com/r/cpp/comments/703k9k/stdvisit_is_eve... There's an interesting thread about how: variant<string, int, bool> mySetting = "Hello!"; Doesn't do what one might thing it should do. The variant ends up holding a `true` boolean instead of a string. Why? >char const* to bool is a standard conversion, but to std::string is a user-defined conversion. Standard conversion wins. If you know C++'s history and its C heritage it makes sense but it sure as hell doesn't make me want to use C++ for my next project.
- humanrebar 9y ago
  C++ is pretty close to treating unqualified string literals the same as native pointers. That is, there for backwards compatibility and for rare edge cases, but generally discouraged. I predict that, given a few years, we'll start adding checks to linters and static analyzers to warn about using them because of the ambiguity and implicit conversion issues. There is already a std::string liberal. Better: MyVariant mySetting = "Hello!"s; But std::string_view more closely models a string literal. std::string is more of a string buffer. So even better: MyVariant mySetting = "Hello!"sv;
mattnewport 9y ago
I'm very familiar with C++, passingly familiar with the modules proposal and unfamiliar with Rust and this article leaves me puzzled exactly what the author expects / wants a modules proposal to do instead. My perspective as a C++ developer has been that modules have one primary purpose, which is to improve build times by eliminating the need to parse huge amounts of C++ in recursively included header files when compiling a compilation unit. If that's all a modules proposal accomplishes I'm fine with that. A modules proposal that doesn't accomplish that is useless. Build tools generally already parse .cpp files for includes to figure out header file dependencies for a .cpp file and have to combine the filenames extracted with information from the build system on include search paths to turn those into dependencies on specific concrete files. That involves information that lives in the build system and not in the code. Modules will require something similar from build tools but as with parsing includes the rules about imports (must be before actual code, must be alone on a line) parsing out the imports is trivial and cheap compared to recursively parsing the full code in header dependencies. So I guess I don't quite understand the problem. Modules are an improvement over headers for build times but they are not going to do everything that module systems perhaps do in some other languages because they have to be adopted incrementally into the many different existing build systems used in C++ code bases (which live outside the standard). Perhaps my ignorance of Rust means I'm missing what the author wants in C++ that's different from the current proposal.
- coliveira 9y ago
  The author is just trying to prove that C++ modules cannot be like Java or Python modules, which apparently some people don't get. My view is that C++ programmers already know this, since the language will not change suddenly just because modules have been introduced. The idea that the build system will take care of that is perfectly fine for normal C++ users.
  - 1ris 9y ago
    >The author is just trying to prove that C++ modules cannot be like Java or Python modules, which apparently some people don't get. I don't think he trys that and i don't think it's true. The committee doesn't want to that, but I don't see why it would be possible by extending the language. Introducte a module - keyword, that works pretty much like a namespace keyword. Force the file name and path to match the name a la java. Introduce a mechanism a la "using namespace" that does import the namespace (like #include header.h) of the module and tells the build system what to build. Neither needs to become the other to do this if we forbid preprocessor commands and if constexprs around that. Allow attributes to specify visibility. Of course it's going to be a way more complicated, but i can't see how it would be impossible. I think c++ devs just don't want that.
namelost 9y ago
I admit I know nothing about C++ modules proposals, but the post talks about an "acyclic graph of translation units". Does this imply that modules can't import one another? What happens when two .cpp files want to do the equivalent of including each other's header files?
rwmj 9y ago
They should really look at how OCaml modules work, since they solve all the same problems already, with desirable properties like separate compilation and easily discoverable mapping of module name to source file. The drawback of OCaml modules is that all dependencies must form an acyclic graph, which can be a pain sometimes although there are well understood workarounds.
- zem 9y ago
  the other huge problem is the poor support for hierarchical namespaces at the package level. nothing like linking your code with a library and having things break because you have a module name clash
  - rwmj 9y ago
    Yes, if you give all your modules a generic "toplevel" name like "Utils". However if you use underscores to namespace things you get something a lot like a hierarchy. You just need to have the discipline to call your modules "Myproject_Utils". Now I didn't say that OCaml has a perfect module system, although some of the first-class module features are stunning, but my point is that it solves a lot of the problems that this C++ guy is complaining about.
    - zem 9y ago
      right, that's the standard workaround, but it really seems like something the compiler should be doing for you. i find it a very surprising wart in an in general very well-designed module system.
- DoofusOfDeath 9y ago
  > The drawback of OCaml modules is that all dependencies must form an acyclic graph, which can be a pain sometimes although there are well understood workarounds. Can you elaborate on that? It's hard to imagine a sane build system where the dependency graph contains cycles.
  - 9y ago
makecheck 9y ago
Perhaps it’s made complex by searching for perfection when really you just need something that makes a high percentage of builds better. You’d be crazy to create a final release of something without starting from a “clean” state, which means they DO NOT need a module implementation with 100% accuracy in all of C++’s asinine corner cases. I wouldn’t trust that accuracy even if they claimed it, I’d “clean” anyway. This means that compilation speedup just has to ensure that MOST incremental rebuilds perform better without adding insane development overhead (such as having to respecify things). If 60% of the time my incremental rebuilds are faster, I’d say that is more than enough to justify a C++1x release. They need to just move forward.
jmgao 9y ago
This seems reminiscent of export templates, which were a resounding success: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2003/n1426.pdf http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2003/n142...
danblick 9y ago
I'm confused; isn't the solution to this problem "gcc -M"?
yahna 9y ago
dumb title. nitpicking. equating innocent, if terse, comments with threats. Why am I reading what this person has to say?
slavik81 9y ago
All I know is that C++ build times are painfully slow for large projects. Even if you spend a lot of time and effort on making them as fast as possible, they're still not good. #include just results in a tremendous amount of code in each and every source file. Even looking at a 6 line program, we see that after the preprocessor runs, the compiler is given 18,162 lines to process. $ cat > main.cpp <<EOF #include <iostream> int main() { std::cout << "Hello World!" << std::endl; return 0; } EOF $ gcc -E main.cpp | wc -l 18162 That is still nearly instantaneous, but it only gets worse from there and it happens to every source file in the project. I just took at look at a 1,848 line source file I'm working with, and it expands out to 147,054 lines after preprocessing. The current state of things is awful and modules are by far the most important feature for the future of C++. We need module support in our build tools and we need them to be fast. Should the TS be delayed to see if the Clang idea is better? I'm curious to know what they're doing, but that's a lot to ask for. The TS is effectively a beta, and pushing back its start means less time in beta before the C++20 standard is finalized. There's a lot of work that's ultimately going to be built on top of this so getting it right is vital, but the clock is ticking.
- pjmlp 9y ago
  CppCon 2016 has some Google presentations about how they use clang modules (aka module maps). "Deploying C++ modules to 100s of millions of lines of code" https://www.youtube.com/watch?v=dHFNpBfemDI https://www.youtube.com/watch?v=dHFNpBfemDI "There and Back Again: An Incremental C++ Modules Design" https://www.youtube.com/watch?v=h1E-XyxqJRE&t=922s https://www.youtube.com/watch?v=h1E-XyxqJRE&t=922s
Const-me 9y ago
I’m unaware about the recent development, but MS has very module-like features in their version of C++ for decades already. For classic C++ they have #import directive (1) Recently, for windows store/UWP platforms, C++ can consume windows runtime components (2) Two things that allow these features to work are standardized ABI, and standardized type info format. I don’t think high-level modules are possible in C++ before ABI and type info are both standardized in a compiler-neutral way. Better yet, in a language neutral way, like MS did that: you can implement a COM object in a script language as a WSC file, and consume it from C++ using #import. [1] https://msdn.microsoft.com/en-us/library/8etzzkb6.aspx https://msdn.microsoft.com/en-us/library/8etzzkb6.aspx [2] https://docs.microsoft.com/en-us/cpp/windows/how-to-activate-and-use-a-windows-runtime-component-using-wrl https://docs.microsoft.com/en-us/cpp/windows/how-to-activate...

128 points by jahnu 9y ago

I love strongly worded technical arguments. I am not deeply familiar with the Modules TS, but couldn't each environment solve the module name <-> translation unit mapping problem in its own way? For systems that can't keep reliable track of changes to the underlying representation (i.e., files) that would mean a fairly strict naming scheme. The mapping doesn't have to be maintained by the compiler. C/C++ already depend on a suite of more-or-less independent tools to produce runnable software. I don't see how this problem conceptually is much worse than what a linker has to do.

humanrebar 9y ago
> I don't see how this problem conceptually is much worse... I'm asking the same question to myself. There's a bit in the article about it being slower than the status quo, but I'd like to see actual benchmarks before making pronouncements. If that hypothesis is true, then surely the TS will fail before it makes it into C++20 proper.

humanrebar 9y ago

> Granted, the Rust community is typically wrong about nearly everything when comparing their language to C++... What percentage of the Rust community is a professional C++ developer, either now or recently in the past? I was under the impression that "nearly all" was a fair estimate. If that's true, does that mean that the C++ community itself is "typically wrong" about their language?

gpderetta 9y ago
It does not follow. It could be that the subset of C++ programmers that are "typically wrong" about C++ are more likely to use rust.
- humanrebar 9y ago
  I think the rest of the article elaborates on why I asked that. If only a subset of the C++ community is "typically wrong", that subset involves quite a lot of people, including those on the standards committee, at CppCon, and on /r/cpp.
flohofwoe 9y ago
It seems most Rust programmers are switching over from Python (but C and C++ are next, followed by Java and Javascript, look near the end): https://blog.rust-lang.org/2016/06/30/State-of-Rust-Survey-2016.html https://blog.rust-lang.org/2016/06/30/State-of-Rust-Survey-2... There is no "right" or "wrong" C++, there's also no single "C++ community". I'd say that C++ isn't even a programming language, it's more like a meta-language to build your own language. Whether that's good or bad is up for discussion, but (a) it gives a lot of freedom, (b) it invites tinkering, (c) it wastes a massive amount of time when trying to communicate with C++ coders from other confessions, and (c) it makes it hard to integrate C++ libraries into C++ projects. edit: confession => 'denomination' seems to be the correct English term
- Koshkin 9y ago
  > a meta-language to build your own language All (good) programming languages are like that. In general, any API, any library that you develop is a language. I agree that C++ provides powerful tools of abstraction that make these languages easier to use (compared to what can be accomplished in C or Go, for example).
  - rtpg 9y ago
    My feeling has been that C++ has a special degree of complexity beyond most languages. Many terms come up in the "day-to-day" that wouldn't come up in other languages (lvalue/rvalue, move semantics), and so much can be overwritten through operators. I think the only other language with a similar feel is Scala. It's the programming language equivalent to Magic The Gathering: half the fun is just in the mechanics of it all
    - deleted 9y ago
      [deleted]
    - Koshkin 9y ago
      > complexity beyond most languages I think complexity of C++ is merely a (partial) reflection of complexity of programming. For example, move semantics that you have mentioned is not specific to C++, and I find it nice when a language offers an explicit formalism for it.
_pmf_ 9y ago
> Granted, the Rust community is typically wrong about nearly everything when comparing their language to C++... Well, where they're right is in having a modest and non-condescending attitude, which is something that C++ pundits should consider adopting.
- ovao 9y ago
  You're generalizing about a group (C++ pundits) in the same way that the author generalizes about another group (the Rust community). There are C++ experts who display modesty, just as there are members of the Rust community who display modesty. There is no need to try to retaliate against one person's words by hurling generalized insults against an entire group.
Manishearth 9y ago
More like half. A large fraction of the Rust community is folks coming from non-systems languages who saw Rust as a gateway to systems. But we have both, and most of the folks doing comparisons with C++ do tend to be actual C++ devs (because if you're coming from Python or Ruby and haven't done C++ you really ... can't? compare with C++?). At least within the community folks pretty strongly care about misrepresenting other languages so if stuff is inaccurate it gets called out; and for most "rust vs foo" blog posts I've only rarely seen glaring mistakes a few times. Small mistakes are common but those are common to any kind of post. I took the comment in the post as a bit of hyperbole and probably referring to subjective bits that Rust+C++ & only-C++ programmers tend to disagree on. In general most of the vocal Rust folks who talk about C++ talk about C++ because they have lived it. shrug
- steveklabnik 9y ago
  It can get more subtle than that too, though. For example, see this recent discussion I had on this topic: https://www.reddit.com/r/programming/comments/72p472/why_did_we_choose_rust_over_golang_or_cc_to/dnlp9m7/ https://www.reddit.com/r/programming/comments/72p472/why_did...
  - SAHChandler 9y ago
    That post in particular was a subject of discussion at CppCon for a hot second and is partially why I mentioned the Rust thing. There are other things as well, but I don't want to rake people if it isn't necessary. I pay attention to the Rust subreddit and I see enough misconceptions about C++ and how it works in various comments that it helped reinforce my choice to include that snarky statement. I do understand why so many C++ programmers at CppCon thought that Rust can be dismissed as a fad because of its community's understanding of C++. That said, this sort of parroting about how C++ works and why it is "bad" is everywhere and you can even see it in the comments on this very submission (such as mention of std::variant and implicit conversion to bool. That specific issue is easily remedied but wasn't part of the specification. It's been noted as a defect and will be fixed "soon"?) I still chuckle when randos make arguments like "C++ can't be parsed with an LALR parser", or "virtual functions are more expensive than function pointers", as if the former is the end of the world, and the latter is a checkmate that will suddenly make my knees week, arms heavy. There's code in my editor, and it's just spaghetti.
sunfish 9y ago
One cause of this phenomenon is both sides sometimes oversimplifying a complex situation in different directions. When people talk about "C++", they may mean: - "C++ as it theoretically could be in when C++20 or so is used pervasively" - "C++ as it can be when certain C++17 features are used pervasively" - "C++ as it is often used in practice today" (and this is still oversimplifying more nuanced realities). It's very common for people to make observations about "C++" which are wrong on one sense and right in another.
epage 9y ago
This abrasive comment from the author really put me off from the piece. Doesn't help that I first read berium's reddit post which disagrees [1]. He seems to be involved in build2 which has implemented support for Modules which garners more of my trust in this conversation. Coming from the C++ world and being fairly knowledgeable about the language (not a full language lawyer but someone people usually turn to) and learning Rust, I find it rare for there to be bad comments from the Rust folks. I don't understand where this is coming from. Now for my own judgemental comment. I see a trend of people in /r/cpp and other places ignoring the differences (like safety) or misrepresent Rust. To be clear, I fully accept people having differences in priority (how much safety needs to be compiler-enforced). Its people who don't recognize the difference that I'm bothered by. Rust is also foreign enough that a trivial glance can be misleading like assuming exceptions are the equivalent of panics because someone got caught up on them having a similar implementation (stack unwinding by default) when in reality, Result is more what people should look at (transport information, ?-operator for unwinding, etc). [1] https://www.reddit.com/r/cpp/comments/75eqal/millennials_are_killing_the_modules_ts/do5r9m3/?st=j8navyeo&sh=f7b886bd https://www.reddit.com/r/cpp/comments/75eqal/millennials_are...

gpderetta 9y ago

"The compiler must not become a build system" does not prevent the compiler generating local dependency info. In fact this is already possible today as many compilers (preprocessors really) are capable of generating header dependency info. You will need a similar mode were the compiler does only minimal parsing of the source file and generate a dep file containing the list of modules defined in this file and the list of modules imported by this file. The build system can then collet all these dep files and reconstruct the full dependency graph. Given the right format, you could probably feed them directly to make.

twoodfin 9y ago
This reminds me a little bit of the C++ compilers (HP's?) that would build a database of instantiated STL types to avoid rebuilding std::basic_string<char> for every translation unit.
- colanderman 9y ago
  In fact, starting with C++11, you can do this in your projects. Make a common header file with contents like this: extern template class std::basic_string<char>; extern template class std::vector<int>; // etc... and a corresponding .cc file like this: template class std::basic_string<char>; template class std::vector<int>; // etc... The template instantiations you list will be compiled in that .cc's object file. Anyone who includes the header will assume those templates were instantiated elsewhere. Net effect is the listed templates are only compiled once. (Of course this doesn't alleviate the problem of parsing the template header files over and over.) See the section on explicit instantiation here: http://en.cppreference.com/w/cpp/language/class_template http://en.cppreference.com/w/cpp/language/class_template
  - corysama 9y ago
    Now I'm wondering what is a good way to measure my project's top 100 redundantly instantiated templates.

wyldfire 9y ago

> Effectively, without support for build tools, modules are effectively dead in the water. ... > "We'll make the build systems understand". This seems to be a critical point that I agree with: you can't divorce these two, their fates must be bound to one another. Hopefully the build systems can be easily modified to accommodate. Rust community folks might be wrong about c++ but they're right about Rust. Rust had the luxury of not being bound to any legacy specifications and it wasn't unreasonable for them to write their own build+dependency/package management tool. The net effect of them having done that means that independent of all the other great things Rust brings, much of the tedium of developing with C/C++ is gone. Make no mistake, C++ standard committee: the bar is high and Rust has set the bar.

pjmlp 9y ago
> the bar is high and Rust has set the bar. Modules were introduced in the 70's as concept, Mesa being one of the first languages to use them, followed by CLU and ML.... The issue with lack of modules in C++, was that Bjarne had the compromise C++ should fit into C toolchains used by AT&T, and later all C shops. EDIT: I get Rust is quite cool, but maybe a bit of programming language research is also welcomed.
- humanrebar 9y ago
  I think "the bar" is talking about languages a developer might write her next application in. Mesa and CLU aren't on the list. OCaml might be, depending.
  - pjmlp 9y ago
    Except Rust modules (which are the subject here) are anything but innovative, there are plenty of languages since Mesa, CLU and ML came to the world with equally or better module systems.
    - wyldfire 9y ago
      Note that I didn't mention innovation or novelty or originality. I just said that Rust "set the bar." To respond to your objection I'd just clarify by qualifying it better and refer to the "systems programming" or whatever domain C/C++ and Rust both belong to.

cousin_it 9y ago

Forget C++ for a moment. In your dream language, if one source file uses functionality from another, where should that fact be specified? 1) In the source file 2) In the makefile 3) In both It seems to me that the only sane option is (1) with a project-wide or system-wide set of "roots". Why is anyone in favor of (2) or (3)?

mpweiher 9y ago
4.) In the source file where the two are composed. In general, modules should not import their dependencies, but have them provided to them by their common parent. See Newspeak's module system as an example.
- cousin_it 9y ago
  Radical! But doesn't that mean the first module can't be typechecked in isolation? I see Newspeak is dynamically typed so maybe it's not a problem there, but what's the right solution for statically typed languages then?
  - qznc 9y ago
    Specify an interface. Alternatively, infer the interface and type check that in the composing module.
  - mpweiher 9y ago
    Newspeak is intended to be statically typed, presumably optionally/gradually. I think the implementation is missing right now, though the syntax already allows type annotations. As the other commenter wrote, the answer is that modules can never "import" concrete modules, only interfaces. And that means some extra effort, though having type-inference assist with the construction of that module should help. Definitely something that needs figuring out. IMHO, explicit composition is something we really need to figure out in general, and it should solve the weird mix of linguistic and extra-linguistic mechanisms we have now.
jmh530 9y ago

opk 9y ago

I'm glad to see this point being made because it covers exactly what worried me when I first saw this C++ modules specification. At work, I maintain the build system for some mixed C++ and Fortran 90 code and prepocessor includes really are a case of worse is better in my view. Generating build dependencies for Fortran 90 modules is a horrible mess and can easily get broken. And I can't really see that modules solve many real problems.

slezyr 9y ago

> Build tools will parse source files over time and keep track of which modules are where and which translation unit needs what. Isn't this is what cmake does? IIRC it already parses the headers.

humanrebar 9y ago
In vanilla cmake, you declare an ordered list of include directories. But the point about "how does it work now?" is fair. I'm confused why a module search path is any worse than an include search path.
- lultimouomo 9y ago
  This is not related to the search path, but to determining which files need to be rebuilt when a file changes. The idea is that when you change a module, you might need to recompile all the modules that include it, since its interface might have changed. What I don't get in this criticism is that you have this exact problem right now with headers, and it has been solved by the preprocessor outputting dependency information (based on #include directives) for the build system to read. Why can't the same be done with modules (of course it might not be the preprocessor that does it, but the compiler or a specific tool)?
  - humanrebar 9y ago
    I think I'm catching on. With a '#include', you give a file name. With an 'import', you give a module name, which doesn't necessarily map to a file name. So the problem could be solved by saying 'import foobar;' maps to a file (on a search path, maybe) named exactly "foobar". But then we need to work out how subdirectories and module names map to each other. How do you import "foo/bar.ixx"? "foo.bar"? "<foo/bar>"? Though all that seems technically solvable to me. Maybe there's just difficulty in designing this in the context of a committee.
- mikepurvis 9y ago
  I think it's worse because even with a list of directories to look in, you still don't know what _file_ contains the module you're looking for. Literally, you need to open up each file in each directory of the specified search path and possibly even _run it through the preprocessor_ in order to determine if it's the module you're looking for. And assuming you don't want to do all this every time, then you need to maintain a cache somewhere, and all of this logic has to be part of the compiler rather than the build system. Anyway, Java dealt with a bunch of this by enforcing that the file naming and directory hierarchy would match the class hierarchy.

_pmf_ 9y ago

Why do C++ developers make it so, so, so, so hard on themselves and the unfortunate poor souls like me who have to interact with their products?

Koshkin 9y ago
Perhaps, they just want to share with everyone their experience interacting with the language of their choice? (I am joking, but the complexity of the language may, in fact, reflect in the relative complexity of the APIs, say.)

simias 9y ago

I wonder if C++ is headed for COBOL-ification. This article is very interesting but doesn't really give any hope for success in implementing a modern module system in C++. I feel like the only way to have truly useful, simple and intuitive modules in C++ would involve breaking a certain amount of backward compatibility and force a few constraints (project layout, file naming scheme, build system, ...) on the user. But clearly that's not really in C++'s DNA and you risk ending up with something that's not quite C++ while at the same time not having the simplicity and elegance of modern languages that have been designed from scratch without all the baggage C++ carries. C++ won't go away any time soon, that's for sure. But when I read articles like TFA or for instance https://bitbashing.io/std-visit.html https://bitbashing.io/std-visit.html my gut reaction is always that maybe if you get frustrated with C++'s extreme complexity, historical baggage and user-unfriendliness you should consider moving to something else instead of trying to bolt on even more exotic features on a language which is already crumbling under the weight of all the features and programing styles it already supports. I know I did. As an example I stumbled upon this std::visit link by perusing the top posts of reddit's /c/cpp (linked by TFA), here's the discussion: https://www.reddit.com/r/cpp/comments/703k9k/stdvisit_is_everything_wrong_with_modern_c/ https://www.reddit.com/r/cpp/comments/703k9k/stdvisit_is_eve... There's an interesting thread about how: variant<string, int, bool> mySetting = "Hello!"; Doesn't do what one might thing it should do. The variant ends up holding a `true` boolean instead of a string. Why? >char const* to bool is a standard conversion, but to std::string is a user-defined conversion. Standard conversion wins. If you know C++'s history and its C heritage it makes sense but it sure as hell doesn't make me want to use C++ for my next project.

humanrebar 9y ago
C++ is pretty close to treating unqualified string literals the same as native pointers. That is, there for backwards compatibility and for rare edge cases, but generally discouraged. I predict that, given a few years, we'll start adding checks to linters and static analyzers to warn about using them because of the ambiguity and implicit conversion issues. There is already a std::string liberal. Better: MyVariant mySetting = "Hello!"s; But std::string_view more closely models a string literal. std::string is more of a string buffer. So even better: MyVariant mySetting = "Hello!"sv;

mattnewport 9y ago

I'm very familiar with C++, passingly familiar with the modules proposal and unfamiliar with Rust and this article leaves me puzzled exactly what the author expects / wants a modules proposal to do instead. My perspective as a C++ developer has been that modules have one primary purpose, which is to improve build times by eliminating the need to parse huge amounts of C++ in recursively included header files when compiling a compilation unit. If that's all a modules proposal accomplishes I'm fine with that. A modules proposal that doesn't accomplish that is useless. Build tools generally already parse .cpp files for includes to figure out header file dependencies for a .cpp file and have to combine the filenames extracted with information from the build system on include search paths to turn those into dependencies on specific concrete files. That involves information that lives in the build system and not in the code. Modules will require something similar from build tools but as with parsing includes the rules about imports (must be before actual code, must be alone on a line) parsing out the imports is trivial and cheap compared to recursively parsing the full code in header dependencies. So I guess I don't quite understand the problem. Modules are an improvement over headers for build times but they are not going to do everything that module systems perhaps do in some other languages because they have to be adopted incrementally into the many different existing build systems used in C++ code bases (which live outside the standard). Perhaps my ignorance of Rust means I'm missing what the author wants in C++ that's different from the current proposal.

coliveira 9y ago
The author is just trying to prove that C++ modules cannot be like Java or Python modules, which apparently some people don't get. My view is that C++ programmers already know this, since the language will not change suddenly just because modules have been introduced. The idea that the build system will take care of that is perfectly fine for normal C++ users.
- 1ris 9y ago
  >The author is just trying to prove that C++ modules cannot be like Java or Python modules, which apparently some people don't get. I don't think he trys that and i don't think it's true. The committee doesn't want to that, but I don't see why it would be possible by extending the language. Introducte a module - keyword, that works pretty much like a namespace keyword. Force the file name and path to match the name a la java. Introduce a mechanism a la "using namespace" that does import the namespace (like #include header.h) of the module and tells the build system what to build. Neither needs to become the other to do this if we forbid preprocessor commands and if constexprs around that. Allow attributes to specify visibility. Of course it's going to be a way more complicated, but i can't see how it would be impossible. I think c++ devs just don't want that.

namelost 9y ago

I admit I know nothing about C++ modules proposals, but the post talks about an "acyclic graph of translation units". Does this imply that modules can't import one another? What happens when two .cpp files want to do the equivalent of including each other's header files?

rwmj 9y ago

They should really look at how OCaml modules work, since they solve all the same problems already, with desirable properties like separate compilation and easily discoverable mapping of module name to source file. The drawback of OCaml modules is that all dependencies must form an acyclic graph, which can be a pain sometimes although there are well understood workarounds.

zem 9y ago
the other huge problem is the poor support for hierarchical namespaces at the package level. nothing like linking your code with a library and having things break because you have a module name clash
- rwmj 9y ago
  Yes, if you give all your modules a generic "toplevel" name like "Utils". However if you use underscores to namespace things you get something a lot like a hierarchy. You just need to have the discipline to call your modules "Myproject_Utils". Now I didn't say that OCaml has a perfect module system, although some of the first-class module features are stunning, but my point is that it solves a lot of the problems that this C++ guy is complaining about.
  - zem 9y ago
    right, that's the standard workaround, but it really seems like something the compiler should be doing for you. i find it a very surprising wart in an in general very well-designed module system.
DoofusOfDeath 9y ago
> The drawback of OCaml modules is that all dependencies must form an acyclic graph, which can be a pain sometimes although there are well understood workarounds. Can you elaborate on that? It's hard to imagine a sane build system where the dependency graph contains cycles.
- 9y ago

makecheck 9y ago

Perhaps it’s made complex by searching for perfection when really you just need something that makes a high percentage of builds better. You’d be crazy to create a final release of something without starting from a “clean” state, which means they DO NOT need a module implementation with 100% accuracy in all of C++’s asinine corner cases. I wouldn’t trust that accuracy even if they claimed it, I’d “clean” anyway. This means that compilation speedup just has to ensure that MOST incremental rebuilds perform better without adding insane development overhead (such as having to respecify things). If 60% of the time my incremental rebuilds are faster, I’d say that is more than enough to justify a C++1x release. They need to just move forward.

jmgao 9y ago

This seems reminiscent of export templates, which were a resounding success: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2003/n1426.pdf http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2003/n142...

danblick 9y ago

I'm confused; isn't the solution to this problem "gcc -M"?

yahna 9y ago

dumb title. nitpicking. equating innocent, if terse, comments with threats. Why am I reading what this person has to say?

slavik81 9y ago

All I know is that C++ build times are painfully slow for large projects. Even if you spend a lot of time and effort on making them as fast as possible, they're still not good. #include just results in a tremendous amount of code in each and every source file. Even looking at a 6 line program, we see that after the preprocessor runs, the compiler is given 18,162 lines to process. $ cat > main.cpp <<EOF #include <iostream> int main() { std::cout << "Hello World!" << std::endl; return 0; } EOF $ gcc -E main.cpp | wc -l 18162 That is still nearly instantaneous, but it only gets worse from there and it happens to every source file in the project. I just took at look at a 1,848 line source file I'm working with, and it expands out to 147,054 lines after preprocessing. The current state of things is awful and modules are by far the most important feature for the future of C++. We need module support in our build tools and we need them to be fast. Should the TS be delayed to see if the Clang idea is better? I'm curious to know what they're doing, but that's a lot to ask for. The TS is effectively a beta, and pushing back its start means less time in beta before the C++20 standard is finalized. There's a lot of work that's ultimately going to be built on top of this so getting it right is vital, but the clock is ticking.

pjmlp 9y ago
CppCon 2016 has some Google presentations about how they use clang modules (aka module maps). "Deploying C++ modules to 100s of millions of lines of code" https://www.youtube.com/watch?v=dHFNpBfemDI https://www.youtube.com/watch?v=dHFNpBfemDI "There and Back Again: An Incremental C++ Modules Design" https://www.youtube.com/watch?v=h1E-XyxqJRE&t=922s https://www.youtube.com/watch?v=h1E-XyxqJRE&t=922s

Const-me 9y ago

I’m unaware about the recent development, but MS has very module-like features in their version of C++ for decades already. For classic C++ they have #import directive (1) Recently, for windows store/UWP platforms, C++ can consume windows runtime components (2) Two things that allow these features to work are standardized ABI, and standardized type info format. I don’t think high-level modules are possible in C++ before ABI and type info are both standardized in a compiler-neutral way. Better yet, in a language neutral way, like MS did that: you can implement a COM object in a script language as a WSC file, and consume it from C++ using #import. [1] https://msdn.microsoft.com/en-us/library/8etzzkb6.aspx https://msdn.microsoft.com/en-us/library/8etzzkb6.aspx [2] https://docs.microsoft.com/en-us/cpp/windows/how-to-activate-and-use-a-windows-runtime-component-using-wrl https://docs.microsoft.com/en-us/cpp/windows/how-to-activate...