6 ms·
C++ Modules Might Be Dead-on-Arrival (2019)
- MontagFTB 5y agoThis post is from January 2019- is there a more recent update on the issues outlined herein?
- dang 5y agoYear added above. Thanks!
- Fronzie 5y agoVisual studio support is mostly there: https://devblogs.microsoft.com/cppblog/moving-a-project-to-cpp-named-modules/ https://devblogs.microsoft.com/cppblog/moving-a-project-to-c... CMake support is an open issue: https://gitlab.kitware.com/cmake/cmake/-/issues/18355 https://gitlab.kitware.com/cmake/cmake/-/issues/18355
- deleted 5y ago[deleted]
- lelanthran 5y agoOkay, this was a convincing and persuasive argument. Did anyone ever respond to this? What do the anti-SG15 (and/or pro-module) people say? I want to see the other side of this argument.
- gfxgirl 5y agoYea, it's a build system. The same problem exists with lots of projects. // foo.cpp #include auto_generated_file.cpp The C++ compiler doesn't know how to build auto_generated_file.cpp, the build system does.
- klodolph 5y agoFrom my perspective, all the complaints are solved by using a more modern build system, like Bazel. If you are already encoding dependencies between modules in your build system, then when you are searching for an import, it's trivial... you just look at the declared module dependencies. Bazel (and other similar systems) already force you to do this for other reasons, and so you theoretically get modules "free". It's not like the same problem doesn't already crop up with generated sources. Like, if you see this: #include "file.h" If file.h is generated, then your build system needs to generate it before you compile anything that includes it. Generated sources are a major pain point in C++ build systems. It throws a wrench in the typical compilation flow... where you don't know which header files you need until you run the compiler, so you have to manually specify header dependencies in your Makefile if any are generated (instead of getting the dependencies from the compiler output). To keep things consistent (and more reliably cacheable), in Bazel, you have to specify all the dependencies in your build system, not just the ones that the compiler can't figure out for you. This solves some other problems with cached builds--like how you can have multiple files named "file.h", and the insertion of a new file earlier in the include search paths should trigger a rebuild, or else you end up with a poisoned build cache. Obviously there are going to be some people stuck on build systems that can't handle this stuff gracefully. Too bad. You can keep using headers, it's not like those are going to stop working.
- dataflow 5y agoDoes anyone know of a good example of C++ module usage out in the wild? Not Hello World examples, but I mean an actual project with a serious build process using it? I'd love to see how they're currently used.
- KyleSanderson 5y agoThese are from 2006~, extensions for a language (SourcePawn) took over in 2009. http://www.metamodsource.net/plugins http://www.metamodsource.net/plugins
- pjmlp 5y agoNothing serious, but you can look at https://github.com/pjmlp/ppm2png https://github.com/pjmlp/ppm2png https://github.com/pjmlp/AStarDemo https://github.com/pjmlp/AStarDemo I am also in the process of porting Raytracing Weekend to C++ modules.
- dataflow 5y agoAh ok, thanks!
- dang 5y agoDiscussed at the time: C++ Modules Might Be Dead-On-Arrival - https://news.ycombinator.com/item?id=19015079 https://news.ycombinator.com/item?id=19015079 - Jan 2019 (193 comments)
- Mr_P 5y agoThere's a 2016 CppCon talk here about making C++ Modules work at scale for Google's 100Mloc mono repo: https://www.youtube.com/watch?v=dHFNpBfemDI https://www.youtube.com/watch?v=dHFNpBfemDI So, apparently it can work. I don't don't how much Google's internal Module incarnation differs (if at all) from what was ultimately published in the standard, but I'd expect it to be similar. The article discusses a problem of resolving dependencies between compiling different c++ files. I'm no expert, but I think this dependency graph is implicitly baked into the build system, with tool-assisted user-written bazel rules.
- Kranar 5y agoThe module system from that video would have been great; there was/is a sound implementation that is based on Objective-C's module system, has been battle tested extensively and works without all kinds of silly quirks, makes packaging and distributing modules relatively easy, and provides performance benefits similar to precompiled headers which are known to improve build times substantially. Of course, for those reasons it did not get standardized and instead what did get standardized was a mostly theoretical module system that has a great deal of ambiguity and implementation divergence across different compilers, places a great burden on build systems, none of which fully support modules yet, does nothing to help with packaging libraries, and only sometimes provides performance benefits, sometimes provides performance regressions. Such is the way of C++, a language whose standardization process has been hijacked by politics and corporate stacking of the committee, legacy interests who insist on holding back key features that would improve the language for the vast majority of developers for the sake of ABI compatibility, and whose advancement has been plagued by numerous defect reports and broken or buggy features that get adopted without any kind of battle testing, user experience or broad community feedback.
- kevin_thibedeau 5y agoPresumably you're expected to act like Ada and put the interface in a separate TU that can be processed first by the build system.
- dgellow 5y agoEdit: I just notice now that you said "TU", my mind autocompleted your sentence with "file" instead the first time I read it. Still keeping my comment here, module :private is not well known and a nice way to declare a distinct interface Translation Unit while keeping everything in one file. You can do this, but you can also use the "module :private" fragment if you want to have everything in one file while still keeping the distinction between interface and implementation. Example: // File "mymodule.ixx" module; #include <string> // declare interface export module mymodule; export namespace mymodule { // templates have to be declared // in interface to be known at // compile-time template<typename T> T add(T x, T y) { return x + y; } std::string greetings(); } // implement interface module :private; std::string mymodule::greetings() { return "hello"; }
- pjmlp 5y agoYou can do that if you wish, they are called module interface units.
- natded 5y agoHow do C++ programs / programmers enforce and manage modularity in the general sense if they don't even have simplest module system? I'm not even talking of having module functors like Ocaml, but something like Rust has. It seems to be one of the most obvious steps to reduce complexity and compose program of smaller parts.
- pubby 5y ago#include and namespaces.
- actinium226 5y agoMostly we rely on the system package manager. If I'm trying to compile a C++ program and I need somelib I `sudo apt install libsomelib-dev` Works about 80% of the time. The other 20% I gotta go digging through either github or sourceforge or someone's personal website that hasn't been updated in 15 years.
- petmon 5y agoIt's not "enforced;" it is managed through various means including namespaces, header includes, and prefixes. For example, SQLite is a library whose functions all start with 'sqlite3'. In practice it works well enough: name collisions are uncommon and can be managed.
- xpressvideoz 5y agoWait, does the C++ module proposal allow modules names to be different from their filenames? This sounds so strange, I've never used such a module system in other languages. I've always assumed C++ modules would somehow restrict the excessive liberty it has had from its headers and preprocessors. How did they arrive at such a weird idea?
- dgellow 5y agoC++ modules have little to do with the concept of "modules" you find in other languages. You can think about it as a more standardized way to define and create what is known as pre-compiled headers. But that's basically it.
- klodolph 5y agoC++ modules are not like pre-compiled headers. They give you freedom to specify what you are and are not exporting from a module. This is unlike header files (including pre-compiled headers), where you are constrained to re-export any dependency your header file has, as well as any code internal to your module which is necessary to declare your exports. C++ headers are a total mess, and modules are the cleanup effort.
- dgellow 5y agoSure, I don't disagree with that.
- pjmlp 5y agoC++ is not alone in this regard, take .NET as one of the ecosystems that follows the same idea, D, Ada and Fortran as well.
- deleted 5y ago[deleted]
- lelanthran 5y ago> Wait, does the C++ module proposal allow modules names to be different from their filenames? It sort of makes sense. The calling program specifies `foo.bmi`, but compiling for ARM might look in "armv5/foo.bmi" while targeting x64 might look in "x64/foo.bmi", and that doesn't even take into account that there might be different debug and release versions.
- mikewarot 5y agoDo C++ folks still not have the equivalent of Turbo Pascal Units (which were from 1987)? I can compile almost anything short of rebuilding my IDE in less than a second, every time, thanks to units in Lazarus/Free Pascal.
- lelanthran 5y ago> Do C++ folks still not have the equivalent of Turbo Pascal Units (which were from 1987)? Well, it's an easy problem to solve when a single compiler is used for everything. I doubt that Turbo Pascal Units could be used with other Pascal compilers. With C++, your module compiled with Clang might be used in a program compiled with gcc. I don't think that C++ modules specify a binary interface in the spec. With new languages it is easier to implement modules, because a new language, by definition, only has a single implementation. If the new language specifies a modules system it does not break existing software because, by definition, it is a new language. > I can compile almost anything short of rebuilding my IDE in less than a second, every time, thanks to units in Lazarus/Free Pascal. I know, I sometimes use Lazarus. I'm not really sure that a FreePascal Unit is compatible with other Passsl implementations (maybe Delphi, because the goal is to be a drop-in replacement for Delphi?). My understanding of the C++ modules specification is that there is no requirement for modules from different compilers to be compatible.
- mikewarot 5y ago>With C++, your module compiled with Clang might be used in a program compiled with gcc. Why do you need more than one C++ compiler installed? The idea is to compile everything only once, and save all the time when you need to use the code later. It makes no sense not to use the same compiler against all the source when building an executable.
- Negitivefrags 5y agoPeople don't build everything on their system from source. Distribution of binaries is a thing too.
- 5y ago
- eyelidlessness 5y agoWild that JS got this right in ESM with its first standard, but the lisper in me is still screaming that both are needlessly complicated by tooling.
- boardwaalk 5y agoI was just tonight watching Bjarne’s cppcon 21 talk[1] and he does not seem to think they’re dead. OTOH, from what I can tell my compiler of choice, clang, has not improved its support since last time I dabbled with it unsuccessfully. But I could be Doing It Wrong. There seems to be very little stuff on the internet about actually using them which is not a great sign. [1] https://youtu.be/15QF2q66NhU https://youtu.be/15QF2q66NhU
- PostThisTooFast 5y ago
- anonnybonny 5y agohttps://www.youtube.com/watch?v=9OWGgkuyFV8 https://www.youtube.com/watch?v=9OWGgkuyFV8
- int_19h 5y agoI can't help but notice that of all the names in the linked docs, there doesn't seem to be any from the companies that implement the actual C++ compilers. What's the stance of Clang/gcc/MVSC/... developers on this?
- klodolph 5y agoEh, the assumption here is that you don't know what the dependencies of a C++ file are before you start compiling. That's not a great assumption. That assumption allows you to write really lazy Makefiles. You just throw a bunch of C++ filenames in your Makefile, add the correct dep flags, and add an "include" line, and your Makefile will correctly rebuild the object files when header files change, without needlessly rebuilds. Here's how such a lazy Makefile works: CXXFLAGS = -MF $*.d -MMD -MP -include $(wildcard *.d) Sure, that's nice, but there are a lot of reasons why you might want a more sophisticated build system than a lazy Makefile. You might have a project with long build times, so you might want a good build cache. You might have generated header files. As your project gets larger and more complicated, the idea of building everything with a Makefile really loses its appeal, and you look for better build systems. The new generation of build systems is Bazel, Buck, Pants, Please, Meson, etc. Plenty of choices to go around. These systems require that you specify the dependencies up-front, rather than relying on -MF & friends. This means that if you add a generated header somewhere, nothing breaks. If you add a new file earlier in the include search path with the same name as a file later in the path, nothing breaks. (By "nothing breaks" I just mean that your incremental builds and clean builds will still both generate the correct output.) The -MF option is then used to trim the dependency set (in a particular way to keep the build correct... rather than record the header dependencies, you record which headers were ignored during compilation). With these build systems, modules are going to work just fine.
- knorker 5y agoI think this post has a clearly missing solution that is (essentially) a first stage in a build system. Yes, C++ can be slow to build. But it's slow to build because of the includes and the code generation (and we still have to do code generation). How much time does the author think it takes to pipe code through the preprocessor (for defines, not includes since in this case there are no includes), and tokenize? I just checked the dependency files of a fairly large project (GNU Radio), and each cc file transitively depends on 414 files on average. In total in order to compile its 1339 cc files the compiler needs to open and read over half a million files. So just in terms of data to read on this first pass it's faster by a factor over 400x. Add to that that you don't have to do anything in this pass except look for module dependencies, I bet it's several thousand times faster than a compile. So in other words, if it takes you an hour to build your big project, are you saying you don't have one single second left over to calculate the dependency tree as a first pass? And you only have to do it once (and then re-do files as source timestamps change).
- WalterBright 5y agoD just reads and compiles each module only once, no matter how many times it is imported, or whatever rat's nest the import graph is. Modules could be precompiled, but I found that it was faster (and easier) to just recompile them. (And not having a preprocessor eliminates several passes.)
- munch117 5y agoIf the compiler and the build system is an integrated product, then right, this shouldn't be too hard. But that's not usually the case. If they're not, what's the interface for the initial build step? Is it still something like "gcc -MD"? But "gcc -MD" can't just spit out a list of files. It will have to output not only file names but also module names (both referenced and declared), because resolving module names to file names requires global knowledge. That means every build tool that wants to work with C++ must be changed to take in this extended information and resolve module dependencies. Because otherwise, if there's even a single module in your 1339 file project, then current tooling breaks. Which will put pressure on developers to not add that first module, slowing adoption.
- WalterBright 5y agoD's module system benefited greatly by getting rid of the preprocessor. I implemented precompiled headers for C++, and so knew that a robust and successful module system wasn't compatible with preprocessing.
- Shadonototra 5y agoit is - no circular dependency allowed - you have to fw declare everything, yet again i gave up on C++ personally, for a little while i went back to C only, now i only use D, and some GO for my backend needs
- garaetjjte 5y agoHow does this design handle cyclic dependencies? Or it doesn't, which would be even worse than headers in which you can at least forward declare some things?