9 ms·

There is one real problem with "modern" C++ that I would have hoped they would focus on: Compile times. All the new stuff is heavily template based. As more li

There is one real problem with "modern" C++ that I would have hoped they would focus on: Compile times. All the new stuff is heavily template based. As more libraries are written in that style, compile times go through the roof. I wrote some code recently that required high precision monetary calculations. So I used boost multiprecision. It turned out to be completely impossible as most compilation units ended up depending on that library directly or indirectly. Already slow compiles became unbearably slow. Facing a tight deadline I quickly wrote an extremely ugly, slow, error prone, pimpl based wrapper around boost multiprecision. Many calculations now require dynamic memory allocation for no good reason. This is just terrible, because I know I can't leave it like this. I'm going to have redesign the whole thing or look for other libraries or look a lot more deeply into the calculations to see if they really cannot be done with large integers. It's a productivity disaster. You could say I made wrong choices, which is true, but I think the C++ standards committee has been making wrong choices in not prioritizing a fix for compile times (modules?). What's the point of having nice new features if you end up spending half your time on carefully cordoning off any code that uses them?

my123 10y ago
And existing compilers that do not support them and that you still have to support...
melling 10y ago
Modules are suppose to help with compile times. They never seem to make it into the standard. Must be a few outstanding complications.
- koja86 10y ago
  There are others who feel like us. Fingers crossed: http://clang.llvm.org/docs/Modules.html#problems-with-the-current-model http://clang.llvm.org/docs/Modules.html#problems-with-the-cu...
jzwinck 10y ago
To speed up parallel compilation, use icecc. It's great if you half even two or three computers on your LAN. For high precision currency, use int64_t (perhaps in a simple POD class). This, with an 8-digit implied decimal part, is enough to represent the finest price unit in any market today, with a max of 92 billion. If you are dealing with more than 92 billion of any currency in a single place, don't complain!
- cm2187 10y ago
  In JPY you get there very fast. Even in EUR/USD/GBP, a typical large bank balance sheet is over a trillon.
  - fauigerzigerk 10y ago
    True. Also, the smallest unit in which prices are quoted is not in all cases sufficient for calculations. A million + 1 rounding errors after the 8th decimal place adds up to an error measured in cents. In my particular case, 64 bit integers might be sufficient. I thought about it but then decided I wanted to be on the safe side. Maybe not the best decision.
    - cm2187 10y ago
      Not sure what sort of assets you are looking at but prices are rarely quoted with a very large number of decimals. Having chunky increments is sort of necessary to match bid and offers on markets.
      - fauigerzigerk 10y ago
        Sure, but division and multiplication add more decimal places. If you do a lot of calculations and you truncate or round every single intermediate result, rounding errors may add up to unacceptable levels in some cases. So I wanted perform the caculations with high precision, then round the final result according to what the law requires.
mrich 10y ago
Modules are the answer, but it seems nobody cares enough to get them in the standard or, more likely, it's just too hard a problem. Having looked at the clang modules implementation you basically have to clean up all the system includes, the standard library and your own headers before you can use them. And you will have to rewrite many things in these headers due to changed preprocessor semantics. I think neither the clang nor MSVC implementations can be used for anything but the most trivial code due to these issues. There are talks from Google who invested heavily into modules using clang and got gains, but as far as I understand they fixed those issues only for themselves. More info welcome. At this point the only cure is brute force (blades with many cores are not that expensive, or a distributed compile cluster) and always taking care with your includes.
- pjmlp 10y ago
  Modules are on the good track, looking from the latest CppCon and LLVM talks,they have progressed a lot on their internal use. I am confident that by C++20 they will be available. The main problem will be the adoption by major libraries.
  - mrich 10y ago
    Correct me if I'm wrong, won't we need an updated OS tool chain, i.e. kernel headers and libc/libstdc++ fully adjusted? That will take some work...
    - pjmlp 10y ago
      Just kind of. Kernel headers are only an issue on free UNIX like OSes, on other OS what matters is what the SDK provides. So at least in what concerns Apple, Google, Microsoft and every other commercial vendor OSes, don't expect a big issue having their SDKs updated when they start supporting modules. The problem is getting the BSD, Linux and open source libraries without a specific agenda to adopt them.
    - gpderetta 10y ago
      Libstdc++ is normally distributed with the compiler, so it will be updated as required by the standard. Regarding the libc and other headers, modules will still fullly support include files, so that won't be an issue. You should be able to include headers and export them as modules.
enqk 10y ago
I'm fighting with this daily. Working on a medium size desktop software application, seeing compile times increasing in the past months as developers use modernist C++. Past a certain point, long compile times make it harder to improve a large code base. People work around it by adding more stuff at the leaves and never fixing fundamental parts of the code base, avoiding to touch heavily shared code. This is almost a guarantee for another increase in compile time (because you end up accreting more stuff) which makes it even harder to work on the code base at large scale. And that has I believe a concrete effect on the quality of a piece of software. The problem is, there are some low hanging fruits (like caching, parallelizing and distributing builds) so people think they can delay the inevitable. The problem is, those counter measures only deal with the average case, while doing nothing about the worst case.
Arnsaste 10y ago
I agree, the compile times are my mine problem with C++ right now, and they are not getting better. If you listen to the C++ advocates, they all say use the STL as much as possible, use boost if there is solution for your problem there, etc.. It's true, you probably will quickly get a solution that works and looks pretty, but if you do this a hundred or a thousand times your code suddenly needs several hours to compile. I am not against the STL, or boost, but there is a cost associated with using them that is often ignored, but IMHO is actually a productivity killer in codebases that are a bit bigger.
logicallee 10y ago
holy cow, I never would have thought in 2016 compiling a C++ program could be so slow that it makes it impossible for someone. Can you put numbers on these figures? By "Unbearably slow" how many minutes (or hours) did you mean? Do you use a heavy desktop with a fast SSD and a lot of RAM? Finally what is the total output program size (roughly). Like are we talking about an application that is in the gigabytes, like the complete Adobe suite or something? I would like to know more information about how it's possible for this to happen. tell us as much as possible :) Thanks!
- quinnftw 10y ago
  To piggy back on this comment: what build system are you using? I've found that bazel has greatly reduced compile times, even when using boost and opencv.
  - jlewallen 10y ago
    I just took a very quick look at Bazel and saw that Windows support is "Highly Experimental" which concerned me. I've been looking around in this space recently as I'm getting into C/C++ again (ARM uC stuff) and all three platforms are fairly important to me for what I'm doing. I'm a big Gradle fan and Bazel seems like a natural next step if it's not problematic.
    - quinnftw 10y ago
      I develop solely in UNIX environments so I can't comment on its performance on windows, but it's fantastic on UNIX and very easy to write / configure.
  - dom0 10y ago
    I have used qbs with great success. It's by far the fastest and most correct¹ build system for C++ that I've used so far, and it's configuration ain't so bad (tm). ¹ I used it in a couple projects and never had a single incorrect build. With most other build tools, be it CMake or Gradle, these happen regularly for me.
- 10y ago
tomohawk 10y ago
This is one of the pain points that caused Google to develop golang.
prefect42 10y ago
In Microsoft-Land, I find that every time we force ourselves to upgrade to the latest compiler (really entire IDE, Visual Studio) then everyone also needs to upgrade their machines. Both the IDE and the C++ compiler run dog slow on "older hardware". We're talking new-ish microarchitecture, faster RAM, and SSDs, to keep things sane. This is just one anecdotal use case of course. Though I'm curious as to what you mean by "unbearably slow", and what your toolchain is.
- mrich 10y ago
  I have found the latest update (VS 2015), were Microsoft significantly invested in the compiler after years of stagnation, to bring better compile speed in addition to standards support. There were also quite some bugs in the codegen, but it seems those are never avoided when working on a C++ compiler. I find the IDE to still be sufficiently fast after disabling some things like Git integration and codelens.
babuskov 10y ago
> There is one real problem with "modern" C++ that I would have hoped they would focus on: Compile times. I found a really simple solution to this: disable all optimizations while you are developing and only use them for production build. I have 70k LOC project that compiles from scratch in 10-11 seconds (using 8 parallel builds). Regular changes to 2-3 .cpp files and link are done in 1-2 seconds. This is esp. important with MSVC which has a huge difference between non-optimized and optimized build. Also, use precompiled headers.
- RBO2 10y ago
  Also I use ccache on all my projects. Rebuilds times are now at 10s for a 50 kloc project.
  - mikepurvis 10y ago
    Sure, but in a "modern" codebase where you're template-everything, then more changes will be in headers, resulting in a lot more translation units being affected by any given change.
- user5994461 10y ago
  There are issues which may arise because of the optimizations. Need to have them in a staging build.
xgdgsc 10y ago
Have you tried using precompiled headers? I' ve started using it recently and want to here some user stories.
- Const-me 10y ago
  In Microsoft world, they are available for a couple decades. Yes, they help a lot with compilation time.
gigatexal 10y ago
What about things like incredibuild? https://www.incredibuild.com/incredibuild-product-overview.html https://www.incredibuild.com/incredibuild-product-overview.h... I'm not paid to use them or shill for them I just think distributed compilation might help
- enqk 10y ago
  If you want to plug this one, you can plug and have a look at a open source, very easy to use alternative called FASTbuild: http://fastbuild.org http://fastbuild.org
xyzzy_plugh 10y ago
Boost is always my last resort. While Boost often implements the latest proposed standards and provides a vast extension to the STL, it is not the STL. Often proposed standards changes are best implemented by changing the compiler itself. Expecting Boost to provide the same functionality as a modified compiler without a cost is foolish. Boost provides great implementations but at a great cost in complexity, compile time and binary size bloat. I don't even really believe modules will solve this. They push the boundaries of what is possible with today's equipment, in order to discover what the next equipment should be. Also I no longer look to Boost libraries as a Seal of Quality. There be dragons. Choose wisely.
digler999 10y ago
> Already slow compiles became unbearably slow. what is unbearably slow ? an hour ? a day ? how many times did you actually have to rebuild the entire code base ? Did you do parallel builds, or see what gains you could get from faster hardware ? I'm genuinely curious as I've never worked on a codebase that large, and the only time you have to rebuild everything is when someone changes a common library header.
DrBazza 10y ago
There are various solutions to compile times in C++ that have been talked about at length over the years. Lakos' book from 1996 talks about this. In fact, a lot of modern software engineering guidelines come from working around C++'s built in shortcomings. Use things such as the Pimpl idiom to help linking. Modular libraries rather than big-ball-of-mud, do you really need to recompile world+dog each time? Top-down includes, and no relative includes, isolating your code. In fact go-lang does exactly this. The trouble is that in C++, fast compile times need to be designed into the code base and often enforced by scripts. I don't think that there's any other language that requires so much 'non-language' engineering as C++.

my123 10y ago

And existing compilers that do not support them and that you still have to support...

melling 10y ago

Modules are suppose to help with compile times. They never seem to make it into the standard. Must be a few outstanding complications.

koja86 10y ago
There are others who feel like us. Fingers crossed: http://clang.llvm.org/docs/Modules.html#problems-with-the-current-model http://clang.llvm.org/docs/Modules.html#problems-with-the-cu...

jzwinck 10y ago

To speed up parallel compilation, use icecc. It's great if you half even two or three computers on your LAN. For high precision currency, use int64_t (perhaps in a simple POD class). This, with an 8-digit implied decimal part, is enough to represent the finest price unit in any market today, with a max of 92 billion. If you are dealing with more than 92 billion of any currency in a single place, don't complain!

cm2187 10y ago
In JPY you get there very fast. Even in EUR/USD/GBP, a typical large bank balance sheet is over a trillon.
- fauigerzigerk 10y ago
  True. Also, the smallest unit in which prices are quoted is not in all cases sufficient for calculations. A million + 1 rounding errors after the 8th decimal place adds up to an error measured in cents. In my particular case, 64 bit integers might be sufficient. I thought about it but then decided I wanted to be on the safe side. Maybe not the best decision.
  - cm2187 10y ago
    Not sure what sort of assets you are looking at but prices are rarely quoted with a very large number of decimals. Having chunky increments is sort of necessary to match bid and offers on markets.
    - fauigerzigerk 10y ago
      Sure, but division and multiplication add more decimal places. If you do a lot of calculations and you truncate or round every single intermediate result, rounding errors may add up to unacceptable levels in some cases. So I wanted perform the caculations with high precision, then round the final result according to what the law requires.

mrich 10y ago

Modules are the answer, but it seems nobody cares enough to get them in the standard or, more likely, it's just too hard a problem. Having looked at the clang modules implementation you basically have to clean up all the system includes, the standard library and your own headers before you can use them. And you will have to rewrite many things in these headers due to changed preprocessor semantics. I think neither the clang nor MSVC implementations can be used for anything but the most trivial code due to these issues. There are talks from Google who invested heavily into modules using clang and got gains, but as far as I understand they fixed those issues only for themselves. More info welcome. At this point the only cure is brute force (blades with many cores are not that expensive, or a distributed compile cluster) and always taking care with your includes.

pjmlp 10y ago
Modules are on the good track, looking from the latest CppCon and LLVM talks,they have progressed a lot on their internal use. I am confident that by C++20 they will be available. The main problem will be the adoption by major libraries.
- mrich 10y ago
  Correct me if I'm wrong, won't we need an updated OS tool chain, i.e. kernel headers and libc/libstdc++ fully adjusted? That will take some work...
  - pjmlp 10y ago
    Just kind of. Kernel headers are only an issue on free UNIX like OSes, on other OS what matters is what the SDK provides. So at least in what concerns Apple, Google, Microsoft and every other commercial vendor OSes, don't expect a big issue having their SDKs updated when they start supporting modules. The problem is getting the BSD, Linux and open source libraries without a specific agenda to adopt them.
  - gpderetta 10y ago
    Libstdc++ is normally distributed with the compiler, so it will be updated as required by the standard. Regarding the libc and other headers, modules will still fullly support include files, so that won't be an issue. You should be able to include headers and export them as modules.

enqk 10y ago

I'm fighting with this daily. Working on a medium size desktop software application, seeing compile times increasing in the past months as developers use modernist C++. Past a certain point, long compile times make it harder to improve a large code base. People work around it by adding more stuff at the leaves and never fixing fundamental parts of the code base, avoiding to touch heavily shared code. This is almost a guarantee for another increase in compile time (because you end up accreting more stuff) which makes it even harder to work on the code base at large scale. And that has I believe a concrete effect on the quality of a piece of software. The problem is, there are some low hanging fruits (like caching, parallelizing and distributing builds) so people think they can delay the inevitable. The problem is, those counter measures only deal with the average case, while doing nothing about the worst case.

Arnsaste 10y ago

I agree, the compile times are my mine problem with C++ right now, and they are not getting better. If you listen to the C++ advocates, they all say use the STL as much as possible, use boost if there is solution for your problem there, etc.. It's true, you probably will quickly get a solution that works and looks pretty, but if you do this a hundred or a thousand times your code suddenly needs several hours to compile. I am not against the STL, or boost, but there is a cost associated with using them that is often ignored, but IMHO is actually a productivity killer in codebases that are a bit bigger.

logicallee 10y ago

holy cow, I never would have thought in 2016 compiling a C++ program could be so slow that it makes it impossible for someone. Can you put numbers on these figures? By "Unbearably slow" how many minutes (or hours) did you mean? Do you use a heavy desktop with a fast SSD and a lot of RAM? Finally what is the total output program size (roughly). Like are we talking about an application that is in the gigabytes, like the complete Adobe suite or something? I would like to know more information about how it's possible for this to happen. tell us as much as possible :) Thanks!

quinnftw 10y ago
To piggy back on this comment: what build system are you using? I've found that bazel has greatly reduced compile times, even when using boost and opencv.
- jlewallen 10y ago
  I just took a very quick look at Bazel and saw that Windows support is "Highly Experimental" which concerned me. I've been looking around in this space recently as I'm getting into C/C++ again (ARM uC stuff) and all three platforms are fairly important to me for what I'm doing. I'm a big Gradle fan and Bazel seems like a natural next step if it's not problematic.
  - quinnftw 10y ago
    I develop solely in UNIX environments so I can't comment on its performance on windows, but it's fantastic on UNIX and very easy to write / configure.
- dom0 10y ago
  I have used qbs with great success. It's by far the fastest and most correct¹ build system for C++ that I've used so far, and it's configuration ain't so bad (tm). ¹ I used it in a couple projects and never had a single incorrect build. With most other build tools, be it CMake or Gradle, these happen regularly for me.
10y ago

tomohawk 10y ago

This is one of the pain points that caused Google to develop golang.

prefect42 10y ago

In Microsoft-Land, I find that every time we force ourselves to upgrade to the latest compiler (really entire IDE, Visual Studio) then everyone also needs to upgrade their machines. Both the IDE and the C++ compiler run dog slow on "older hardware". We're talking new-ish microarchitecture, faster RAM, and SSDs, to keep things sane. This is just one anecdotal use case of course. Though I'm curious as to what you mean by "unbearably slow", and what your toolchain is.

mrich 10y ago
I have found the latest update (VS 2015), were Microsoft significantly invested in the compiler after years of stagnation, to bring better compile speed in addition to standards support. There were also quite some bugs in the codegen, but it seems those are never avoided when working on a C++ compiler. I find the IDE to still be sufficiently fast after disabling some things like Git integration and codelens.

babuskov 10y ago

> There is one real problem with "modern" C++ that I would have hoped they would focus on: Compile times. I found a really simple solution to this: disable all optimizations while you are developing and only use them for production build. I have 70k LOC project that compiles from scratch in 10-11 seconds (using 8 parallel builds). Regular changes to 2-3 .cpp files and link are done in 1-2 seconds. This is esp. important with MSVC which has a huge difference between non-optimized and optimized build. Also, use precompiled headers.

RBO2 10y ago
Also I use ccache on all my projects. Rebuilds times are now at 10s for a 50 kloc project.
- mikepurvis 10y ago
  Sure, but in a "modern" codebase where you're template-everything, then more changes will be in headers, resulting in a lot more translation units being affected by any given change.
user5994461 10y ago
There are issues which may arise because of the optimizations. Need to have them in a staging build.

xgdgsc 10y ago

Have you tried using precompiled headers? I' ve started using it recently and want to here some user stories.

Const-me 10y ago
In Microsoft world, they are available for a couple decades. Yes, they help a lot with compilation time.

gigatexal 10y ago

What about things like incredibuild? https://www.incredibuild.com/incredibuild-product-overview.html https://www.incredibuild.com/incredibuild-product-overview.h... I'm not paid to use them or shill for them I just think distributed compilation might help

enqk 10y ago
If you want to plug this one, you can plug and have a look at a open source, very easy to use alternative called FASTbuild: http://fastbuild.org http://fastbuild.org

xyzzy_plugh 10y ago

Boost is always my last resort. While Boost often implements the latest proposed standards and provides a vast extension to the STL, it is not the STL. Often proposed standards changes are best implemented by changing the compiler itself. Expecting Boost to provide the same functionality as a modified compiler without a cost is foolish. Boost provides great implementations but at a great cost in complexity, compile time and binary size bloat. I don't even really believe modules will solve this. They push the boundaries of what is possible with today's equipment, in order to discover what the next equipment should be. Also I no longer look to Boost libraries as a Seal of Quality. There be dragons. Choose wisely.

digler999 10y ago

> Already slow compiles became unbearably slow. what is unbearably slow ? an hour ? a day ? how many times did you actually have to rebuild the entire code base ? Did you do parallel builds, or see what gains you could get from faster hardware ? I'm genuinely curious as I've never worked on a codebase that large, and the only time you have to rebuild everything is when someone changes a common library header.

DrBazza 10y ago

There are various solutions to compile times in C++ that have been talked about at length over the years. Lakos' book from 1996 talks about this. In fact, a lot of modern software engineering guidelines come from working around C++'s built in shortcomings. Use things such as the Pimpl idiom to help linking. Modular libraries rather than big-ball-of-mud, do you really need to recompile world+dog each time? Top-down includes, and no relative includes, isolating your code. In fact go-lang does exactly this. The trouble is that in C++, fast compile times need to be designed into the code base and often enforced by scripts. I don't think that there's any other language that requires so much 'non-language' engineering as C++.