3 ms·

I think your rant isn't long enough to include everything relevant ;) The Blink web engine (which I sometimes compile for qtwebengine) takes a really long time

I think your rant isn't long enough to include everything relevant ;) The Blink web engine (which I sometimes compile for qtwebengine) takes a really long time to compile, several times longer than Gecko according to some info I found online. Google has a policy of not using forward declarations, including everything instead. That's a pretty big WTF for anyone who has ever optimized build time. Google probably just throws hardware and (distributed) caching at the problem, not giving a shit about anyone else building it. Oh, it also needs about 2 GB of RAM per build thread - basically nothing else does.

LtdJorge 1y ago
Even with Firefox using Rust and requiring a build of many crates, qtwebengine takes more time. It was so bad that I had to remove packaged from my system (Gentoo) that were pulling qtwebengine. And I build all Rust crates (including rustc) with -O3, same as C/C++.
bialpio 1y ago
Chromium deviates from Google-wide policy and allows forward-declarations: https://chromium.googlesource.com/chromium/src/+/main/styleguide/c++/c++.md https://chromium.googlesource.com/chromium/src/+/main/styleg..., "Forward declarations vs. #includes".
- ahartmetz 1y ago
  That is really nice to hear, but AFAICS it only means that it may change in the future. Because in current code, it was ~all includes last time I checked. Well, I remember one - very biased - example where I had a look at a class that was especially expensive to compile, like 40 seconds (on a Ryzen 7950X) and maybe 2 GB of RAM. It had under 200 LOC and didn't seem to do anything that's typically expensive to compile... except for the stuff it included. Which also didn't seem to do anything fancy. But transitive includes can snowball if you don't add any "compile firewalls".
  - stick_figure 1y ago
    This is actually tracked at a publicly visible URL: https://commondatastorage.googleapis.com/chromium-browser-clang/build-time.html https://commondatastorage.googleapis.com/chromium-browser-cl... And the include graph analysis: https://commondatastorage.googleapis.com/chromium-browser-clang/include-analysis.html https://commondatastorage.googleapis.com/chromium-browser-cl... The annotated red dots correspond to the last time Chrome developers did a big push to prune the include graph to optimize build time. It was effective, but there was push back. C++ developers just want magic, they don't want to think about dependency management, and it's hard to blame them. But, at the end of the day, builds scale with sources times dependencies, and if you aren't disciplined, you can expect superlinear build times.
    - ahartmetz 1y ago
      Good that it's being tracked, but Jesus, these numbers! 110 CPU hours for a build. (Fortunately, it seems to be a little over half that for my CPU. "Cloud CPUs" are kinda slow.) I picked the 5001st largest file with includes. It's zoom_view_controller.cc, 140 lines in the .cc file, size with includes: 19.5 MB. Initially I picked the 5000th largest file with includes, but for devtools_target_ui.cc, I see a bit more legitimacy for having lots of includes. It has 384 "own" lines in he .cc file and, of course, also about 19.5 MB size with includes. A C++20 source file including some standard library headers easily bloats to a little under 1 MB IIRC, and that's already kind of unreasonable. 20x of that is very unreasonable. I don't think that I need to tell anyone on the Chrome team how to improve performance in software: you measure and then you grab the dumb low-hanging fruit first. From these results, it doesn't seem like anyone is working with the actual goal to improve the situation as long as the guidelines are followed on paper.
      - bialpio 1y ago
        > I picked the 5001st largest file with includes. It's zoom_view_controller.cc, 140 lines in the .cc file, size with includes: 19.5 MB. > Initially I picked the 5000th largest file with includes, but for devtools_target_ui.cc, I see a bit more legitimacy for having lots of includes. It has 384 "own" lines in he .cc file and, of course, also about 19.5 MB size with includes. > A C++20 source file including some standard library headers easily bloats to a little under 1 MB IIRC, and that's already kind of unreasonable. 20x of that is very unreasonable. I think you're not arguing pro-forward-declarations vs anti-forward-declarations here though - it sounds more like an argument for more granular header/source files? In .cc file, each and every include should be necessary for the file to compile (although looking at your example, bind.h seems to be unused and could be removed - looks like the file was refactored and the includes weren't cleaned up). With that said, in the corresponding zoom_view_controller.h, the tab_interface.h include looks to be unnecessary so you did find one good example. :)

LtdJorge 1y ago

Even with Firefox using Rust and requiring a build of many crates, qtwebengine takes more time. It was so bad that I had to remove packaged from my system (Gentoo) that were pulling qtwebengine. And I build all Rust crates (including rustc) with -O3, same as C/C++.

bialpio 1y ago

Chromium deviates from Google-wide policy and allows forward-declarations: https://chromium.googlesource.com/chromium/src/+/main/styleguide/c++/c++.md https://chromium.googlesource.com/chromium/src/+/main/styleg..., "Forward declarations vs. #includes".

ahartmetz 1y ago
That is really nice to hear, but AFAICS it only means that it may change in the future. Because in current code, it was ~all includes last time I checked. Well, I remember one - very biased - example where I had a look at a class that was especially expensive to compile, like 40 seconds (on a Ryzen 7950X) and maybe 2 GB of RAM. It had under 200 LOC and didn't seem to do anything that's typically expensive to compile... except for the stuff it included. Which also didn't seem to do anything fancy. But transitive includes can snowball if you don't add any "compile firewalls".
- stick_figure 1y ago
  This is actually tracked at a publicly visible URL: https://commondatastorage.googleapis.com/chromium-browser-clang/build-time.html https://commondatastorage.googleapis.com/chromium-browser-cl... And the include graph analysis: https://commondatastorage.googleapis.com/chromium-browser-clang/include-analysis.html https://commondatastorage.googleapis.com/chromium-browser-cl... The annotated red dots correspond to the last time Chrome developers did a big push to prune the include graph to optimize build time. It was effective, but there was push back. C++ developers just want magic, they don't want to think about dependency management, and it's hard to blame them. But, at the end of the day, builds scale with sources times dependencies, and if you aren't disciplined, you can expect superlinear build times.
  - ahartmetz 1y ago
    Good that it's being tracked, but Jesus, these numbers! 110 CPU hours for a build. (Fortunately, it seems to be a little over half that for my CPU. "Cloud CPUs" are kinda slow.) I picked the 5001st largest file with includes. It's zoom_view_controller.cc, 140 lines in the .cc file, size with includes: 19.5 MB. Initially I picked the 5000th largest file with includes, but for devtools_target_ui.cc, I see a bit more legitimacy for having lots of includes. It has 384 "own" lines in he .cc file and, of course, also about 19.5 MB size with includes. A C++20 source file including some standard library headers easily bloats to a little under 1 MB IIRC, and that's already kind of unreasonable. 20x of that is very unreasonable. I don't think that I need to tell anyone on the Chrome team how to improve performance in software: you measure and then you grab the dumb low-hanging fruit first. From these results, it doesn't seem like anyone is working with the actual goal to improve the situation as long as the guidelines are followed on paper.
    - bialpio 1y ago
      > I picked the 5001st largest file with includes. It's zoom_view_controller.cc, 140 lines in the .cc file, size with includes: 19.5 MB. > Initially I picked the 5000th largest file with includes, but for devtools_target_ui.cc, I see a bit more legitimacy for having lots of includes. It has 384 "own" lines in he .cc file and, of course, also about 19.5 MB size with includes. > A C++20 source file including some standard library headers easily bloats to a little under 1 MB IIRC, and that's already kind of unreasonable. 20x of that is very unreasonable. I think you're not arguing pro-forward-declarations vs anti-forward-declarations here though - it sounds more like an argument for more granular header/source files? In .cc file, each and every include should be necessary for the file to compile (although looking at your example, bind.h seems to be unused and could be removed - looks like the file was refactored and the includes weren't cleaned up). With that said, in the corresponding zoom_view_controller.h, the tab_interface.h include looks to be unnecessary so you did find one good example. :)