6 ms·

I am extremely interested in this. I am stuck in an environment with CMake, GCC and Unix Make (no clang, no ninja) and getting detailed information about WHY t

I am extremely interested in this. I am stuck in an environment with CMake, GCC and Unix Make (no clang, no ninja) and getting detailed information about WHY the build is taking so long is nearly impossible. It's also a bit of a messy build with steps like copying a bunch of files from the source into the build folder. Multiple languages (C, C++, Fortran, Python), custom cmake steps, etc. If this tool can handle that kind of mess, I'll be very interested to see what I can learn.

unddoch 1y ago
I wrote a little GCC plugin for compile time tracing/profiling, if that's something you're interested in: https://github.com/royjacobson/externis https://github.com/royjacobson/externis
- Night_Thastus 1y ago
  I just went ahead and tried it out :) I can get it to work for some sub-sets of our project, but for quite a bit of it I get the following error: cc1: error: cannot load plugin /opt/rh/gcc-toolset-13/root/usr/lib/gcc/x86_64-redhat-linux/13/plugin/externis.so: /opt/rh/gcc-toolset-13/root/usr/lib/gcc/x86_64-redhat-linux/13/plugin/externis.so: undefined symbol: _Z14decl_as_stringP9tree_nodei I suspect this is because these are C or Fortran sub-projects. I'm looking for some clean way to tell Cmake to apply externis to all the C++ only subprojects if possible. I'll see what I can come up with. I'd also like to know, if multiple GCC commands end up pointing to the same trace.json, especially in a parallel build, will externis automagically ensure that it doesn't step over itself?
- Night_Thastus 1y ago
  I figured out a way to set it at a top level, so it only happens with C++ files: target_compile_options(${NAME} PUBLIC $<$<COMPILE_LANGUAGE:CXX>:-fplugin=externis -fplugin-arg-externis-trace-dir=(where I want to put traces)> ) But as I suspected, it is not a single trace file. It's thousands of trace files. Is there some way to collate all the data into one larger picture of how the build progressed?
phaedrus 1y ago
When I was trying to improve compile time for my game engine, I ended up using compiled size as a proxy measure. Although it is an imperfect correlation, the fact that compiled size is deterministic across build runs and even across builds on different machines makes it easier to work with than wall clock time.
- mlsu 1y ago
  Wait, this is not intuitive at all for me. If the compiler is working harder wouldn't that result in a more compact binary? Maybe I'm thinking too much from an embedded software POV. I suppose the compiler does eventually do IO but IO isn't really the constraint most of the time right?
  - staticfloat 1y ago
    While you can cause the compiler to run longer to squeeze the binary size down, the compiler has a baseline number of compiler passes that it runs over the IR of the program being compiled. These compiler passes generally take time proportional to the input IR length, so a larger program takes longer to compile. Most compiler passes aren't throwing away huge amounts of instructions (dead code elimination being a notable exception, but the analysis to figure out which pieces of dead code can be eliminated still is operating on the input IR). So it's not a perfect proxy, but in general, if the output of your compiler is 2MB of code, it probably took longer to process all the input and spit out that 2MB than if the output of your compiler was 200KB.
  - johannes1234321 1y ago
    Of course there are the cases where a huge template structure with complex instantiation and constexpr code compiles down to a single constant, but for most parts of the code I would assume there is a proportion from code size, via compile time to binary size.
pklausler 1y ago
strace might help, if you have it.
- touisteur 1y ago
  perf record -e "syscalls:,sched:" is my go-to there - or just apply it on whole-system, aasone does.
1718627440 1y ago
Can you set CC=time gcc ?
wakawaka28 1y ago
It's not "nearly impossible" but actually built in: https://cmake.org/cmake/help/latest/manual/cmake.1.html#cmdoption-cmake-profiling-output https://cmake.org/cmake/help/latest/manual/cmake.1.html#cmdo... For the actual compile time you can easily insert a wrapper script. To be honest I haven't done that in over 4 years, but it has been done by many and it is easy. There may be times when CMake itself is the bottleneck but it is almost certainly an issue with your dependencies and so on. CMake has many features to assist you in speeding up your compile and link time too. But it would take a series of blog posts to describe how you should try to speed it up.
- Night_Thastus 1y ago
  Does that just profile cmake's time to configure and generate, or the underlying compile of each file as well? Configuration and generation are only seconds when done from scratch - the build is more like 20 minutes. Just trying to add that argument with 3.26.5 on Rocky Linux 9, I get 'Unknown argument --profiling-format=google-trace'. Not sure why, as cmake --help clearly states it should be there... --profiling-format=<fmt> = Output data for profiling CMake scripts. Supported formats: google-trace --profiling-output=<file> = Select an output path for the profiling data enabled through --profiling-format.
  - tom_ 1y ago
    You might need to specify --profiling-output= as well. I get an error fdrom cmake 3.31 if only the format is provided: CMake Error: --profiling-format specified but no --profiling-output! Anyway, it looks like it only profiles the configure/generate steps. Not much use on Linux, but on Windows/macOS, perhaps. Due to lack of any standard package manager, it's a good idea to build every dependency from source on those OSs, and the time can mounts up. My project is not that large, but it takes 1 minute to configure from scratch on Windows, and 10 minutes (!) on macOS.
    - Night_Thastus 1y ago
      I specified both. In any case, yeah, cmake time isn't useful in my case. Bizarre that you see 10 minutes on MacOS. Something's definitely busted there. It's not even that bad for me on Windows, and that's saying something.
deleted 1y ago
[deleted]
hagendaasalpine 1y ago
Tsoding wrote https://github.com/tsoding/nob.h https://github.com/tsoding/nob.h, single header C library for cross platform builds, only requirement is cc. GDB profiling tools can then be used to look at your build step. It's a neat idea. I suspect this is not an option but Nix is great build tool if you are dealing with multiple languages.
- jppittma 1y ago
  Btw, he has a YouTube channel and streams. I recommend it if you’re seeking imposter syndrome.
fransje26 1y ago
> I am stuck in an environment with CMake, GCC and Unix Make (no clang, no ninja) and getting detailed information about WHY the build is taking so long is nearly impossible. I have a similar problem, with a tangential question that I think about from time to time without really having the time to investigate it further, unfortunately. I notice sometimes that CMake recompiles files that shouldn't have been affected by the code changes made previously. Like recompiling some independent objects after only slight changes to a .cpp file without any interface changes. So I often wonder if CMake is not making some file more inter-dependent than what they are, leading to longer compilation times.

unddoch 1y ago

I wrote a little GCC plugin for compile time tracing/profiling, if that's something you're interested in: https://github.com/royjacobson/externis https://github.com/royjacobson/externis

Night_Thastus 1y ago
I just went ahead and tried it out :) I can get it to work for some sub-sets of our project, but for quite a bit of it I get the following error: cc1: error: cannot load plugin /opt/rh/gcc-toolset-13/root/usr/lib/gcc/x86_64-redhat-linux/13/plugin/externis.so: /opt/rh/gcc-toolset-13/root/usr/lib/gcc/x86_64-redhat-linux/13/plugin/externis.so: undefined symbol: _Z14decl_as_stringP9tree_nodei I suspect this is because these are C or Fortran sub-projects. I'm looking for some clean way to tell Cmake to apply externis to all the C++ only subprojects if possible. I'll see what I can come up with. I'd also like to know, if multiple GCC commands end up pointing to the same trace.json, especially in a parallel build, will externis automagically ensure that it doesn't step over itself?
Night_Thastus 1y ago
I figured out a way to set it at a top level, so it only happens with C++ files: target_compile_options(${NAME} PUBLIC $<$<COMPILE_LANGUAGE:CXX>:-fplugin=externis -fplugin-arg-externis-trace-dir=(where I want to put traces)> ) But as I suspected, it is not a single trace file. It's thousands of trace files. Is there some way to collate all the data into one larger picture of how the build progressed?

phaedrus 1y ago

When I was trying to improve compile time for my game engine, I ended up using compiled size as a proxy measure. Although it is an imperfect correlation, the fact that compiled size is deterministic across build runs and even across builds on different machines makes it easier to work with than wall clock time.

mlsu 1y ago
Wait, this is not intuitive at all for me. If the compiler is working harder wouldn't that result in a more compact binary? Maybe I'm thinking too much from an embedded software POV. I suppose the compiler does eventually do IO but IO isn't really the constraint most of the time right?
- staticfloat 1y ago
  While you can cause the compiler to run longer to squeeze the binary size down, the compiler has a baseline number of compiler passes that it runs over the IR of the program being compiled. These compiler passes generally take time proportional to the input IR length, so a larger program takes longer to compile. Most compiler passes aren't throwing away huge amounts of instructions (dead code elimination being a notable exception, but the analysis to figure out which pieces of dead code can be eliminated still is operating on the input IR). So it's not a perfect proxy, but in general, if the output of your compiler is 2MB of code, it probably took longer to process all the input and spit out that 2MB than if the output of your compiler was 200KB.
- johannes1234321 1y ago
  Of course there are the cases where a huge template structure with complex instantiation and constexpr code compiles down to a single constant, but for most parts of the code I would assume there is a proportion from code size, via compile time to binary size.

pklausler 1y ago

strace might help, if you have it.

touisteur 1y ago
perf record -e "syscalls:,sched:" is my go-to there - or just apply it on whole-system, aasone does.

1718627440 1y ago

Can you set CC=time gcc ?

wakawaka28 1y ago

It's not "nearly impossible" but actually built in: https://cmake.org/cmake/help/latest/manual/cmake.1.html#cmdoption-cmake-profiling-output https://cmake.org/cmake/help/latest/manual/cmake.1.html#cmdo... For the actual compile time you can easily insert a wrapper script. To be honest I haven't done that in over 4 years, but it has been done by many and it is easy. There may be times when CMake itself is the bottleneck but it is almost certainly an issue with your dependencies and so on. CMake has many features to assist you in speeding up your compile and link time too. But it would take a series of blog posts to describe how you should try to speed it up.

Night_Thastus 1y ago
Does that just profile cmake's time to configure and generate, or the underlying compile of each file as well? Configuration and generation are only seconds when done from scratch - the build is more like 20 minutes. Just trying to add that argument with 3.26.5 on Rocky Linux 9, I get 'Unknown argument --profiling-format=google-trace'. Not sure why, as cmake --help clearly states it should be there... --profiling-format=<fmt> = Output data for profiling CMake scripts. Supported formats: google-trace --profiling-output=<file> = Select an output path for the profiling data enabled through --profiling-format.
- tom_ 1y ago
  You might need to specify --profiling-output= as well. I get an error fdrom cmake 3.31 if only the format is provided: CMake Error: --profiling-format specified but no --profiling-output! Anyway, it looks like it only profiles the configure/generate steps. Not much use on Linux, but on Windows/macOS, perhaps. Due to lack of any standard package manager, it's a good idea to build every dependency from source on those OSs, and the time can mounts up. My project is not that large, but it takes 1 minute to configure from scratch on Windows, and 10 minutes (!) on macOS.
  - Night_Thastus 1y ago
    I specified both. In any case, yeah, cmake time isn't useful in my case. Bizarre that you see 10 minutes on MacOS. Something's definitely busted there. It's not even that bad for me on Windows, and that's saying something.

deleted 1y ago

[deleted]

hagendaasalpine 1y ago

Tsoding wrote https://github.com/tsoding/nob.h https://github.com/tsoding/nob.h, single header C library for cross platform builds, only requirement is cc. GDB profiling tools can then be used to look at your build step. It's a neat idea. I suspect this is not an option but Nix is great build tool if you are dealing with multiple languages.

jppittma 1y ago
Btw, he has a YouTube channel and streams. I recommend it if you’re seeking imposter syndrome.

fransje26 1y ago

> I am stuck in an environment with CMake, GCC and Unix Make (no clang, no ninja) and getting detailed information about WHY the build is taking so long is nearly impossible. I have a similar problem, with a tangential question that I think about from time to time without really having the time to investigate it further, unfortunately. I notice sometimes that CMake recompiles files that shouldn't have been affected by the code changes made previously. Like recompiling some independent objects after only slight changes to a .cpp file without any interface changes. So I often wonder if CMake is not making some file more inter-dependent than what they are, leading to longer compilation times.