12 ms·

An Introduction to Modern CMake

131 points by betamark 7y ago

leggomylibro 7y ago
Nice to see a CMake guide here, but I don't see any discussion of generators at a cursory glance, even in the "IDEs" section. CMake is used a lot in embedded development with large or cross-platform projects, like an RTOS or the famous ESP-IDF. Unfortunately, the embedded development space is a messy hodge-podge of IDEs. There are shiny commercial ones like Keil which support most kinds of devices, but most vendors also provide a free Eclipse-based option which targets their chips, and there are usually flavors of GCC for the adventurous. CMake does have a nice way to add support for a new IDE or compiler in a modular way; you write a "generator": https://cmake.org/cmake/help/latest/manual/cmake-generators.7.html https://cmake.org/cmake/help/latest/manual/cmake-generators.... My pet peeve is, it seems like nobody does that. Instead, they try to re-write all of the logic which generates build files either using CMake's internal logic, or one-off scripts which get called from CMake. If (for example) Espressif provided a CMake generator for their style of `xtensa-elf-gcc` Makefiles, everybody could use that to quickly add build support for ESP32s to their CMake systems. What they use now doesn't seem to be very easy to re-use. Ditto for companies like ST/TI/Microchip/etc; if they provided CMake generators for their Eclipse IDEs, people wouldn't have to keep re-inventing the wheel and making it square-shaped. It makes me dread using embedded libraries that use CMake-based build systems, even though I really like CMake as a cross-platform build system.
- tjoff 7y ago
  Not sure you want to tie yourself to a buggy inconsistent IDE and force that upon all developers. Why wouldn't you do it with CMakes internal logic? Isn't that the point, especially since it is cross-platform.
  - leggomylibro 7y ago
    I don't think I understand the question. Generators are CMake's way of creating IDE-specific build files, and I wish that people used them instead of writing their own scripts to create IDE-specific build files from CMake. Generators are there so that you don't get tied to any one IDE; you select a different generator when you want to use a different IDE or compiler, but your CMake logic can largely stay the same besides that. If I were a cynic, I would say that the vendors probably don't mind it being difficult to interoperate with their competitors' products, so there's little incentive to support things like that.
- cardiffspaceman 7y ago
  I think the kind of embedded development you are doing is not the same as I am doing. Where I work we were looking at CMake and were "forced" to start learning it when we embarked on a pilot project that needed to compile WebKit for several embedded platforms (the kind that can usefully use WebKit to display something). There were at least 3 ways to build WebKit but someone before me chose CMake. Fast-forward to today, and our main build system is actually a wrapper around buildroot, which can use CMake projects for the packages if desired. We use CMake for one of our Linux kernel modules and for our middleware. We don't use generators, or to be precise we only use the Unix Makefile generator. For the CMake projects we adapt the compilers of our various platforms by using toolchain files, which we write ourselves. Toolchain files are chosen by an option to the initial CMake command and primarily describe the location and preferred settings of the toolchain's tools. There is a temptation, a very practical temptation, to merge toolchain considerations with the ways targets vary, and handle both subjects with the toolchain file. We have resisted that temptation across over a dozen target platforms. We create platform files separately and we have devised a convention for specifying which one to use. Our developers all have their own IDE/non-IDE ways of working with the source code and the build system has not asked to integrate in any way with that. A lot of our developers work primarily within the middleware, and we have a documented way to make that work easy. Others have to cope with buildroot as much as with the build system of the buildroot package they are working on, and that is not refined yet.
- AlotOfReading 7y ago
  I've built and maintained a generator for one of those vendor-specific eclipse IDEs. It was an absolute nightmare I don't recommend to anyone, but not for any of the reasons you'd imagine. CMake itself is easy to build / read. Implement a few functions and you'll quickly have something workable. The issues came from elsewhere: * None of the graphical CMake tools allow you to select the CMake binary, so you'd get users using the wrong binary and not seeing the generator mentioned in docs. * When things break elsewhere, it's "your responsibility" to fix it in the generator. People put meaningless, nonsensical junk in their CMake files and expect it to output something that builds. Doesn't matter that the build script or the IDE are fundamentally broken, people just want the output so they can get back to their work. They don't want to yak shave. * Vendor tooling code quality is somehow even worse than the actual product they provide. Dropping every 4096th character from stdin? Why not? Not generating debug info for files with uppercase letters in the filename? Sure. I won't name names, but they're a billion dollar EDA vendor whose initials are MG. Avoid CMake generators like the plague unless you want to spend the next bit of your life debugging your vendor tools and supporting every workflow quirk your coworkers have.
  - leggomylibro 7y ago
    Interesting perspective, thanks for sharing your experience! That doesn't sound too much worse than working with modern CMake build systems, but it also doesn't sound mucn better. >People put meaningless, nonsensical junk in their CMake files and expect it to output something that builds. Doesn't matter that the build script or the IDE are fundamentally broken, people just want the output so they can get back to their work. When you put it that way, maybe that's my real pet peeve with CMake in big projects :)
einpoklum 7y ago
I'm a person who has been easing into CMake use over the past few years. Something I've noticed is that there is no such thing as a single good introduction: * CMake itself is changing. While the "revolution" of modern CMake is upon us already from version 3.0 (or a little earlier), a bunch of thing change with versions 3.12, 3.13 etc. See: https://youtu.be/y7ndUhdQuU8 https://youtu.be/y7ndUhdQuU8 * Different treatments - a mini-website, PDF presentation or conference video - focus on different aspects of work with CMake, but you probably need most or all of them in focus in order to actually do what you need done. That's how you can see videos entitled "More Modern CMake" and "Oh No! More Modern CMake"... * Setting up the CMake machinery to have your repository/project installed is the opposite of straightforward, and very easy to get confused about. * There are a half-dozen different ways to have your project depend on on other code: External projects, package management CMake modules, relying on other projects being installed in a CMake-idiomatic way and then find_package working, and more. Regardless of which you think is the right one, you'll need to live with others doing something else. So, do follow the link, but don't expect it be sufficient.
- ausjke 7y ago
  then what's the point of cmake? won't legacy gnu make suffice? nothing is perfect including Make, but it seems gnu Make can put all its doc into one html manual at least. KISS. https://www.gnu.org/software/make/manual/make.html https://www.gnu.org/software/make/manual/make.html
  - ska 7y ago
    cross platform mostly works.
  - ridiculous_fish 7y ago
    Writing Makefiles by hand is great for small projects. But make becomes unmanageable for large projects, and is bad at dealing with system and compiler differences.
    - einpoklum 7y ago
ddevault 7y ago
Don't use cmake for new projects. You'll regret it. Try meson instead: https://mesonbuild.com/ https://mesonbuild.com/
- aprdm 7y ago
  Unfortunately even if you're starting a new project chances are that your dependencies are using cmake and they expect their dependencies to use cmake... in the end of the day you spend a lot of time fixing the upstream projects build files. More than that CMake at this point is the ""standard"" and there are many others with a similar share of the market and selling points, it makes it harder to pick one that is not CMake.
  - TonCherAmi 7y ago
    Meson supports CMake subprojects: https://mesonbuild.com/CMake-module.html#cmake-subprojects https://mesonbuild.com/CMake-module.html#cmake-subprojects
    - Johnnynator 7y ago
      CMake subprojects was far from reliable last time I tested it. But this might also have been due to meson targeting unreleased CMake versions...
  - stefan_ 7y ago
    If your dependencies use CMake or not doesn't really matter, you still end up writing some FindX.cmake file. (Which is a whole other thing with this "modern CMake" by the way: only half of them that come with CMake are "upgraded" to modern and if you don't want to require the absolute latest version, you end up having to check what the FindX script did or redefine stuff manually)
- mhd 7y ago
  Doesn't meson bring in all of python3 as a dependency for a project? Similar to scons. (It's been a while since I did semi-serious C++, maybe nowadays we're playing a bit more loose with deps and polyglot setups)
dang 7y ago
Related from 2018: https://news.ycombinator.com/item?id=17897685 https://news.ycombinator.com/item?id=17897685
ajross 7y ago
This has the same kind of infuriating quality that most cmake documentation does. It explains in detail how to do obvious stuff but has nothing to say about what the execution model behind the syntax is. I mean, if you learn classic make, one of the very first things you learn is "this is a target; it's just a file; the stuff on the right are dependencies; the commands below will be run whenever the dependencies are newer than the target". Now, there are implications to learn, but that is the model, and it's obvious: make runs programs to create files when the stuff needed to create the file changes. Now, how do I do that in cmake? It's almost not even here! It's all about C++ files and libraries and how to define variables and write functions and include other cmake files. But what is it going to do with that stuff? It's all just magic. Type it like this and it'll work. Cmake understands your compiler better than you do. What docs there are on creating a custom target are buried in two paragraphs inside a subsection named "Running a command at build time". And it barely tells you how to do this at all (this being a topic I've actually struggled with in the past). In particular it fails to mention that a target is is not the same thing as a file in cmake, and that this has implications that are likely to be surprising to make experts.
- tannhaeuser 7y ago
  I'm glad I'm not the only one finding cmake's "magic" approach to software building totally unhelpful, and more on the problem rather than solution side of things. You know there's gone something out of hand if your very meta build tool's build itself takes much longer than the project you wanted to build in the first place.
- jdance 7y ago
  Thank you! I cannot count the number of times I have had a CMake problem and had the usual inclination to try to understand the framework behind it, only to be stonewalled by the seemlingly complete lack of context in the documentation. I have given up on it time and time again. Somehow my project manages to barely get over every CMake hurdle but I still dream about the day when I would actually understand it, and have the same feeling of control that I usually strive for.
peter_d_sherman 7y ago
Excerpt: "The --trace option will print every line of CMake that is run. Since this is very verbose, CMake 3.7 added --trace-source="filename", which will print out every executed line of just the file you are interested in when it runs. If you select the name of the file you are interested in debugging (usually by selecting the parent directory when debugging a CMakeLists.txt, since all of those have the same name), you can just see the lines that run in that file. Very useful!"
Robadob 7y ago
Recent versions of CMake have also been growing the support for CUDA. There's still a number of edge cases (e.g. passing arguments to the device linker) that have open tickets on their bugtracker, but there's more than enough stable functionality now for CUDA projects and generally a number of relevant improvements with each new release. We started using it about 6 months ago for a cross platform CUDA project, and have found it to be a big improvement over how we previously managed similar CUDA projects (e.g. manually). I'm especially a fan of it's ability to download external CMake project dependencies. As others have mentioned, some of the documentation is a little unclear for particular edge cases though so it takes some familiarisation.
irishcoffee 7y ago
I sure wish Qmake had won over CMake.
- j1elo 7y ago
  Even they recognized how messy Qmake is/was, and then started working on QBS
  - pjmlp 7y ago
    And now Qbs is in maintenance and they will also converge into CMake.
- deleted 7y ago
  [deleted]
smabie 7y ago
What's wrong with just a real language as a build system? Speaking for Scala, everyone just uses sbt, which is just Scala. Nothing special about it. I'm not saying you should use Scala to compile your C project, but maybe we could settle on some Lisp dialect or something for compilation, regardless of the language? That would seem ideal to me. Programmers in general have this problem of creating non-turing complete shit to solve problems that require turing completeness. And sometimes it gets worse: first a non turing complete language is created and then when it finally becomes necessary, some loops and shit are grafted on (or a shell script is called in the middle). It's a mess.
- pjmlp 7y ago
  Eventually those scripts will turn into full fledged programs that no one knows what they do, specially since most of these build programming languages lack debugging features.
waruqi 7y ago
Or we can also try xmake. https://xmake.io/ https://xmake.io/
- waruqi 7y ago
  The dev branch version is already as fast as ninja now, and there is no additional step to generate a makefile/build.ninja.
maple3142 7y ago
I always wondering why CMake can't properly support filenames with white space. Is there a reason why no one can fix it rather than recommend CMake users to avoid white space?
lenkite 7y ago
For full-fledged CMake project following best practices, see C++ Weeklys CMake tutorial: https://www.youtube.com/watch?v=YbgH7yat-Jo https://www.youtube.com/watch?v=YbgH7yat-Jo For C++ starter project, see: https://github.com/lefticus/cpp_starter_project https://github.com/lefticus/cpp_starter_project (I agree with nearly everything except generating the conanfile)

131 points by betamark 7y ago

Nice to see a CMake guide here, but I don't see any discussion of generators at a cursory glance, even in the "IDEs" section. CMake is used a lot in embedded development with large or cross-platform projects, like an RTOS or the famous ESP-IDF. Unfortunately, the embedded development space is a messy hodge-podge of IDEs. There are shiny commercial ones like Keil which support most kinds of devices, but most vendors also provide a free Eclipse-based option which targets their chips, and there are usually flavors of GCC for the adventurous. CMake does have a nice way to add support for a new IDE or compiler in a modular way; you write a "generator": https://cmake.org/cmake/help/latest/manual/cmake-generators.7.html https://cmake.org/cmake/help/latest/manual/cmake-generators.... My pet peeve is, it seems like nobody does that. Instead, they try to re-write all of the logic which generates build files either using CMake's internal logic, or one-off scripts which get called from CMake. If (for example) Espressif provided a CMake generator for their style of `xtensa-elf-gcc` Makefiles, everybody could use that to quickly add build support for ESP32s to their CMake systems. What they use now doesn't seem to be very easy to re-use. Ditto for companies like ST/TI/Microchip/etc; if they provided CMake generators for their Eclipse IDEs, people wouldn't have to keep re-inventing the wheel and making it square-shaped. It makes me dread using embedded libraries that use CMake-based build systems, even though I really like CMake as a cross-platform build system.

tjoff 7y ago
Not sure you want to tie yourself to a buggy inconsistent IDE and force that upon all developers. Why wouldn't you do it with CMakes internal logic? Isn't that the point, especially since it is cross-platform.
- leggomylibro 7y ago
  I don't think I understand the question. Generators are CMake's way of creating IDE-specific build files, and I wish that people used them instead of writing their own scripts to create IDE-specific build files from CMake. Generators are there so that you don't get tied to any one IDE; you select a different generator when you want to use a different IDE or compiler, but your CMake logic can largely stay the same besides that. If I were a cynic, I would say that the vendors probably don't mind it being difficult to interoperate with their competitors' products, so there's little incentive to support things like that.
cardiffspaceman 7y ago
I think the kind of embedded development you are doing is not the same as I am doing. Where I work we were looking at CMake and were "forced" to start learning it when we embarked on a pilot project that needed to compile WebKit for several embedded platforms (the kind that can usefully use WebKit to display something). There were at least 3 ways to build WebKit but someone before me chose CMake. Fast-forward to today, and our main build system is actually a wrapper around buildroot, which can use CMake projects for the packages if desired. We use CMake for one of our Linux kernel modules and for our middleware. We don't use generators, or to be precise we only use the Unix Makefile generator. For the CMake projects we adapt the compilers of our various platforms by using toolchain files, which we write ourselves. Toolchain files are chosen by an option to the initial CMake command and primarily describe the location and preferred settings of the toolchain's tools. There is a temptation, a very practical temptation, to merge toolchain considerations with the ways targets vary, and handle both subjects with the toolchain file. We have resisted that temptation across over a dozen target platforms. We create platform files separately and we have devised a convention for specifying which one to use. Our developers all have their own IDE/non-IDE ways of working with the source code and the build system has not asked to integrate in any way with that. A lot of our developers work primarily within the middleware, and we have a documented way to make that work easy. Others have to cope with buildroot as much as with the build system of the buildroot package they are working on, and that is not refined yet.
AlotOfReading 7y ago
I've built and maintained a generator for one of those vendor-specific eclipse IDEs. It was an absolute nightmare I don't recommend to anyone, but not for any of the reasons you'd imagine. CMake itself is easy to build / read. Implement a few functions and you'll quickly have something workable. The issues came from elsewhere: * None of the graphical CMake tools allow you to select the CMake binary, so you'd get users using the wrong binary and not seeing the generator mentioned in docs. * When things break elsewhere, it's "your responsibility" to fix it in the generator. People put meaningless, nonsensical junk in their CMake files and expect it to output something that builds. Doesn't matter that the build script or the IDE are fundamentally broken, people just want the output so they can get back to their work. They don't want to yak shave. * Vendor tooling code quality is somehow even worse than the actual product they provide. Dropping every 4096th character from stdin? Why not? Not generating debug info for files with uppercase letters in the filename? Sure. I won't name names, but they're a billion dollar EDA vendor whose initials are MG. Avoid CMake generators like the plague unless you want to spend the next bit of your life debugging your vendor tools and supporting every workflow quirk your coworkers have.
- leggomylibro 7y ago
  Interesting perspective, thanks for sharing your experience! That doesn't sound too much worse than working with modern CMake build systems, but it also doesn't sound mucn better. >People put meaningless, nonsensical junk in their CMake files and expect it to output something that builds. Doesn't matter that the build script or the IDE are fundamentally broken, people just want the output so they can get back to their work. When you put it that way, maybe that's my real pet peeve with CMake in big projects :)

einpoklum 7y ago

I'm a person who has been easing into CMake use over the past few years. Something I've noticed is that there is no such thing as a single good introduction: * CMake itself is changing. While the "revolution" of modern CMake is upon us already from version 3.0 (or a little earlier), a bunch of thing change with versions 3.12, 3.13 etc. See: https://youtu.be/y7ndUhdQuU8 https://youtu.be/y7ndUhdQuU8 * Different treatments - a mini-website, PDF presentation or conference video - focus on different aspects of work with CMake, but you probably need most or all of them in focus in order to actually do what you need done. That's how you can see videos entitled "More Modern CMake" and "Oh No! More Modern CMake"... * Setting up the CMake machinery to have your repository/project installed is the opposite of straightforward, and very easy to get confused about. * There are a half-dozen different ways to have your project depend on on other code: External projects, package management CMake modules, relying on other projects being installed in a CMake-idiomatic way and then find_package working, and more. Regardless of which you think is the right one, you'll need to live with others doing something else. So, do follow the link, but don't expect it be sufficient.

ausjke 7y ago
then what's the point of cmake? won't legacy gnu make suffice? nothing is perfect including Make, but it seems gnu Make can put all its doc into one html manual at least. KISS. https://www.gnu.org/software/make/manual/make.html https://www.gnu.org/software/make/manual/make.html
- ska 7y ago
  cross platform mostly works.
- ridiculous_fish 7y ago
  Writing Makefiles by hand is great for small projects. But make becomes unmanageable for large projects, and is bad at dealing with system and compiler differences.
  - einpoklum 7y ago

ddevault 7y ago

Don't use cmake for new projects. You'll regret it. Try meson instead: https://mesonbuild.com/ https://mesonbuild.com/

aprdm 7y ago
Unfortunately even if you're starting a new project chances are that your dependencies are using cmake and they expect their dependencies to use cmake... in the end of the day you spend a lot of time fixing the upstream projects build files. More than that CMake at this point is the ""standard"" and there are many others with a similar share of the market and selling points, it makes it harder to pick one that is not CMake.
- TonCherAmi 7y ago
  Meson supports CMake subprojects: https://mesonbuild.com/CMake-module.html#cmake-subprojects https://mesonbuild.com/CMake-module.html#cmake-subprojects
  - Johnnynator 7y ago
    CMake subprojects was far from reliable last time I tested it. But this might also have been due to meson targeting unreleased CMake versions...
- stefan_ 7y ago
  If your dependencies use CMake or not doesn't really matter, you still end up writing some FindX.cmake file. (Which is a whole other thing with this "modern CMake" by the way: only half of them that come with CMake are "upgraded" to modern and if you don't want to require the absolute latest version, you end up having to check what the FindX script did or redefine stuff manually)
mhd 7y ago
Doesn't meson bring in all of python3 as a dependency for a project? Similar to scons. (It's been a while since I did semi-serious C++, maybe nowadays we're playing a bit more loose with deps and polyglot setups)

dang 7y ago

Related from 2018: https://news.ycombinator.com/item?id=17897685 https://news.ycombinator.com/item?id=17897685

ajross 7y ago

This has the same kind of infuriating quality that most cmake documentation does. It explains in detail how to do obvious stuff but has nothing to say about what the execution model behind the syntax is. I mean, if you learn classic make, one of the very first things you learn is "this is a target; it's just a file; the stuff on the right are dependencies; the commands below will be run whenever the dependencies are newer than the target". Now, there are implications to learn, but that is the model, and it's obvious: make runs programs to create files when the stuff needed to create the file changes. Now, how do I do that in cmake? It's almost not even here! It's all about C++ files and libraries and how to define variables and write functions and include other cmake files. But what is it going to do with that stuff? It's all just magic. Type it like this and it'll work. Cmake understands your compiler better than you do. What docs there are on creating a custom target are buried in two paragraphs inside a subsection named "Running a command at build time". And it barely tells you how to do this at all (this being a topic I've actually struggled with in the past). In particular it fails to mention that a target is is not the same thing as a file in cmake, and that this has implications that are likely to be surprising to make experts.

tannhaeuser 7y ago
I'm glad I'm not the only one finding cmake's "magic" approach to software building totally unhelpful, and more on the problem rather than solution side of things. You know there's gone something out of hand if your very meta build tool's build itself takes much longer than the project you wanted to build in the first place.
jdance 7y ago
Thank you! I cannot count the number of times I have had a CMake problem and had the usual inclination to try to understand the framework behind it, only to be stonewalled by the seemlingly complete lack of context in the documentation. I have given up on it time and time again. Somehow my project manages to barely get over every CMake hurdle but I still dream about the day when I would actually understand it, and have the same feeling of control that I usually strive for.

peter_d_sherman 7y ago

Excerpt: "The --trace option will print every line of CMake that is run. Since this is very verbose, CMake 3.7 added --trace-source="filename", which will print out every executed line of just the file you are interested in when it runs. If you select the name of the file you are interested in debugging (usually by selecting the parent directory when debugging a CMakeLists.txt, since all of those have the same name), you can just see the lines that run in that file. Very useful!"

Robadob 7y ago

Recent versions of CMake have also been growing the support for CUDA. There's still a number of edge cases (e.g. passing arguments to the device linker) that have open tickets on their bugtracker, but there's more than enough stable functionality now for CUDA projects and generally a number of relevant improvements with each new release. We started using it about 6 months ago for a cross platform CUDA project, and have found it to be a big improvement over how we previously managed similar CUDA projects (e.g. manually). I'm especially a fan of it's ability to download external CMake project dependencies. As others have mentioned, some of the documentation is a little unclear for particular edge cases though so it takes some familiarisation.

irishcoffee 7y ago

I sure wish Qmake had won over CMake.

j1elo 7y ago
Even they recognized how messy Qmake is/was, and then started working on QBS
- pjmlp 7y ago
  And now Qbs is in maintenance and they will also converge into CMake.
deleted 7y ago
[deleted]

smabie 7y ago

What's wrong with just a real language as a build system? Speaking for Scala, everyone just uses sbt, which is just Scala. Nothing special about it. I'm not saying you should use Scala to compile your C project, but maybe we could settle on some Lisp dialect or something for compilation, regardless of the language? That would seem ideal to me. Programmers in general have this problem of creating non-turing complete shit to solve problems that require turing completeness. And sometimes it gets worse: first a non turing complete language is created and then when it finally becomes necessary, some loops and shit are grafted on (or a shell script is called in the middle). It's a mess.

pjmlp 7y ago
Eventually those scripts will turn into full fledged programs that no one knows what they do, specially since most of these build programming languages lack debugging features.

waruqi 7y ago

Or we can also try xmake. https://xmake.io/ https://xmake.io/

waruqi 7y ago
The dev branch version is already as fast as ninja now, and there is no additional step to generate a makefile/build.ninja.

maple3142 7y ago

I always wondering why CMake can't properly support filenames with white space. Is there a reason why no one can fix it rather than recommend CMake users to avoid white space?

lenkite 7y ago

For full-fledged CMake project following best practices, see C++ Weeklys CMake tutorial: https://www.youtube.com/watch?v=YbgH7yat-Jo https://www.youtube.com/watch?v=YbgH7yat-Jo For C++ starter project, see: https://github.com/lefticus/cpp_starter_project https://github.com/lefticus/cpp_starter_project (I agree with nearly everything except generating the conanfile)