Does CMake have to generate the file and pass it through the special compiler? Can't it be done separately, before CMake is run?
One thing that I find make CMake configurations completely unreadable is to try to have it do everything, coffee included. Where often a script could be run separately first (e.g. to generate those files), and then CMake could run normally afterwards.
> Does CMake have to generate the file and pass it through the special compiler? Can't it be done separately, before CMake is run?
No, it cannot be done before CMake runs. It is a target like any other build target. It depends on other files or targets, and should be rerun if any of them change, and if it is rebuilt, there are other targets that depend on it and should be rebuilt, and so on.
The entire point of the build system is to concisely and accurate specify the build graph, and only rebuild what is necessary.
Ok let's take a concrete example. Say you have proto files (seems likely in a big system). Whenever you change a proto file, you know that you should run protoc, and then run CMake. If you add a proto file, not only you need to run protoc on this new file, but you need to tell CMake about the new generated file(s).
For the big majority of projects out there, it seems to me that it is perfectly fine to handle that manually, because projects are not that big. Do we agree on that?
Then of course, if you are a GAFAM with a huge monorepo, probably you need more advanced tools (they don't build their own tools for nothing). My feeling, though, is that most people complaining that CMake is crap actually work on small projects and can't be bothered to run protoc manually "because their philosophy is that it should all work with one button".
No, I don't agree that having to manually run protoc after modifying a proto file is even slightly reasonable. That is exactly the sort of thing which the build system is there for, and the idea of not spending the few minutes it takes to automate that is baffling to me.
I automate it by putting it in a script that I call manually when needed, instead of spending days trying to have cmake run it.
Now it's not a pain for me, and apparently you're on the side of those who find cmake painful. Not sure how unreasonable my way is :)
> For the big majority of projects out there, it seems to me that it is perfectly fine to handle that manually, because projects are not that big. Do we agree on that?
Let's also agree that every time you change a .cpp file you compile it, and every time you change a .h file you compile all the affected .cpp files.
No we don't agree on that. That is the entire point of having a build system.
Even proto files you gave as an example, may import other proto files. When you change one of the imported files you have to recompile all proto files that import that file, transitively.
I have, unfortunately, worked with build systems that do not capture all dependencies and systems that capture way too many dependencies. The former resulted in many hours spent debugging non existent problems, and the latter resulted in many hours spent waiting for compilation to finish.
No, in most build systems it often can't be expressed precisely. It can be done in a way that will mostly work or be over approximated, though. So having some amount of abstraction capability is really important. Especially when you have a build pipeline involving something like "compile program G -> run G -> compile output of G", the output of G can often have many dependencies that are not statically known, or you have to repeat the logic of the generator program G inside the build system to declare them correctly, which is often error prone. So you normally just either have some incorrect logic where the build will fail -- because G started outputting code that needed "foobar.h" and someone forgot to update the dependency list to include "foobar.h", so now dependencies aren't tracked -- or some wildly conservative dependency graph that rebuilds things way more often than necessary, "the output of G depends on every header file possible." Now imagine that there are basically infinite variations on this theme and the cycle can continue infinitely, and it gets to be a problem.
Most people either work on programs small enough where having a conservative approximation of the dependency graph is "fine" and excessive rebuilds are OK, they are just compiling a simple static DAG of known source files, and if they encounter these issues, just structure the project with a little repetition between the between the build time dependencies and the code. Or, they just let the module system in their language of language choice figure it out. These are all completely OK. But it's very expensive to do conservative recompilation and very error prone to do manual repetition in larger projects, and gets very complex in multi-language setups.
You can sometimes structure programs to avoid these sorts of problems with enough foresight, but when you can't, or you need these things, you really need them, and the lack of it will really really hurt.
> […] and excessive rebuilds are OK.
Where does the presupposition of just some build tools, but not the others (if I am reading it correctly), resulting in excessive rebuilds come from?
Apart from building a dependencies DAG for a project, the next fundamental responsibility of a build tool is to watch for / detect changed graph nodes and apply relevant build rules (expressed as graph edges) to rebuild only the necessary parts of the graph by traversing the path upward (if the apex node is up at the top). Naturally, if it is a dependency at the very bottom of a very leafy graph that has changed and upstream nodes have a dependency on it, it will trigger a large (of a very large) rebuild of the entire dependency path. But that is true of all build systems, not just select ones, and that is what we need buid tools for.
The problem is that many people either can't or don't express dependency rules correctly or dependencies generate build byproducts that are to be dynamically injected into the DAG, and they not properly accounted for in the build rules, and that results in an incorrectly built DAG at the build system run time resulting in excessive rebuilds.
In the ideal world, dependency management is relegated to a module system of a programming language. But since long serving incumbents cough cough C, C++ and Java (not until Java 9 anyway) do not have a module system, people have created a kludge (make) and later multiple kludges (autoconf/automake, cmake, maven/ivy etc) to bandaid the problem.
However a generic build tool can't account for a permutation of all possible scenarios, all possible programming languages, linkers, pre- and post-processors and whatever else needs to be done to produce a build artefact and excel at it in every possible use case. Some do it better than the others in very specific scenarios but do poorly in some other or fringe use cases. Then there are personal preferences. Therefore, the discourse over which build system is better than the other(s) will never end.
> One thing that I find make CMake configurations completely unreadable is to try to have it do everything, coffee included.
and to me, build procedures that aren't completely implemented in cmake but require to install who knows what shell / platform / language / etc... in addition are the bane of my life as they make everything harder and more brittle and less cross-platform.
The build system should encapsulate everything needed to build the software, ideally with a single standard command. If your build system requires you to manually run build steps before or after it, your build system is broken (cmake unfortunately also often falls down in this regard, especially for cross-compiled projects).
So it's fine for you to have to manually run a sequence of steps (start your computer, open a terminal, navigate to the project, run the build, run the program), but it's unbearable if the build is split in a couple commands?
I don't get it. Also shell scripts are pretty good at running a couple commands, if really that's important.
If you start using shell scripts, you are throwing away the whole advantage of build systems, which is only rebuilding what is necessary. Using multiple manual build phases (even if you have a shell script calling them all) also increases contributor friction and is terrible for FLOSS.
Well, I run the scripts only when necessary. If I change an IDL file, I run the generation script. If I change a cpp file, I run the cmake build only.
I actually have maintained FLOSS projects, and in my experience, throwing everything in one cmake project makes it harder for contributors to understand. Because they only ever use one command, and the day something fails they have to dig into it to understand what that custom command does. On the contrary, if they need to run the generation script first, then they know that there is a generation step involved, and suddenly the whole thing feels less like magic.
So that is my opinion: if your FLOSS project uses a big custom build system (because you customized your cmake to do all sorts of custom things), then probably I will not contribute. Because I have other things to do than learning your custom integration of standards steps I actually would already know.