3 ms·

I should rephrase, by dependencies I mean third party dependencies. That is on the build system not the compiler. And make tends to fall significantly behind N

by dagmx 3y ago

I should rephrase, by dependencies I mean third party dependencies. That is on the build system not the compiler. And make tends to fall significantly behind Ninja in my experience for cold builds. Here’s a post from someone else that echoes my experience https://david.rothlis.net/ninja-benchmark/ https://david.rothlis.net/ninja-benchmark/ but for a sufficiently complex project even a well tuned make build is about 20% slower on CI, which is about 10-20m per build. It also seems like you’re giving make the benefit of the doubt when it comes to “most being badly written but my own is great”, while simultaneously deriding cmake without giving it the same benefit. But to your larger point, and going back to my C vs other languages comparison (let’s pick C++ for arguments sake). Yes Make can do it all, but even by your own statements it requires doing everything correctly and there’s significant room for error. CMake makes the trade off of abstraction for a more consistent experience with minimal work. This is similar to C and C++, where yes C can (1) do much better in the very very specific right hands, but (2) it can also become an unwieldy mess. C++ with RAII may seem perplexing from the lens of only accepting the former and not the latter. It lets people spend their time elsewhere on the system.

t43562 3y ago
Ninja excludes a lot of features of make which are expensive to implement and cause it to be slower to parse so naturally in certain situations ninja runs faster than make. If your build time is actually affected by how long it takes to read the makefiles then this is important (e.g. building Android).
- adrian_b 3y ago
  When the makefiles are well organized, all of them have just a few lines, with a few definitions, perhaps at most ten lines, but usually less, and they include a single bigger common Makefile, with most of the definitions, all the rules and all the make targets. So reading all the makefiles should take a negligible time, even in a big project. I have seen too many projects with huge makefiles and with much more makefiles than necessary, but all of those are examples of misuse of make, which are easier solved by an efficient use of make, instead of by replacing make with another tool. For instance, there are many projects with a Makefile in each directory with source files, which is a very bad choice that multiplies the number of makefiles. Makefiles should exist only in a dedicated build directory, not in source directories, except for very small projects with a handful of source files. The best way IMO, is to have a single very small Makefile for each final file that must be built, e.g. executable file or library, in a dedicated subdirectory of the build directory. This allows for a maximum simplification of the Makefiles and their number and size is independent on the number of source directories and on the number of source files.
  - t43562 3y ago
    Make does a lot more than just reading makefiles - sometimes it re-reads them as a result of something that happened in the makefile itself. It's also doing a lot of variable expansion and if the makefile uses macros like define....endef to generate rules it has to expand those. Make is also often trying all sorts of pattern rules to see if they will allow it to fulfill dependencies. This takes time. If makefiles are small and you run make on each one separately then you miss dependencies across modules and you get the syndrome where program A doesn't get rebuild when library B changes. So there's a price to pay for everything. Ninja is a lot faster because it just doesn't have some of these features and you can make a big makefile where dependencies work properly with it and not pay the price of slow parsing. I worked on builds that took 12 hours on a large cluster of build machines where parsing was 45 minutes - so it matters. The more general solution would be to use a packaging system like the Linux distributions but even that doesn't save you from every problem as I also found out when working with OBS on a linux phone distribution.
adrian_b 3y ago
The link provided by you shows correctly that Ninja is indeed faster in the cases when almost nothing is recompiled, so the command execution time is dominated by the dependency evaluation time, which is faster in Ninja. However, the same link shows that even for relatively large projects the time difference is less than a second, which matches my experience. Ninja requires much more effort for using, as it is not a complete build system, but just the execution component, and except for projects of chrome size that effort is not worthwhile for a subsecond gain in certain scenarios. I cannot imagine how a "tuned" make build can be 20% slower, except when the project is not really tuned, but it is badly organized. A "tuned" Makefile means that the execution time for "make" is negligible in comparison with the execution times of the compilers and linkers. The time spent in "make" should be measured in seconds an most, never in minutes. Only a very slow file system can cause "make" to be much slower than Ninja, because the dependencies for "make" are stored in many ".d" files, equal in number with the source files. On modern SSDs or RAM disks, that is never a problem. Perhaps you are right about CMake. I cannot be certain whether CMake is good or bad, because I have never created a CMake project myself, I have only built CMake projects created by others. Nevertheless, there certainly is a problem with the CMake users that I have not seen yet one that can explain which are the advantages of CMake. All the tutorials that I have ever seen about CMake were showing how to do in a complex way things that can be done in a simple way with GNU make, so I never had any reason to investigate any further. What I know for sure is that much fewer understand how to use CMake than how to use make, because the frequency of bugs in CMake projects is much greater. I would like to see how CMake can provide minimal work. With GNU make, creating a new very simple project needs only copying a template Makefile in the source directory. For a more typical software project, the template Makefile must be edited to add a list of directories, those that must be searched for source files (I actually write the list as a prefix directory plus a list of subdirectories for it, as that is how I normally structure the projects). For external dependencies, up to 3 lists may be added, of libraries, include directories and perhaps of library directories, which should suffice. That is all. Nothing more. I normally build each target file of a project, e.g. executable or library, in a separate subdirectory of the build directory, so by default it is not necessary to write it in the Makefile, because the name of the subdirectory will be used for the target. The root of the build directory contains a Makefile that I never change and which descends in each subdirectory to build its target. How can be CMake more simple to use than this? All CMake projects that I have ever seen were much more complicated and they were very difficult to modify, with a very large number of project-specific details that have no place in the project configuration files, because the building system should be able to deduce them. All the examples of using CMake that I have ever seen have demonstrated a tool with a much lower level of abstraction than GNU make and which is tedious to use. GNU make does not need any such thing like the CMakeLists that are ubiquitous with CMake.

by dagmx 3y ago

t43562 3y ago

Ninja excludes a lot of features of make which are expensive to implement and cause it to be slower to parse so naturally in certain situations ninja runs faster than make. If your build time is actually affected by how long it takes to read the makefiles then this is important (e.g. building Android).

adrian_b 3y ago
When the makefiles are well organized, all of them have just a few lines, with a few definitions, perhaps at most ten lines, but usually less, and they include a single bigger common Makefile, with most of the definitions, all the rules and all the make targets. So reading all the makefiles should take a negligible time, even in a big project. I have seen too many projects with huge makefiles and with much more makefiles than necessary, but all of those are examples of misuse of make, which are easier solved by an efficient use of make, instead of by replacing make with another tool. For instance, there are many projects with a Makefile in each directory with source files, which is a very bad choice that multiplies the number of makefiles. Makefiles should exist only in a dedicated build directory, not in source directories, except for very small projects with a handful of source files. The best way IMO, is to have a single very small Makefile for each final file that must be built, e.g. executable file or library, in a dedicated subdirectory of the build directory. This allows for a maximum simplification of the Makefiles and their number and size is independent on the number of source directories and on the number of source files.
- t43562 3y ago
  Make does a lot more than just reading makefiles - sometimes it re-reads them as a result of something that happened in the makefile itself. It's also doing a lot of variable expansion and if the makefile uses macros like define....endef to generate rules it has to expand those. Make is also often trying all sorts of pattern rules to see if they will allow it to fulfill dependencies. This takes time. If makefiles are small and you run make on each one separately then you miss dependencies across modules and you get the syndrome where program A doesn't get rebuild when library B changes. So there's a price to pay for everything. Ninja is a lot faster because it just doesn't have some of these features and you can make a big makefile where dependencies work properly with it and not pay the price of slow parsing. I worked on builds that took 12 hours on a large cluster of build machines where parsing was 45 minutes - so it matters. The more general solution would be to use a packaging system like the Linux distributions but even that doesn't save you from every problem as I also found out when working with OBS on a linux phone distribution.

adrian_b 3y ago

The link provided by you shows correctly that Ninja is indeed faster in the cases when almost nothing is recompiled, so the command execution time is dominated by the dependency evaluation time, which is faster in Ninja. However, the same link shows that even for relatively large projects the time difference is less than a second, which matches my experience. Ninja requires much more effort for using, as it is not a complete build system, but just the execution component, and except for projects of chrome size that effort is not worthwhile for a subsecond gain in certain scenarios. I cannot imagine how a "tuned" make build can be 20% slower, except when the project is not really tuned, but it is badly organized. A "tuned" Makefile means that the execution time for "make" is negligible in comparison with the execution times of the compilers and linkers. The time spent in "make" should be measured in seconds an most, never in minutes. Only a very slow file system can cause "make" to be much slower than Ninja, because the dependencies for "make" are stored in many ".d" files, equal in number with the source files. On modern SSDs or RAM disks, that is never a problem. Perhaps you are right about CMake. I cannot be certain whether CMake is good or bad, because I have never created a CMake project myself, I have only built CMake projects created by others. Nevertheless, there certainly is a problem with the CMake users that I have not seen yet one that can explain which are the advantages of CMake. All the tutorials that I have ever seen about CMake were showing how to do in a complex way things that can be done in a simple way with GNU make, so I never had any reason to investigate any further. What I know for sure is that much fewer understand how to use CMake than how to use make, because the frequency of bugs in CMake projects is much greater. I would like to see how CMake can provide minimal work. With GNU make, creating a new very simple project needs only copying a template Makefile in the source directory. For a more typical software project, the template Makefile must be edited to add a list of directories, those that must be searched for source files (I actually write the list as a prefix directory plus a list of subdirectories for it, as that is how I normally structure the projects). For external dependencies, up to 3 lists may be added, of libraries, include directories and perhaps of library directories, which should suffice. That is all. Nothing more. I normally build each target file of a project, e.g. executable or library, in a separate subdirectory of the build directory, so by default it is not necessary to write it in the Makefile, because the name of the subdirectory will be used for the target. The root of the build directory contains a Makefile that I never change and which descends in each subdirectory to build its target. How can be CMake more simple to use than this? All CMake projects that I have ever seen were much more complicated and they were very difficult to modify, with a very large number of project-specific details that have no place in the project configuration files, because the building system should be able to deduce them. All the examples of using CMake that I have ever seen have demonstrated a tool with a much lower level of abstraction than GNU make and which is tedious to use. GNU make does not need any such thing like the CMakeLists that are ubiquitous with CMake.