5 ms·

I think you’re overstating the ease of use of make. How does one discover dependencies in a cross platform way with Make without writing the logic themselves t

by dagmx 3y ago

I think you’re overstating the ease of use of make. How does one discover dependencies in a cross platform way with Make without writing the logic themselves to stay up to date with platform changes? Or picking up configuration options from dependencies. How does one make use of Ninja with make? Or discover changes to sdk paths for new platforms? You end up having to duplicate that logic across every repo that needs it. Along the way you accrue lots of variance. In the end you get something that is difficult to debug and scale. That’s why so many projects moved to cmake. It scales better. In much the same way that you can technically do anything you need with C, but other languages add ergonomics and scalability that people value more.

adrian_b 3y ago
Discovering the dependencies depends on the compilers, not on the platform. I have stopped using MSVC many years ago, so I do not know how it handles dependencies, but with gcc or clang that works regardless of platform, with compiler-specific options. Moreover, software for one platform can be built on another platform. What matters is only the platform used for building, where the compilers are hosted, not the target platform for the built software. In general, I extract all platform specific definitions, like the names of the compilers and their command-line options in platform-specific Makefiles, which include a generic Makefile with platform-independent definitions, rules and make targets. Any platform-specific details, like SDK paths, are encapsulated in the definitions of certain "make" variables. While I have not needed to update the platform-independent generic Makefile for decades, the platform-dependent sections need updates from time to time, but that is not done for every software project, but perhaps once per year or more seldom, when significant new compiler versions, standard libraries or other new software tools become available. I have not seen yet any reason for using Ninja. Perhaps it may be faster when building something like chrome, but because chrome is likely to be the slowest-compiling software project known to mankind, it is hard to tell how much is gained by Ninja. For software projects of more typical sizes I have not seen noticeable advantages of Ninja. Traditional make can keep all the cores of a CPU 100% busy, so there is no way any other building system can be appreciably faster. There are some very bad Makefiles that are intrinsically slow, but that is a problem of those Makefiles, not of "make" in general. Using a minimum project-specific Makefile section can only simplify debugging, not making it difficult. I have never seen any evidence for the claim that cmake scales better. On the contrary, at least in all the open-source software projects that I have ever seen cmake is a major source of bugs in the building process that nobody knows how to fix. (i.e. after some software updates cmake fails to find files that exist, presumably due to some errors in the cmakelists that appear to be arcane enough that they are hard to find) I have never seen in projects that use traditional make such errors that are so frequent in all the projects that use cmake. I have a lot of experience in building software projects, because for many decades, with the exception of a few professional programs without alternatives, I have been using only programs that I compile from sources. Most projects have very bad Makefiles, which are very hard to maintain, i.e. to change when files are added, deleted, moved or renamed, but at least they build the software projects reliably. Whenever cmake is used, building a new version is always an adventure with unpredictable outcome.
- dagmx 3y ago
  I should rephrase, by dependencies I mean third party dependencies. That is on the build system not the compiler. And make tends to fall significantly behind Ninja in my experience for cold builds. Here’s a post from someone else that echoes my experience https://david.rothlis.net/ninja-benchmark/ https://david.rothlis.net/ninja-benchmark/ but for a sufficiently complex project even a well tuned make build is about 20% slower on CI, which is about 10-20m per build. It also seems like you’re giving make the benefit of the doubt when it comes to “most being badly written but my own is great”, while simultaneously deriding cmake without giving it the same benefit. But to your larger point, and going back to my C vs other languages comparison (let’s pick C++ for arguments sake). Yes Make can do it all, but even by your own statements it requires doing everything correctly and there’s significant room for error. CMake makes the trade off of abstraction for a more consistent experience with minimal work. This is similar to C and C++, where yes C can (1) do much better in the very very specific right hands, but (2) it can also become an unwieldy mess. C++ with RAII may seem perplexing from the lens of only accepting the former and not the latter. It lets people spend their time elsewhere on the system.
  - t43562 3y ago
    Ninja excludes a lot of features of make which are expensive to implement and cause it to be slower to parse so naturally in certain situations ninja runs faster than make. If your build time is actually affected by how long it takes to read the makefiles then this is important (e.g. building Android).
    - adrian_b 3y ago
      When the makefiles are well organized, all of them have just a few lines, with a few definitions, perhaps at most ten lines, but usually less, and they include a single bigger common Makefile, with most of the definitions, all the rules and all the make targets. So reading all the makefiles should take a negligible time, even in a big project. I have seen too many projects with huge makefiles and with much more makefiles than necessary, but all of those are examples of misuse of make, which are easier solved by an efficient use of make, instead of by replacing make with another tool. For instance, there are many projects with a Makefile in each directory with source files, which is a very bad choice that multiplies the number of makefiles. Makefiles should exist only in a dedicated build directory, not in source directories, except for very small projects with a handful of source files. The best way IMO, is to have a single very small Makefile for each final file that must be built, e.g. executable file or library, in a dedicated subdirectory of the build directory. This allows for a maximum simplification of the Makefiles and their number and size is independent on the number of source directories and on the number of source files.
nocombination 3y ago
Funny to mention cmake when GNU Autotools handles all of these dependency resolution with grace using only a few lines of autoconf. CMake script syntax is not very great, and debugging is no better than sprinkling printf's throughout... So, why not use Autotools? /bin/sh has been the norm for most build processes, to the point where the later-designed YaML syntax is _effectively_ the SAME as a Makefile with /bin/sh statements running the pipeline procedures. CMake came along and tried to fix complexity in Autotools while adding kitchen-sink baggage along the way.
- dagmx 3y ago
  I’m not specifically advocating for CMake , though it is my preference for many reasons like faster uptake of multiple platform and compiler features. I was merely pushing back on the person perplexing on why people don’t just use Make.
  - angra_mainyu 3y ago
    Why cmake and not meson?
    - dagmx 3y ago
      Covered in my statement above but to expand a bit: meson lags behind for supporting new features like different languages (Swift, ObjC etc), Xcode/msvc updates, IDE integration (CLion is amazing), build outputs like frameworks. basically, Mesons better ergonomics lose out to practicality for my use cases. To get it do what I need, I’d be making my own meta system around it.
      - nocombination 3y ago
        I recommend taking a look at Autotools—if anything for historical context. It quickly becomes apparent that many features in CMake (and likely meson—no experience there myself)—are derived from some functionality in Autotools. It also helps one appreciate the package maintainers role in the software development lifecycle, as many of the GNU coding standards have influenced software packaging/distribution—be it RPM, Deb, Pacman, etc. While there is certainly value in using CMake (as so many projects have chosen it as their build system)—it also becomes a sort of cautionary tale whence projects try to reinvent the wheel to fix some deficiency, only to build a complex system with it's own set of deficiencies. Obviously, part of this is due to the fact that commercial entities (cough Microsoft) deviated from any sort of standards-based approach to building software. But for every build-configuration system, Autotools seems to be the only one which dictates that a developer machine merely have POSIX tools installed. Nowadays, that's all the big players, considering that MSYS2 on Windows is better than ever. And it supports many languages to boot: C/C++, Objective C/C++, Go, Fortran, Erlang.

by dagmx 3y ago

adrian_b 3y ago

Discovering the dependencies depends on the compilers, not on the platform. I have stopped using MSVC many years ago, so I do not know how it handles dependencies, but with gcc or clang that works regardless of platform, with compiler-specific options. Moreover, software for one platform can be built on another platform. What matters is only the platform used for building, where the compilers are hosted, not the target platform for the built software. In general, I extract all platform specific definitions, like the names of the compilers and their command-line options in platform-specific Makefiles, which include a generic Makefile with platform-independent definitions, rules and make targets. Any platform-specific details, like SDK paths, are encapsulated in the definitions of certain "make" variables. While I have not needed to update the platform-independent generic Makefile for decades, the platform-dependent sections need updates from time to time, but that is not done for every software project, but perhaps once per year or more seldom, when significant new compiler versions, standard libraries or other new software tools become available. I have not seen yet any reason for using Ninja. Perhaps it may be faster when building something like chrome, but because chrome is likely to be the slowest-compiling software project known to mankind, it is hard to tell how much is gained by Ninja. For software projects of more typical sizes I have not seen noticeable advantages of Ninja. Traditional make can keep all the cores of a CPU 100% busy, so there is no way any other building system can be appreciably faster. There are some very bad Makefiles that are intrinsically slow, but that is a problem of those Makefiles, not of "make" in general. Using a minimum project-specific Makefile section can only simplify debugging, not making it difficult. I have never seen any evidence for the claim that cmake scales better. On the contrary, at least in all the open-source software projects that I have ever seen cmake is a major source of bugs in the building process that nobody knows how to fix. (i.e. after some software updates cmake fails to find files that exist, presumably due to some errors in the cmakelists that appear to be arcane enough that they are hard to find) I have never seen in projects that use traditional make such errors that are so frequent in all the projects that use cmake. I have a lot of experience in building software projects, because for many decades, with the exception of a few professional programs without alternatives, I have been using only programs that I compile from sources. Most projects have very bad Makefiles, which are very hard to maintain, i.e. to change when files are added, deleted, moved or renamed, but at least they build the software projects reliably. Whenever cmake is used, building a new version is always an adventure with unpredictable outcome.

dagmx 3y ago
I should rephrase, by dependencies I mean third party dependencies. That is on the build system not the compiler. And make tends to fall significantly behind Ninja in my experience for cold builds. Here’s a post from someone else that echoes my experience https://david.rothlis.net/ninja-benchmark/ https://david.rothlis.net/ninja-benchmark/ but for a sufficiently complex project even a well tuned make build is about 20% slower on CI, which is about 10-20m per build. It also seems like you’re giving make the benefit of the doubt when it comes to “most being badly written but my own is great”, while simultaneously deriding cmake without giving it the same benefit. But to your larger point, and going back to my C vs other languages comparison (let’s pick C++ for arguments sake). Yes Make can do it all, but even by your own statements it requires doing everything correctly and there’s significant room for error. CMake makes the trade off of abstraction for a more consistent experience with minimal work. This is similar to C and C++, where yes C can (1) do much better in the very very specific right hands, but (2) it can also become an unwieldy mess. C++ with RAII may seem perplexing from the lens of only accepting the former and not the latter. It lets people spend their time elsewhere on the system.
- t43562 3y ago
  Ninja excludes a lot of features of make which are expensive to implement and cause it to be slower to parse so naturally in certain situations ninja runs faster than make. If your build time is actually affected by how long it takes to read the makefiles then this is important (e.g. building Android).
  - adrian_b 3y ago
    When the makefiles are well organized, all of them have just a few lines, with a few definitions, perhaps at most ten lines, but usually less, and they include a single bigger common Makefile, with most of the definitions, all the rules and all the make targets. So reading all the makefiles should take a negligible time, even in a big project. I have seen too many projects with huge makefiles and with much more makefiles than necessary, but all of those are examples of misuse of make, which are easier solved by an efficient use of make, instead of by replacing make with another tool. For instance, there are many projects with a Makefile in each directory with source files, which is a very bad choice that multiplies the number of makefiles. Makefiles should exist only in a dedicated build directory, not in source directories, except for very small projects with a handful of source files. The best way IMO, is to have a single very small Makefile for each final file that must be built, e.g. executable file or library, in a dedicated subdirectory of the build directory. This allows for a maximum simplification of the Makefiles and their number and size is independent on the number of source directories and on the number of source files.

nocombination 3y ago

Funny to mention cmake when GNU Autotools handles all of these dependency resolution with grace using only a few lines of autoconf. CMake script syntax is not very great, and debugging is no better than sprinkling printf's throughout... So, why not use Autotools? /bin/sh has been the norm for most build processes, to the point where the later-designed YaML syntax is _effectively_ the SAME as a Makefile with /bin/sh statements running the pipeline procedures. CMake came along and tried to fix complexity in Autotools while adding kitchen-sink baggage along the way.

dagmx 3y ago
I’m not specifically advocating for CMake , though it is my preference for many reasons like faster uptake of multiple platform and compiler features. I was merely pushing back on the person perplexing on why people don’t just use Make.
- angra_mainyu 3y ago
  Why cmake and not meson?
  - dagmx 3y ago
    Covered in my statement above but to expand a bit: meson lags behind for supporting new features like different languages (Swift, ObjC etc), Xcode/msvc updates, IDE integration (CLion is amazing), build outputs like frameworks. basically, Mesons better ergonomics lose out to practicality for my use cases. To get it do what I need, I’d be making my own meta system around it.
    - nocombination 3y ago
      I recommend taking a look at Autotools—if anything for historical context. It quickly becomes apparent that many features in CMake (and likely meson—no experience there myself)—are derived from some functionality in Autotools. It also helps one appreciate the package maintainers role in the software development lifecycle, as many of the GNU coding standards have influenced software packaging/distribution—be it RPM, Deb, Pacman, etc. While there is certainly value in using CMake (as so many projects have chosen it as their build system)—it also becomes a sort of cautionary tale whence projects try to reinvent the wheel to fix some deficiency, only to build a complex system with it's own set of deficiencies. Obviously, part of this is due to the fact that commercial entities (cough Microsoft) deviated from any sort of standards-based approach to building software. But for every build-configuration system, Autotools seems to be the only one which dictates that a developer machine merely have POSIX tools installed. Nowadays, that's all the big players, considering that MSYS2 on Windows is better than ever. And it supports many languages to boot: C/C++, Objective C/C++, Go, Fortran, Erlang.