5 ms·

Does ninja fit your needs? It’s available on just about all of the Linux distros and it’s extremely fast with very few bells and whistles. The language is (for

Does ninja fit your needs? It’s available on just about all of the Linux distros and it’s extremely fast with very few bells and whistles. The language is (for better or worse) designed to be generated by a higher-level tool, so it strips out most of the complexity of GNU make, but it might go too far if you’re looking to do list/map processing in it.

hedora 4y ago
People always claim ninja is fast, but I can't figure out what they mean by that. A typical C++ project build uses 10-10,000 CPU core minutes, and make takes (maybe, in some pessimal situation) 100 milliseconds to schedule and coordinate the build invocations. Even if ninja is 100x faster, it really, really doesn't matter, at all.
- nine_k 4y ago
  If you build something much smaller, a difference betwen 5 sec and 0.5 sec is pretty noticeable for interactive work, even though 5 sec is not prohibitively long at all.
- drothlis 4y ago
  They probably mean incremental builds, where the actual compilation doesn't overshadow the "coordination" work. In my (artificial) benchmark, make scaled poorly, taking 70 seconds to process 100k C files worth of dependencies, vs. ninja's 1.5 seconds: https://david.rothlis.net/ninja-benchmark/ https://david.rothlis.net/ninja-benchmark/ Most of make's time was spent processing the ".d" files containing header dependencies (Ninja has a special optimisation for these files, where it reads them the first time they're created, inserts the dependency information into a binary database, then deletes them so it doesn't have to parse them in future invocations). In real world projects, you often end up "abusing" make to add behaviour such as detecting if the compilation flags have changed, and this can make your makefiles slower; whereas ninja has those features built in. Apparently this made a big difference in build times for Chromium (where ninja was born). See this comment by the ninja's author: https://news.ycombinator.com/item?id=23182469 https://news.ycombinator.com/item?id=23182469
- stabbles 4y ago
  If you have a slow filesystem and you’re compiling C, make has a “lot” of overhead due to implicit rules. There are many more rules than you write, unless you set stuff like .SUFFIXES: to nothing
- aseipp 4y ago
  Ninja isn't necessarily faster in the slow path of "rebuilding the whole project" vs make, but it's often significantly faster in the fast path of "incremental rebuild given a small change in the input code" vs make. Which is what you're doing most of the time. You also do not have to abuse ninja for it to record certain changes; for example tracking CFLAGS as a dependency in Make can be awkward (e.g. write it to a file and all the associated overhead from the filesystem), but in Ninja it's "just" a variable binding, and the usage of variables in commands is tracked as a dependency, and so changing that variable and re-computing the needed set of commands to run is much, much faster. Those things add up in large builds. I have personally had Ninja turn multi-second long no-op rebuild times (e.g. run 'make' with no changes) into the 10s of milliseconds range. The no-op build is often the most extreme case but closest to the average case, which is "recompile after a small edit." The difference in interactivity is quite large in these scenarios. If your project builds in under like 5 minutes from scratch on a modern laptop it probably is not large enough to see huge benefits (outside of pathological cases), but probably some benefit; but for larger projects the difference can be very pronounced, very quickly.
- saghm 4y ago
  One possible culprit is that ninja parallelizes builds by default, whereas make requires you to explicitly specify the number of jobs with a flag, although I've run into issues with this occasionally if the project happens to do manual parallelization (e.g. with the `parallel` tool). I've seen people not realize this or just happen to forget the flag often enough that I wouldn't be that surprised if a decent number of those claims stem from this.
  - gpderetta 4y ago
    Ninja also allows different level of parallelism for different stages which is useful if the process itself is already parallelized internally or you need to limit it for other reasons (ld consuming huge amount of memory is a typical usecase)

hedora 4y ago

People always claim ninja is fast, but I can't figure out what they mean by that. A typical C++ project build uses 10-10,000 CPU core minutes, and make takes (maybe, in some pessimal situation) 100 milliseconds to schedule and coordinate the build invocations. Even if ninja is 100x faster, it really, really doesn't matter, at all.

nine_k 4y ago
If you build something much smaller, a difference betwen 5 sec and 0.5 sec is pretty noticeable for interactive work, even though 5 sec is not prohibitively long at all.
drothlis 4y ago
They probably mean incremental builds, where the actual compilation doesn't overshadow the "coordination" work. In my (artificial) benchmark, make scaled poorly, taking 70 seconds to process 100k C files worth of dependencies, vs. ninja's 1.5 seconds: https://david.rothlis.net/ninja-benchmark/ https://david.rothlis.net/ninja-benchmark/ Most of make's time was spent processing the ".d" files containing header dependencies (Ninja has a special optimisation for these files, where it reads them the first time they're created, inserts the dependency information into a binary database, then deletes them so it doesn't have to parse them in future invocations). In real world projects, you often end up "abusing" make to add behaviour such as detecting if the compilation flags have changed, and this can make your makefiles slower; whereas ninja has those features built in. Apparently this made a big difference in build times for Chromium (where ninja was born). See this comment by the ninja's author: https://news.ycombinator.com/item?id=23182469 https://news.ycombinator.com/item?id=23182469
stabbles 4y ago
If you have a slow filesystem and you’re compiling C, make has a “lot” of overhead due to implicit rules. There are many more rules than you write, unless you set stuff like .SUFFIXES: to nothing
aseipp 4y ago
Ninja isn't necessarily faster in the slow path of "rebuilding the whole project" vs make, but it's often significantly faster in the fast path of "incremental rebuild given a small change in the input code" vs make. Which is what you're doing most of the time. You also do not have to abuse ninja for it to record certain changes; for example tracking CFLAGS as a dependency in Make can be awkward (e.g. write it to a file and all the associated overhead from the filesystem), but in Ninja it's "just" a variable binding, and the usage of variables in commands is tracked as a dependency, and so changing that variable and re-computing the needed set of commands to run is much, much faster. Those things add up in large builds. I have personally had Ninja turn multi-second long no-op rebuild times (e.g. run 'make' with no changes) into the 10s of milliseconds range. The no-op build is often the most extreme case but closest to the average case, which is "recompile after a small edit." The difference in interactivity is quite large in these scenarios. If your project builds in under like 5 minutes from scratch on a modern laptop it probably is not large enough to see huge benefits (outside of pathological cases), but probably some benefit; but for larger projects the difference can be very pronounced, very quickly.
saghm 4y ago
One possible culprit is that ninja parallelizes builds by default, whereas make requires you to explicitly specify the number of jobs with a flag, although I've run into issues with this occasionally if the project happens to do manual parallelization (e.g. with the `parallel` tool). I've seen people not realize this or just happen to forget the flag often enough that I wouldn't be that surprised if a decent number of those claims stem from this.
- gpderetta 4y ago
  Ninja also allows different level of parallelism for different stages which is useful if the process itself is already parallelized internally or you need to limit it for other reasons (ld consuming huge amount of memory is a typical usecase)