3 ms·

Yeah, meaningful benchmarks take a lot of work. - Does the reported ninja time include the cmake step that generates the ninja files? (which typically doesn't

Yeah, meaningful benchmarks take a lot of work. - Does the reported ninja time include the cmake step that generates the ninja files? (which typically doesn't need to be done for incremental builds). - "make -j" doesn't limit the number of jobs, so you can run into resource contention if building too many things at once. Ninja (by default) limits the parallel jobs to the number of CPUs. - Disabling make's built-in rules (with `-r -R`) can make a big difference to performance. - Make spends a lot of time processing the ".d" files with header dependency information -- 98% of the time in my (artificial) benchmarks!

jmiskovic 4y ago
This wasn't a "meaningful benchmark", just a single data point to show how tup makes practical sense, when everyone else was criticizing "yet another build system" on theoretical points. I included the time to generate ninja files, the measured time is for project to build from scratch. Tup generates its initial files _much_ faster (~0.2 seconds compared to ~4 seconds). The strength of tup is actually in incremental builds, but that's harder to measure: you get shorter times with larger relative variations between repeated tries. I wasn't interested in working around the bad make defaults, I also didn't spend any time optimizing tup flags.
kazinator 4y ago
There shouldn't be any .d files in a from-scratch clean build. They get generated during compilation and are not needed initially because every object file is out of date due to its nonexistence. However, they do add a lot of bulk once made and increase with the size of the project. A lot of the information is redundant. Among the files you may find many pairs which only differ in the principal file, the headers being the same: foo.o: foo.c $SOME_HEADERS bar.o: bar.c $SOME_HEADERS In theory, if we have GNU Make, we should be able to compress these into static pattern rules. Concretely, in the above case, the pair condenses to the static pattern rule: foo.o bar.o: %.o: %.c $SOME_HEADERS When, we check whether, say, bar.o is out of date, we find that it as a target in the left side of the rule. Then in the middle, the name matches the %.o target pattern, identifying the stem as "bar", and from there the full dependency rule is instantiated with all the prerequisites. To generalize that, we partition the $(OBJECTS) into subsets based on which patterns they belong to. Hopefully there are a lot fewer subsets than files. Then for each subset we do # The a.h b.h subset $(OBJS_0): %.o: %.c a.h b.h # The a.h d.h subset $(OBJS_1): %.o: %.c a.h d.h and so on. The only problem is managing these subsets (in particular, incrementally) in such a way that more time isn't spent doing that than time saved loading the rules. Stale dependencies can be troublesome when header files are renamed or deleted; this sort of scheme could make it worse. One way to handle these subsets incrementally would be to keep them in some separate database outside of the Makefile. Whenever a file is compiled and the raw .d is generated, we run a script which looks up the header signature and matches it to a subset. Then it assigns that file to the correct subset (perhaps deleting it from an existing one). A step at the start of the build would regenerate the dependency file seen by make if the database indicates that it's out of date.
- drothlis 4y ago
  Ninja stores the header dependencies in a database and then deletes the “.d” file. As far as I know it doesn’t do your deduplication – presumably because it hasn’t been necessary; reading and parsing hundreds of “.d” files was the slow part.

jmiskovic 4y ago

This wasn't a "meaningful benchmark", just a single data point to show how tup makes practical sense, when everyone else was criticizing "yet another build system" on theoretical points. I included the time to generate ninja files, the measured time is for project to build from scratch. Tup generates its initial files _much_ faster (~0.2 seconds compared to ~4 seconds). The strength of tup is actually in incremental builds, but that's harder to measure: you get shorter times with larger relative variations between repeated tries. I wasn't interested in working around the bad make defaults, I also didn't spend any time optimizing tup flags.

kazinator 4y ago

There shouldn't be any .d files in a from-scratch clean build. They get generated during compilation and are not needed initially because every object file is out of date due to its nonexistence. However, they do add a lot of bulk once made and increase with the size of the project. A lot of the information is redundant. Among the files you may find many pairs which only differ in the principal file, the headers being the same: foo.o: foo.c $SOME_HEADERS bar.o: bar.c $SOME_HEADERS In theory, if we have GNU Make, we should be able to compress these into static pattern rules. Concretely, in the above case, the pair condenses to the static pattern rule: foo.o bar.o: %.o: %.c $SOME_HEADERS When, we check whether, say, bar.o is out of date, we find that it as a target in the left side of the rule. Then in the middle, the name matches the %.o target pattern, identifying the stem as "bar", and from there the full dependency rule is instantiated with all the prerequisites. To generalize that, we partition the $(OBJECTS) into subsets based on which patterns they belong to. Hopefully there are a lot fewer subsets than files. Then for each subset we do # The a.h b.h subset $(OBJS_0): %.o: %.c a.h b.h # The a.h d.h subset $(OBJS_1): %.o: %.c a.h d.h and so on. The only problem is managing these subsets (in particular, incrementally) in such a way that more time isn't spent doing that than time saved loading the rules. Stale dependencies can be troublesome when header files are renamed or deleted; this sort of scheme could make it worse. One way to handle these subsets incrementally would be to keep them in some separate database outside of the Makefile. Whenever a file is compiled and the raw .d is generated, we run a script which looks up the header signature and matches it to a subset. Then it assigns that file to the correct subset (perhaps deleting it from an existing one). A step at the start of the build would regenerate the dependency file seen by make if the database indicates that it's out of date.

drothlis 4y ago
Ninja stores the header dependencies in a database and then deletes the “.d” file. As far as I know it doesn’t do your deduplication – presumably because it hasn’t been necessary; reading and parsing hundreds of “.d” files was the slow part.