4 ms·

I work on a project where a huge build system has been developed. It uses the same source to build a dynamic library, the same library in static and many test p

I work on a project where a huge build system has been developed. It uses the same source to build a dynamic library, the same library in static and many test programs. It handles well dependencies whatever the directory structure. It is developed in gnumake. I think you will be less criticized if you use gnumake instead of python. It has all the features needed for dynamic generation of dependency rules. And if someone want to improve the build system, a good knowledge of gnumake seems a reasonable prerequisite. The main defect is that it needs 10s before starting any compilation.

jstimpfle 9y ago
However, a few disadvantages to Make: Only files and timestamps means that the granularity of your project's modules needs to be at least as fine as required by the build (I think that's one reason why C builds are slow: Too many files written). Pattern matches are not enough IMO. The scripting language is bad for advanced things. It's like implementing a large project in shell code. Having the full power of Python really is something else. So I was considering generating simple repetitive Make rules. But I wanted to explore what it takes to do the rest myself, as well. Setting up dependencies for Make is not nice, either. Having the code under control means being able to adapt to any requirement. It would be easy for example to convert this into a build server kind of thing. Having a 10s delay is not acceptable to me. I didn't experience that myself though.
- reacweb 9y ago
  The 10s delay is caused by the correct handling of dependency in a full source tree (around 5000 source files). It is the price of perfectionism. There is no simple repetitive make rules in the build system of the project I am on: all the rules are generated using macros. gnumake is a full scripting language that is dedicated to timestamp dependencies. It can manipulate easily lists of string (as long as they do not contain spaces). What are the advanced things that requires a better language ?
  - jstimpfle 9y ago
    > The 10s delay is caused by the correct handling of dependency in a full source tree (around 5000 source files) There is something awfully wrong if processing 5000 files and cached dependendencies takes 10s. It should take a few ms at most, even if written in a scripting language like Python. I would not be surprised if this is because that build system heavily misuses GNU Make's variable substitution features, instead of being written in a language that features proper data structures (e.g. set, map)
    - AstralStorm 9y ago
      Heck, even cmake does this better then without fancy data structures. My bet is this either does many shell calls or wildcards multiple times through file system.

jstimpfle 9y ago

However, a few disadvantages to Make: Only files and timestamps means that the granularity of your project's modules needs to be at least as fine as required by the build (I think that's one reason why C builds are slow: Too many files written). Pattern matches are not enough IMO. The scripting language is bad for advanced things. It's like implementing a large project in shell code. Having the full power of Python really is something else. So I was considering generating simple repetitive Make rules. But I wanted to explore what it takes to do the rest myself, as well. Setting up dependencies for Make is not nice, either. Having the code under control means being able to adapt to any requirement. It would be easy for example to convert this into a build server kind of thing. Having a 10s delay is not acceptable to me. I didn't experience that myself though.

reacweb 9y ago
The 10s delay is caused by the correct handling of dependency in a full source tree (around 5000 source files). It is the price of perfectionism. There is no simple repetitive make rules in the build system of the project I am on: all the rules are generated using macros. gnumake is a full scripting language that is dedicated to timestamp dependencies. It can manipulate easily lists of string (as long as they do not contain spaces). What are the advanced things that requires a better language ?
- jstimpfle 9y ago
  > The 10s delay is caused by the correct handling of dependency in a full source tree (around 5000 source files) There is something awfully wrong if processing 5000 files and cached dependendencies takes 10s. It should take a few ms at most, even if written in a scripting language like Python. I would not be surprised if this is because that build system heavily misuses GNU Make's variable substitution features, instead of being written in a language that features proper data structures (e.g. set, map)
  - AstralStorm 9y ago
    Heck, even cmake does this better then without fancy data structures. My bet is this either does many shell calls or wildcards multiple times through file system.