4 ms·
I have a machine with two quad-core CPUs, I want to run 8 parallel jobs at a time! Most of these make replacements start with design decisions (like using Pyth
by sorbits 17y ago
I have a machine with two quad-core CPUs, I want to run 8 parallel jobs at a time!
Most of these make replacements start with design decisions (like using Python) that pretty much rule out ever reaching what make does best, and which is essential to get my build times improved by at least a factor of 6.
Using a tool (like CMake) to generate your Makefiles is an excellent solution, heck, even a Python script that generated a Makefile and then ran make would be to prefer over this tool.
- vr 17y agoUsing Python does not prevent parallel execution of tasks. It's just the build tasks graph that gets created by a single process, but then the build system can use all the available CPUs to actually execute it. See e.g. -j flag to SCons http://www.scons.org/doc/production/HTML/scons-user.html#AEN2099 http://www.scons.org/doc/production/HTML/scons-user.html#AEN....
- sorbits 17y agoAs I said, generating a Makefile (i.e. the dependency graph) from Python would be fine, which conceptually sounds like what SCons do (from your comment). But look at the build file example on the fabricate page, it is a user supplied build file with a build function which calls a compile function which loops over sources. Having the fabricate system take input like that and do parallel execution is as hard as solving the halting problem.
- benhoyt 17y agoFair call. So far that hasn't been a problem for us -- but if you want to make it one, please donate us some cores! :-) Seriously, as vr said, Python's not stopping us. And I imagine it wouldn't be too hard to add code to spawn off a few background tasks based on the dependency list.