3 ms·

using make -j $num is another way to go to get parallelism that is sometimes convenient.

by harry8 8y ago

flukus 8y ago
Just make sure $num is set, otherwise it tries to do everything it can at once. I found out the hard way compiling gcc last week, system was unresponsive so I reset, but I'm curious if it would have worked eventually.
- btschaegg 8y ago
  This is a pet peeve I have had for a while now. I still have a definition for MAKEFLAGS (with -j [# of cpus-1]) in my profile and rc files and had to learn that if you allow make to start too many things at once, the whole machine would hang. Even switching to a different TTY took ages. To me, it sounds like a tremendous scheduling issue if a build job can drown a whole system like that. Shouldn't there be some emphasis on non-waiting threads/processes that haven't done much over those who have hogged the CPU in the past without me having to manually adjust the niceness of every build task?
  - __blockcipher__ 8y ago
    I can’t offer specifics, but it’s probably at the point that you’d compile a different scheduler?
- jcelerier 8y ago
  IIRC make -j is equivalent to make -j256...
  - benchaney 8y ago
    From the man page: > If the -j option is given without an argument, make will not limit the number of jobs that can run simultaneously. Although other implementations might be different of course.
    - JdeBP 8y ago
      It's not a notion that I carried into my implementation of redo, for example. % redo --jobs redo: ERROR: jobs: missing option argument %
      - flukus 8y ago
        What made you go with that rather than trying to pick a sensible default like the number of cores available? Or was it just a general goal to require explicitness?
        JdeBP 8y ago
        It was simplicity, of design, implementation, use, and indeed explanation. There's only one default, the default if the --jobs option is not used; rather than there being two defaults, one for not having the option and another for having the option but not the option argument. More generally, I try to avoid optional arguments to command options, in line with the guideline in the Single Unix Specification. (I actually picked such ideas up many years ago, before the first POSIX specification, from a book by Eric Foxley titled Unix for Super Users, where there was an appendix on command line option syntax.) * http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap12.html#tag_12_02 http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_... Of course, if the user wants to use the number of cores available, then xe can work that number out and pass it as an argument; and indeed I have done that very thing in some of the package/make scripts that I have published.

using make -j $num is another way to go to get parallelism that is sometimes convenient.

by harry8 8y ago

using make -j $num is another way to go to get parallelism that is sometimes convenient.

Just make sure $num is set, otherwise it tries to do everything it can at once. I found out the hard way compiling gcc last week, system was unresponsive so I reset, but I'm curious if it would have worked eventually.

btschaegg 8y ago
This is a pet peeve I have had for a while now. I still have a definition for MAKEFLAGS (with -j [# of cpus-1]) in my profile and rc files and had to learn that if you allow make to start too many things at once, the whole machine would hang. Even switching to a different TTY took ages. To me, it sounds like a tremendous scheduling issue if a build job can drown a whole system like that. Shouldn't there be some emphasis on non-waiting threads/processes that haven't done much over those who have hogged the CPU in the past without me having to manually adjust the niceness of every build task?
- __blockcipher__ 8y ago
  I can’t offer specifics, but it’s probably at the point that you’d compile a different scheduler?
jcelerier 8y ago
IIRC make -j is equivalent to make -j256...
- benchaney 8y ago
  From the man page: > If the -j option is given without an argument, make will not limit the number of jobs that can run simultaneously. Although other implementations might be different of course.
  - JdeBP 8y ago
    It's not a notion that I carried into my implementation of redo, for example. % redo --jobs redo: ERROR: jobs: missing option argument %
    - flukus 8y ago
      What made you go with that rather than trying to pick a sensible default like the number of cores available? Or was it just a general goal to require explicitness?
      - JdeBP 8y ago
        It was simplicity, of design, implementation, use, and indeed explanation. There's only one default, the default if the --jobs option is not used; rather than there being two defaults, one for not having the option and another for having the option but not the option argument. More generally, I try to avoid optional arguments to command options, in line with the guideline in the Single Unix Specification. (I actually picked such ideas up many years ago, before the first POSIX specification, from a book by Eric Foxley titled Unix for Super Users, where there was an appendix on command line option syntax.) * http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap12.html#tag_12_02 http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_... Of course, if the user wants to use the number of cores available, then xe can work that number out and pass it as an argument; and indeed I have done that very thing in some of the package/make scripts that I have published.