7 ms·

Monner - Monitor CPU, memory and network whilst running a program

56 points by colinhowe 14y ago

aw3c2 14y ago
Nice! Would be ace if you could add i/o to it (eg maybe through iotop?).
- colinhowe 14y ago
  I believe prstat (the Python library I'm using) has this. I'll probably add it in the next few hours :)
- colinhowe 14y ago
  Added!
  - rabbitfang 14y ago
    Even more awesome would be to add metrics for inbound and outbound network packets per second! ;)
    - colinhowe 14y ago
      I've made it easier to start adding more metrics... could you pop this one in a github issue? I'll be able to actually remember to do it :)
pyre 14y ago
Why not just fork the subprocess rather than forking the monitoring process?
- colinhowe 14y ago
  The monitor is stopped as soon as the target process finishes - which means that the very last reading emitted will only contain information from during the run of the target. If it were done the other way around then we'd have to poll for the termination of the forked target process. Either the poll duration would have to be really small or the monitor would contain readings for some time after the end of the target. This isn't so bad when running with an interval of 1 second... but for an interval of 1 minute (or longer) it can start to give odd readings. To be honest, I'm thinking of removing the very final reading anyway. Or, at least documenting that the final reading may be slightly misleading.
  - pyre 14y ago
    Ah. I was mucking around in the subprocess module at one point, and IIRC it already forks the subprocess that it launches, even if you're just doing something like subprocess.call().
deleted 14y ago
[deleted]
kenferry 14y ago
How does this compare to Instruments? http://goo.gl/BFWEv http://goo.gl/BFWEv Its advantages are not obvious to me..
- iheartmemcache 14y ago
  It's multiplatform for one.. not everyone uses OS X especially in production
andrewcooke 14y ago
what does "memory" mean here? it would be nice to have more details (eg virtual v physical). also disk io (with paging separate) would be good. and maybe files opened (like /prod/PID/fd/*) it would be really nice to have something that shows all this for a single process (and perhaps also for other processes that are "top" in some category). at the moment, when i am trying to balance code to run as well as possible within given hardware constraints, i have to juggle various monitor programs - this is a good first step to using just one...
sigil 14y ago
Useful idea. I sometimes use time [1] to print stats at the end of the program's run, like so. /usr/bin/time -f 'user=%U system=%S maxrss=%M ins=%I outs=%O" PROGRAM Seeing stats while the program is running is nice though! Also, cumulative monitoring of an entire foreground process group would be super handy (a long build, for instance). Drop the --target-output option, you don't need it. Same thing can be achieved with: monner.py sh -c 'exec 1>/dev/null 2>&1 $0 "$@"' PROGRAM I also agree with the other commenter that monner should be the parent process. [1] Note, time(1) is not the same as the shell builtin of the same name.
- colinhowe 14y ago
  Good point! Changing it now :)
  - nviennot 14y ago
    I personally prefer strace's options: -f -- follow forks, -ff -- with output into separate files -o file -- send trace output to FILE instead of stderr
- colinhowe 14y ago
  Switched around the process architecture. Less code now :) Will look at dropping the target-output option.
repoman 14y ago
htop will solve all your problem. Why even bother Monner?
- colinhowe 14y ago
  htop is nice but it's not so usable for generating output that you can graph up and use for comparing things.
lectrick 14y ago
What is all this methods-starting-with-an-underscore crap? That's a pretty shitty way to namespace methods. /ruby guy
- pearkes 14y ago
  "Use one leading underscore only for non-public methods and instance variables." [1] [1] http://www.python.org/dev/peps/pep-0008/#method-names-and-instance-variables http://www.python.org/dev/peps/pep-0008/#method-names-and-in...
rabbitfang 14y ago
What is the minimum python version required to run this?
xnt14 14y ago
Does this work on child threads?

56 points by colinhowe 14y ago

aw3c2 14y ago

Nice! Would be ace if you could add i/o to it (eg maybe through iotop?).

colinhowe 14y ago
I believe prstat (the Python library I'm using) has this. I'll probably add it in the next few hours :)
colinhowe 14y ago
Added!
- rabbitfang 14y ago
  Even more awesome would be to add metrics for inbound and outbound network packets per second! ;)
  - colinhowe 14y ago
    I've made it easier to start adding more metrics... could you pop this one in a github issue? I'll be able to actually remember to do it :)

pyre 14y ago

Why not just fork the subprocess rather than forking the monitoring process?

colinhowe 14y ago
The monitor is stopped as soon as the target process finishes - which means that the very last reading emitted will only contain information from during the run of the target. If it were done the other way around then we'd have to poll for the termination of the forked target process. Either the poll duration would have to be really small or the monitor would contain readings for some time after the end of the target. This isn't so bad when running with an interval of 1 second... but for an interval of 1 minute (or longer) it can start to give odd readings. To be honest, I'm thinking of removing the very final reading anyway. Or, at least documenting that the final reading may be slightly misleading.
- pyre 14y ago
  Ah. I was mucking around in the subprocess module at one point, and IIRC it already forks the subprocess that it launches, even if you're just doing something like subprocess.call().

deleted 14y ago

[deleted]

kenferry 14y ago

How does this compare to Instruments? http://goo.gl/BFWEv http://goo.gl/BFWEv Its advantages are not obvious to me..

iheartmemcache 14y ago
It's multiplatform for one.. not everyone uses OS X especially in production

andrewcooke 14y ago

what does "memory" mean here? it would be nice to have more details (eg virtual v physical). also disk io (with paging separate) would be good. and maybe files opened (like /prod/PID/fd/*) it would be really nice to have something that shows all this for a single process (and perhaps also for other processes that are "top" in some category). at the moment, when i am trying to balance code to run as well as possible within given hardware constraints, i have to juggle various monitor programs - this is a good first step to using just one...

sigil 14y ago

Useful idea. I sometimes use time [1] to print stats at the end of the program's run, like so. /usr/bin/time -f 'user=%U system=%S maxrss=%M ins=%I outs=%O" PROGRAM Seeing stats while the program is running is nice though! Also, cumulative monitoring of an entire foreground process group would be super handy (a long build, for instance). Drop the --target-output option, you don't need it. Same thing can be achieved with: monner.py sh -c 'exec 1>/dev/null 2>&1 $0 "$@"' PROGRAM I also agree with the other commenter that monner should be the parent process. [1] Note, time(1) is not the same as the shell builtin of the same name.

colinhowe 14y ago
Good point! Changing it now :)
- nviennot 14y ago
  I personally prefer strace's options: -f -- follow forks, -ff -- with output into separate files -o file -- send trace output to FILE instead of stderr
colinhowe 14y ago
Switched around the process architecture. Less code now :) Will look at dropping the target-output option.

repoman 14y ago

htop will solve all your problem. Why even bother Monner?

colinhowe 14y ago
htop is nice but it's not so usable for generating output that you can graph up and use for comparing things.

lectrick 14y ago

What is all this methods-starting-with-an-underscore crap? That's a pretty shitty way to namespace methods. /ruby guy

pearkes 14y ago
"Use one leading underscore only for non-public methods and instance variables." [1] [1] http://www.python.org/dev/peps/pep-0008/#method-names-and-instance-variables http://www.python.org/dev/peps/pep-0008/#method-names-and-in...

rabbitfang 14y ago

What is the minimum python version required to run this?

xnt14 14y ago

Does this work on child threads?