5 ms·

Is there a tool that would allow collecting historical data on memory and CPU usage patterns of individual processes? In troubleshooting you are frequently deal

by stiff 13y ago

Is there a tool that would allow collecting historical data on memory and CPU usage patterns of individual processes? In troubleshooting you are frequently dealing with the situation that some process is "exploding" in memory or/and CPU usage and either you are not there at the moment to run htop or you might not even be able to easily log in on the server to do checks.

Erwin 13y ago
"atop" can do it to some degree. When you run atop interactively, it uses the process accounting facility to find out not what is running exactly when it takes a snapshot of the system, but also what processes started and exited since the last refresh interval. Installing atop will also (depending on your distro etc.) set it up to snapshot the system state every 600 seconds. If you run "atop -r" you can review that legacy old data from today or an older day, and switch between the 10-minute snapshot with t and T. Personally i like "sar" for quick text only overview (sysstat package). Once enabled you have a 10 minute snapshot of a huge amount of performance metrics (e.g. sar -r for memory, sar -b for disk). Of course, it's even better if you use something to collect them centrally (I signed up for DataDog which takes very little effort to integrate compared to rolling your own stuff).
- stiff 13y ago
  pidstat from the sysstat package seems to be excellent for what I was looking to do. Thanks a lot, both tools are keepers!
- rodgerd 13y ago
  I was really surprised not to see atop in his list. Very handy interactive monitor, and the history mode, as you not, puts it well ahead.
blueblob 13y ago
An answer somewhat stolen from stackoverflow says "ps -o rss $(pgrep executablename)" but I guess that assumes that you only have one process running, maybe it would be easier to put it in a script and use "ps -o rss $!" http://stackoverflow.com/questions/7278326/how-to-monitor-a-processes-memory-usage-over-time-ubuntu http://stackoverflow.com/questions/7278326/how-to-monitor-a-...
kyrra 13y ago
Real simple way: Cron job to dump "top" to a file. It will tell you all processes and their memory/CPU usage every x minutes. Once you need data on a specific process, you can just grep its pid.
snewman 13y ago
Scalyr [1] can do this. (Disclaimer: I am the founder of Scalyr, and it's a commercial product.) We aim to be a one-stop-shopping monitoring tool: collect everything you might want to collect, and let you analyze it in any way you want. To your question, we can collect CPU, memory, I/O, and other stats for specified processes [2], and give you graphs, rolled-up dashboards, and alerts on that data. We're always looking for feedback, and we're happy to give out discounted or free accounts to startups. Drop me a line -- steve@[company domain] -- if you're interested. [1]: https://www.scalyr.com https://www.scalyr.com [2]: https://www.scalyr.com/appDashboard https://www.scalyr.com/appDashboard
- rburhum 13y ago
  You need more images on your site...
  - benjarrell 13y ago
    I agree, I want to see screenshots.
    - gk1 13y ago
      Thanks for the feedback. We're hearing this often so we're including plenty of screenshots in our new design, which is currently in the works.
jlgaddis 13y ago
$ man sar

by stiff 13y ago

Erwin 13y ago

"atop" can do it to some degree. When you run atop interactively, it uses the process accounting facility to find out not what is running exactly when it takes a snapshot of the system, but also what processes started and exited since the last refresh interval. Installing atop will also (depending on your distro etc.) set it up to snapshot the system state every 600 seconds. If you run "atop -r" you can review that legacy old data from today or an older day, and switch between the 10-minute snapshot with t and T. Personally i like "sar" for quick text only overview (sysstat package). Once enabled you have a 10 minute snapshot of a huge amount of performance metrics (e.g. sar -r for memory, sar -b for disk). Of course, it's even better if you use something to collect them centrally (I signed up for DataDog which takes very little effort to integrate compared to rolling your own stuff).

stiff 13y ago
pidstat from the sysstat package seems to be excellent for what I was looking to do. Thanks a lot, both tools are keepers!
rodgerd 13y ago
I was really surprised not to see atop in his list. Very handy interactive monitor, and the history mode, as you not, puts it well ahead.

blueblob 13y ago

An answer somewhat stolen from stackoverflow says "ps -o rss $(pgrep executablename)" but I guess that assumes that you only have one process running, maybe it would be easier to put it in a script and use "ps -o rss $!" http://stackoverflow.com/questions/7278326/how-to-monitor-a-processes-memory-usage-over-time-ubuntu http://stackoverflow.com/questions/7278326/how-to-monitor-a-...

kyrra 13y ago

Real simple way: Cron job to dump "top" to a file. It will tell you all processes and their memory/CPU usage every x minutes. Once you need data on a specific process, you can just grep its pid.

snewman 13y ago

Scalyr [1] can do this. (Disclaimer: I am the founder of Scalyr, and it's a commercial product.) We aim to be a one-stop-shopping monitoring tool: collect everything you might want to collect, and let you analyze it in any way you want. To your question, we can collect CPU, memory, I/O, and other stats for specified processes [2], and give you graphs, rolled-up dashboards, and alerts on that data. We're always looking for feedback, and we're happy to give out discounted or free accounts to startups. Drop me a line -- steve@[company domain] -- if you're interested. [1]: https://www.scalyr.com https://www.scalyr.com [2]: https://www.scalyr.com/appDashboard https://www.scalyr.com/appDashboard

rburhum 13y ago
You need more images on your site...
- benjarrell 13y ago
  I agree, I want to see screenshots.
  - gk1 13y ago
    Thanks for the feedback. We're hearing this often so we're including plenty of screenshots in our new design, which is currently in the works.

jlgaddis 13y ago

$ man sar