4 ms·
It would be high time for some capable & easy to use IO instrumentation in Linux. This (vmstat) has been the state of the art for ~ 20 years and you often long
by zurn 13y ago
It would be high time for some capable & easy to use IO instrumentation in Linux. This (vmstat) has been the state of the art for ~ 20 years and you often long for something better. Not to mention it's pretty inaccurate sometimes - reports these huge bursts of IO between periods of idleness even though the disk is munching constantly. Especially with write loads.
My ideal tool would let me switch a knob to turn on io profiling and then would let me drill down into the trace, show what processes caused the io, was the io like (contiguous vs seek heavy, fsyncs, etc). A second need is report what kind of io a process incurred during its runtime, a bit like "time".
I guess much like "perf" now but for io.
For all I know all the kernel pieces are there already. We do have some newer tools like iotop/iostat that are small steps forward. But it's pretty quiet.
- simcop2387 13y agoiotop can show you some of that, but doesn't let you drill down quite so far. It's a step in the right direction though.
- philjr 13y agoAgreed - it's really frustrating that iotop / vmstat are basically still the top tools for debugging. strace can help, but the impact on performance really hurts. We should be taking a lead from dtrace in this space
- thinkersilver 13y agoYou could also use iostat. I monitor my servers' disk i/o using iostat -xNm 1 10. The metrics await* and avgqu* are normally what I am interested in. They give the disk wait timings for serving disk i/o requests. You also have access to the dtrace probes.
- dibbsonline 13y agoWhat about sysstat that provides sar (and iostat). I have this logging every minute in some environments and I can look back and find the root cause of performance issues over multiple devices and networks. kSar can graph this quickly. vmstat is essential in it's own way, the run queue is critical in threaded environments like weblogic with osb/soa where a default install on a 2 vCPU instance won't perform. Also; dtrace for linux and osx.
- mprovost 13y agoIf you're still using sar have a look at atop instead. It's great, it's like top crossed with Time Machine, you can log your system's performance like sar but then look at it like top and rewind until you find the performance problem.
- lobster_johnson 13y ago> show what processes caused the io pidstat (part of Sysstat, "apt-get install sysstat" on Debian/Ubuntu) does this pretty well. # sudo pidstat -d 1 11:21:54 PM PID kB_rd/s kB_wr/s kB_ccwr/s Command 11:21:55 PM 855 0.00 7.92 0.00 kjournald 11:21:55 PM 8134 0.00 79.21 0.00 rsyslogd 11:21:55 PM PID kB_rd/s kB_wr/s kB_ccwr/s Command 11:21:56 PM 829 0.00 28.00 0.00 kjournald 11:21:56 PM 8134 0.00 60.00 0.00 rsyslogd 11:21:56 PM PID kB_rd/s kB_wr/s kB_ccwr/s Command 11:21:57 PM 8134 0.00 104.00 0.00 rsyslogd 11:21:57 PM PID kB_rd/s kB_wr/s kB_ccwr/s Command 11:21:58 PM 8134 0.00 56.00 0.00 rsyslogd Since 2.6.something, the kernel has had per-process I/O counters. Still, it's pretty far from DTrace's iosnoop script (on OS X, do "sudo iosnoop" to see it). Something I dearly miss on Linux, but I don't know what the situation is with SystemTap or DTrace these days.
- ape4 13y agoGnome and KDE have GUI apps that show info.
- otterley 13y agoiotop(8) is pretty useful.