5 ms·

A syscall zombie is pretty easy to find, I often run into them when making NFS mistakes (like leaving the local net without properly ending some dep)... is ther

A syscall zombie is pretty easy to find, I often run into them when making NFS mistakes (like leaving the local net without properly ending some dep)... is there a general way to see what syscall the undead process is waiting for? Is there a simple way to make it return ERRx? Can I get a list of pending syscalls by age?

TheDong 8y ago
It's very easy to find the current stack of running syscalls and how long they take using perf [0]. In fact, a flame graph will give you a wonderful way to visualize this [1]. I could go on and on about perf, but there's no reason since Brendan Gregg's page is so darned good. Just read that, play around with the perf cli, find the answers to all your questions... Well, all of them except how to cause that syscall to error out. There's no simple way to force an in-progress syscall to error out as far as I know. If any one else knows of a simple way, I'd be curious to hear of it. [0]: http://www.brendangregg.com/perf.html http://www.brendangregg.com/perf.html [1]: http://www.brendangregg.com/perf.html#FlameGraphs http://www.brendangregg.com/perf.html#FlameGraphs
- jakeogh 8y ago
  Very cool. I'm trying to start with: perf stat -e 'syscalls' but haven't figured out how to specify all events... if that's even the right route. The examples that are close dont work for me: https://bpaste.net/raw/455ebe2dd400 https://bpaste.net/raw/455ebe2dd400
cyphar 8y ago
If you have the process that is currently blocked (note that "zombie" doesn't apply here -- zombies are processes that are dead but are waiting for someone to collect their exit code) you can look in /proc/$pid/stack to see what the current kernel stack is (the bottom-most entries tell you what syscall the process started by doing). This is the easiest way to start debugging a kernel problem (what is the function it's blocked in -- then go read that function and see whether the the pathology is obvious).

TheDong 8y ago

It's very easy to find the current stack of running syscalls and how long they take using perf [0]. In fact, a flame graph will give you a wonderful way to visualize this [1]. I could go on and on about perf, but there's no reason since Brendan Gregg's page is so darned good. Just read that, play around with the perf cli, find the answers to all your questions... Well, all of them except how to cause that syscall to error out. There's no simple way to force an in-progress syscall to error out as far as I know. If any one else knows of a simple way, I'd be curious to hear of it. [0]: http://www.brendangregg.com/perf.html http://www.brendangregg.com/perf.html [1]: http://www.brendangregg.com/perf.html#FlameGraphs http://www.brendangregg.com/perf.html#FlameGraphs

jakeogh 8y ago
Very cool. I'm trying to start with: perf stat -e 'syscalls' but haven't figured out how to specify all events... if that's even the right route. The examples that are close dont work for me: https://bpaste.net/raw/455ebe2dd400 https://bpaste.net/raw/455ebe2dd400

cyphar 8y ago

If you have the process that is currently blocked (note that "zombie" doesn't apply here -- zombies are processes that are dead but are waiting for someone to collect their exit code) you can look in /proc/$pid/stack to see what the current kernel stack is (the bottom-most entries tell you what syscall the process started by doing). This is the easiest way to start debugging a kernel problem (what is the function it's blocked in -- then go read that function and see whether the the pathology is obvious).