3 ms·

> To help track that down, I override Python's os.open and os.close and display the pathname and fd so I have a history of who is using what fd's. Nice! I'd re

by bhawks 4y ago

> To help track that down, I override Python's os.open and os.close and display the pathname and fd so I have a history of who is using what fd's. Nice! I'd recommend trying strace to track system calls in the future though. Monkey patching open and close will only catch code using those functions and not any c libraries or very smart people using ctypes. Strace captures every syscall and dumps the parameters and returns.

mst 4y ago
I've used both depending on the situation - I am a huge fan of strace (especially 'strace -f') and have done some truly stupid shit with it. Best example being dealing with a situation where a client's code could only have one connection to a particular external streaming API they were using and the developer in charge of connecting to it was a tad territorial and wouldn't provide me a way to access the data that actually worked for the task I'd been assigned. Solution: strace his process with a -s argument that was larger than any read his process ever did, then backprocess it into the original bytes on the wire, then handle them myself. Yes, this was a horrible hack, but it allowed me to prove the concept of the thing the client wanted building without causing massive political drama that would've been more trouble for everybody involved than it was worth. Ideal for that sort of situation would (for me) likely be trapping the open and close functions and having them emit information to stderr while -also- having strace log to the same stderr so I could see how the two compare - with the caveat that whether it's viable to do that is highly variable depending on context. But as a last comment on strace, I present an old entry from my quotefile: <@mst> actually, I think my first thing to try would be to strace the code <@mst> and try and match up the new value of $! with a failed syscall <@mst> but I mean if strace was a person I'd totally be asking them out to dinner so maybe I'm biased
Too 4y ago
Newer python versions come with what they call audit hooks, which can log events on for example file open. The open function already has such audit built in. https://peps.python.org/pep-0578/ https://peps.python.org/pep-0578/ It can be seen as somewhere in between home made monkey patching and strace. Strace is great, because it shows absolutely everything. But showing everything can also be too much, a pre filtered view is often what you really want. If you want a similar tracing mechanism that spans your whole system I think you could do it with eBPF.

by bhawks 4y ago

mst 4y ago

I've used both depending on the situation - I am a huge fan of strace (especially 'strace -f') and have done some truly stupid shit with it. Best example being dealing with a situation where a client's code could only have one connection to a particular external streaming API they were using and the developer in charge of connecting to it was a tad territorial and wouldn't provide me a way to access the data that actually worked for the task I'd been assigned. Solution: strace his process with a -s argument that was larger than any read his process ever did, then backprocess it into the original bytes on the wire, then handle them myself. Yes, this was a horrible hack, but it allowed me to prove the concept of the thing the client wanted building without causing massive political drama that would've been more trouble for everybody involved than it was worth. Ideal for that sort of situation would (for me) likely be trapping the open and close functions and having them emit information to stderr while -also- having strace log to the same stderr so I could see how the two compare - with the caveat that whether it's viable to do that is highly variable depending on context. But as a last comment on strace, I present an old entry from my quotefile: <@mst> actually, I think my first thing to try would be to strace the code <@mst> and try and match up the new value of $! with a failed syscall <@mst> but I mean if strace was a person I'd totally be asking them out to dinner so maybe I'm biased

Too 4y ago

Newer python versions come with what they call audit hooks, which can log events on for example file open. The open function already has such audit built in. https://peps.python.org/pep-0578/ https://peps.python.org/pep-0578/ It can be seen as somewhere in between home made monkey patching and strace. Strace is great, because it shows absolutely everything. But showing everything can also be too much, a pre filtered view is often what you really want. If you want a similar tracing mechanism that spans your whole system I think you could do it with eBPF.