5 ms·

Tracing Python

104 points by jimyl 3y ago

sys_64738 3y ago
A more traditional debugger for Python. https://github.com/inducer/pudb https://github.com/inducer/pudb
Spivak 3y ago
Fantastic article and something I've been looking for better tools for. None of them seem to have my ideal workflow though -- providing an external manifest of what and how to trace that can be used against an existing program without changing the code. Something like Otel's auto instrumentation.
peterkelly 3y ago
Python has a built-in API for tracing: https://docs.python.org/3/library/sys.html#sys.settrace https://docs.python.org/3/library/sys.html#sys.settrace Here's an example of how to use it (obviously a minimal case, you can customise your handler do do whatever you need): def mytrace(frame, event, arg): if event == "call": print("call", frame.f_code.co_name, frame.f_locals) elif event == "return": print("return", frame.f_code.co_name, arg) return mytrace import sys sys.settrace(mytrace) For example, given the following code: def three(z): return 3 def two(y): return three("BBB") - 1 def one(): return two("AAA") - 1 one() It will output: call one {} call two {'y': 'AAA'} call three {'z': 'BBB'} return three 3 return two 2 return one 1
- taeric 3y ago
  I confess I'm surprised the builtin wasn't one of the demoed options. Curious what the reason is.
  - isotopp 3y ago
    I just pushed an addendum, and it is being rendered right now.
    - taeric 3y ago
      Kudos on making the update! Makes sense that you might not find it if you didn't know it was there. Python's "batteries included" means most things you may want to do, you should check to see if there is a standard lib option, as well. Sometimes it is barebones enough that you may want more, of course. So, really nice to see all of the options you presented.
- yi_xuan 3y ago
  Pdb also employs this approach for debug. I had wondered about pdb's implementation, when I saw it, I felt a little surprised.
- RJIb8RBYxzAMX9u 3y ago
  Wow, I can't believe I didn't know about this until now! I'm the lucky 10k[0] today. [0] https://xkcd.com/1053/ https://xkcd.com/1053/
lathiat 3y ago
There are many different tools, but when I want to quickly understand what the code was doing I have been reaching for pysnooper: https://pypi.org/project/PySnooper/ https://pypi.org/project/PySnooper/ Has a pretty nice and importantly colored output by default, which I find much easier to follow. In many cases snooping one specific function has done the job. It has an easy to use decorator to decorate any function without having to otherwise intercept the main python invocation or set something up at the start. Good for libraries etc. pysnooper has mostly been enough for me but the "snoop" mentioned in the article seems to be a superset and more. Will have to give it a go next time. Outside of this if I want to understand performance or more roughly what code paths are executing (and not a per-statement trace) then austin and py-spy are both amazing.. convert to flamescope and drop into flamescope.app.
nomel 3y ago
What's the modern practical use case for tracing in python? Environments without an IDE/remote debugger? I can't recall the last time I needed to enable tracing, in the last couple of decades, besides bash scripts.
- isotopp 3y ago
  Originally the question was "What is Honeycomb.io and Opentracing doing and how can I get this into my existing application?" on IRC. On explaining, the discussion went into various directions (Fred Fish C Debugging Macros and tracing Python).
knlb2022 3y ago
I'll throw in my own too: https://github.com/kunalb/panopticon https://github.com/kunalb/panopticon
chrisshroba 3y ago
Fyi, the table of contents gets cut off (its box is smaller than its contents) at zoom levels of 90% or larger for me. Here's a screenshot at 100% zoom https://imgur.com/a/0t9cKs5 https://imgur.com/a/0t9cKs5 Great article!

104 points by jimyl 3y ago

A more traditional debugger for Python. https://github.com/inducer/pudb https://github.com/inducer/pudb

Fantastic article and something I've been looking for better tools for. None of them seem to have my ideal workflow though -- providing an external manifest of what and how to trace that can be used against an existing program without changing the code. Something like Otel's auto instrumentation.

peterkelly 3y ago

Python has a built-in API for tracing: https://docs.python.org/3/library/sys.html#sys.settrace https://docs.python.org/3/library/sys.html#sys.settrace Here's an example of how to use it (obviously a minimal case, you can customise your handler do do whatever you need): def mytrace(frame, event, arg): if event == "call": print("call", frame.f_code.co_name, frame.f_locals) elif event == "return": print("return", frame.f_code.co_name, arg) return mytrace import sys sys.settrace(mytrace) For example, given the following code: def three(z): return 3 def two(y): return three("BBB") - 1 def one(): return two("AAA") - 1 one() It will output: call one {} call two {'y': 'AAA'} call three {'z': 'BBB'} return three 3 return two 2 return one 1

taeric 3y ago
I confess I'm surprised the builtin wasn't one of the demoed options. Curious what the reason is.
- isotopp 3y ago
  I just pushed an addendum, and it is being rendered right now.
  - taeric 3y ago
    Kudos on making the update! Makes sense that you might not find it if you didn't know it was there. Python's "batteries included" means most things you may want to do, you should check to see if there is a standard lib option, as well. Sometimes it is barebones enough that you may want more, of course. So, really nice to see all of the options you presented.
yi_xuan 3y ago
Pdb also employs this approach for debug. I had wondered about pdb's implementation, when I saw it, I felt a little surprised.
RJIb8RBYxzAMX9u 3y ago
Wow, I can't believe I didn't know about this until now! I'm the lucky 10k[0] today. [0] https://xkcd.com/1053/ https://xkcd.com/1053/

lathiat 3y ago

There are many different tools, but when I want to quickly understand what the code was doing I have been reaching for pysnooper: https://pypi.org/project/PySnooper/ https://pypi.org/project/PySnooper/ Has a pretty nice and importantly colored output by default, which I find much easier to follow. In many cases snooping one specific function has done the job. It has an easy to use decorator to decorate any function without having to otherwise intercept the main python invocation or set something up at the start. Good for libraries etc. pysnooper has mostly been enough for me but the "snoop" mentioned in the article seems to be a superset and more. Will have to give it a go next time. Outside of this if I want to understand performance or more roughly what code paths are executing (and not a per-statement trace) then austin and py-spy are both amazing.. convert to flamescope and drop into flamescope.app.

nomel 3y ago

What's the modern practical use case for tracing in python? Environments without an IDE/remote debugger? I can't recall the last time I needed to enable tracing, in the last couple of decades, besides bash scripts.

isotopp 3y ago
Originally the question was "What is Honeycomb.io and Opentracing doing and how can I get this into my existing application?" on IRC. On explaining, the discussion went into various directions (Fred Fish C Debugging Macros and tracing Python).

knlb2022 3y ago

I'll throw in my own too: https://github.com/kunalb/panopticon https://github.com/kunalb/panopticon

chrisshroba 3y ago

Fyi, the table of contents gets cut off (its box is smaller than its contents) at zoom levels of 90% or larger for me. Here's a screenshot at 100% zoom https://imgur.com/a/0t9cKs5 https://imgur.com/a/0t9cKs5 Great article!