5 ms·

What I don't understand about Python is why it's not printing the str() of the variables in the corresponding lines of the traceback. About 95% of all debugging

by erikb 12y ago

What I don't understand about Python is why it's not printing the str() of the variables in the corresponding lines of the traceback. About 95% of all debugging starts with adding prints or logging messages before the lines mentioned in the traceback. Or am I doing something wrong here (since 5 years...)?

VMG 12y ago
The string representation (repr() would be a better fit actually) and the amount of local variables could be enormous.
andreasvc 12y ago
The str or repr of a variable might result in megabytes of output (or worse, calling it might have a side effect). Still, it'd be nice to at least show the first 100 bytes or so.
Walkman 12y ago
If you __str__ is broken, the traceback would fail which would be awful. Once I wrote a __str__ method on a class which got into an infinite loop :) When I showed the traceback to an advanced developer, he told me immediately what was happening. How could we figure out the problem if my __str__ was broken and traceback would fail? Also recently I struggled with py.test which automatically does that (calling str() on every local variable in the test case) but it is broken, so py.test showed me strange Unicode Decode exceptions (it turned out, nothing wrong was in my code but py.test) and I had no idea what was going on. Took me hours to figure it out py.test has a issues with __future__ unicode_literals... tl;dr: not a good idea
- raldi 12y ago
  Doesn't seem like a showstopper to me; just fall back to current behavior if str() fails or takes too long, and truncate if it's too verbose.
  - Walkman 12y ago
    How you define "too long" and "too verbose"?
    - raldi 12y ago
      Keep a running tally of time spent stringifying. If it passes 1000 msec over the entire life of the program, disable the feature. As for length, maybe 100 chars? I think that's around what repr() often uses.
      - Walkman 12y ago
        > Simple is better than complex. I think Python exceptions are verbose enough, not necessary to make it more verbose, more complicated.
      - pmahoney 12y ago
        Can str() call into a C extension that then loops forever?
        moe 12y ago
        Can we contrive esoteric situations where this feature would fail spectacularly? Yes. Do they matter in reality? No. Make it an optional feature to be toggled with an interpreter flag. People like me can turn it on, you can keep it turned off.
        army 12y ago
        That doesn't seem that esoteric to me: in general the problem is that repr() can result in execution of arbitrary code, and that there's no way to cleanly and reliably terminate arbitrary code. This includes Python code: it's possible to terminate it at an arbitrary point, but that leaves things in an indeterminate state. I think it's pretty important that you shouldn't add features with unpredictable behaviour into the core of the language, particularly into error handling code.
mlader 12y ago
I typically will throw in a debugger instead of a print statement if I'm debugging locally. import ipdb; ipdb.set_trace() Then I'll explore the context variables and figure out what the hell is going on.
- est 12y ago
  I think 99.9999% experienced python devs do this. Why can't PSF make it default? At least some kind of commandline switch to break to last traceback context by default. e.g. python --debug my_code.py
  - the_real_bto 12y ago
    they did: python -m pdb my_code.py
- parfe 12y ago
  ipdb just made my day. Thanks.
aurelianito 12y ago
For one, I think repr is more fit. And if repr (or str) has a bug, now you are chasing two issues! Besides, repr (or str) may take a lot of processing time and it is not unusual to log an exception stack trace but keep running. At last, if you are going to debug, a lot of times is better to use a debugger instead of putting print (or log) statements all around. I prefer pydev for this. You can even attach to a python process running as a different user or even on a different computer. I documented how to do it in a post on my blog (sorry, in Spanish). http://aurelianito.blogspot.com.ar/2013/05/debugueando-en-root-con-pydev.html http://aurelianito.blogspot.com.ar/2013/05/debugueando-en-ro...
- StavrosK 12y ago
  If you prefer the console, pudb is an absolutely fantastic curses debugger. I love it.
masklinn 12y ago
cgitb is your friend.
- erikb 12y ago
  That one looks nice!
the_real_bto 12y ago
This is a great idea, although repr() would probably be better than str(). I've spent a lot of time debugging Python over the years. This honestly never occured to me until you mentioned it. Edit: Everyone else jumped in before me. I think it would be useful as a command line switch to python to turn on the behavior (maybe call it extended exceptions). Python would need to run the extended exception generation under a try/except that fell back to regular exceptions if an exception was raised while calling repr()
babs474 12y ago
How about automatically giving you a repl context at the exception and letting you move up and down the call stack. http://werkzeug.pocoo.org/docs/debug/#using-the-debugger http://werkzeug.pocoo.org/docs/debug/#using-the-debugger

by erikb 12y ago

VMG 12y ago

The string representation (repr() would be a better fit actually) and the amount of local variables could be enormous.

andreasvc 12y ago

The str or repr of a variable might result in megabytes of output (or worse, calling it might have a side effect). Still, it'd be nice to at least show the first 100 bytes or so.

Walkman 12y ago

If you __str__ is broken, the traceback would fail which would be awful. Once I wrote a __str__ method on a class which got into an infinite loop :) When I showed the traceback to an advanced developer, he told me immediately what was happening. How could we figure out the problem if my __str__ was broken and traceback would fail? Also recently I struggled with py.test which automatically does that (calling str() on every local variable in the test case) but it is broken, so py.test showed me strange Unicode Decode exceptions (it turned out, nothing wrong was in my code but py.test) and I had no idea what was going on. Took me hours to figure it out py.test has a issues with __future__ unicode_literals... tl;dr: not a good idea

raldi 12y ago
Doesn't seem like a showstopper to me; just fall back to current behavior if str() fails or takes too long, and truncate if it's too verbose.
- Walkman 12y ago
  How you define "too long" and "too verbose"?
  - raldi 12y ago
    Keep a running tally of time spent stringifying. If it passes 1000 msec over the entire life of the program, disable the feature. As for length, maybe 100 chars? I think that's around what repr() often uses.
    - Walkman 12y ago
      > Simple is better than complex. I think Python exceptions are verbose enough, not necessary to make it more verbose, more complicated.
    - pmahoney 12y ago
      Can str() call into a C extension that then loops forever?
      - moe 12y ago
        Can we contrive esoteric situations where this feature would fail spectacularly? Yes. Do they matter in reality? No. Make it an optional feature to be toggled with an interpreter flag. People like me can turn it on, you can keep it turned off.
        army 12y ago
        That doesn't seem that esoteric to me: in general the problem is that repr() can result in execution of arbitrary code, and that there's no way to cleanly and reliably terminate arbitrary code. This includes Python code: it's possible to terminate it at an arbitrary point, but that leaves things in an indeterminate state. I think it's pretty important that you shouldn't add features with unpredictable behaviour into the core of the language, particularly into error handling code.

mlader 12y ago

I typically will throw in a debugger instead of a print statement if I'm debugging locally. import ipdb; ipdb.set_trace() Then I'll explore the context variables and figure out what the hell is going on.

est 12y ago
I think 99.9999% experienced python devs do this. Why can't PSF make it default? At least some kind of commandline switch to break to last traceback context by default. e.g. python --debug my_code.py
- the_real_bto 12y ago
  they did: python -m pdb my_code.py
parfe 12y ago
ipdb just made my day. Thanks.

aurelianito 12y ago

For one, I think repr is more fit. And if repr (or str) has a bug, now you are chasing two issues! Besides, repr (or str) may take a lot of processing time and it is not unusual to log an exception stack trace but keep running. At last, if you are going to debug, a lot of times is better to use a debugger instead of putting print (or log) statements all around. I prefer pydev for this. You can even attach to a python process running as a different user or even on a different computer. I documented how to do it in a post on my blog (sorry, in Spanish). http://aurelianito.blogspot.com.ar/2013/05/debugueando-en-root-con-pydev.html http://aurelianito.blogspot.com.ar/2013/05/debugueando-en-ro...

StavrosK 12y ago
If you prefer the console, pudb is an absolutely fantastic curses debugger. I love it.

masklinn 12y ago

cgitb is your friend.

erikb 12y ago
That one looks nice!

the_real_bto 12y ago

This is a great idea, although repr() would probably be better than str(). I've spent a lot of time debugging Python over the years. This honestly never occured to me until you mentioned it. Edit: Everyone else jumped in before me. I think it would be useful as a command line switch to python to turn on the behavior (maybe call it extended exceptions). Python would need to run the extended exception generation under a try/except that fell back to regular exceptions if an exception was raised while calling repr()

babs474 12y ago

How about automatically giving you a repl context at the exception and letting you move up and down the call stack. http://werkzeug.pocoo.org/docs/debug/#using-the-debugger http://werkzeug.pocoo.org/docs/debug/#using-the-debugger