6 ms·

This is a thoughtful hack but the real solution here is a) make debuggers easier to set up and b) make your project easily debuggable with a debugger from the b

This is a thoughtful hack but the real solution here is a) make debuggers easier to set up and b) make your project easily debuggable with a debugger from the beginning. Like, debugging should be considered part of programming, and a local dev environment that can’t be debugged should be viewed to be as broken as a codebase without e.g. a way to run the server in watch mode. Also someone should make a debugger that supports the equivalent of print statements, e.g. set print breakpoint on a variable to print its value every time it’s run, instead of typing print everywhere.

josteink 7y ago
Yeah. Python is dead easy to debug via pdb. If you don’t want to go CLI-debugging both VSCode and Emacs have more integrated, in-editor options. Many bugs can be isolated and reproduced in unit-tests. That’s just a click away from being debuggable inside a real debugger. Why use anything but that? To me, not using a debugger to debug seems kinda crazy.
- JHer 7y ago
  Debugging python processes running in containers unfortunately requires a bit of setup in VS Code (no idea about Emacs).
strunz 7y ago
I love using PDB, but even still this has a different use case. Say you have a bug in a function and it depends on timing or it's a race condition, or it has a lot of statements. Being able to see the state throughout the function in your output may be able to catch things or speed up debugging.
- BenFrantzDale 7y ago
  Agreed. My go-to for debugging is `import epdb;epdb.st()` which is a quick and dirty breakpoint. There may be more elegant solutions but that has served me very well.
- askvictor 7y ago
  True, though pysnooper (and print statements for that matter) will change the execution speed, so won't necessarily catch timing bugs (or introduce new ones).
OrgNet 7y ago
With python being a language that tries to only have one way of doing each thing, maybe they should have figured that out at the beginning stages of the language's design phase.
cknd 7y ago
One situation where I would have wanted a real debugger and couldn't use one was in crash reporting on remote systems (like, machine learning code running in some container somewhere, or on a headless box far from the network). Since Python's built-in tracebacks are pretty minimal, the default crash logs don't offer much help other than a line number. I ended up writing a tool that prints tracebacks along with code context and local variables [i], sort of a souped up version of the built-in crash message. It's surprising how much that already helped in a few situations, makes me wonder why it isn't the default. So, yes, more debuggers, but also abundant logging everywhere! [i] https://github.com/cknd/stackprinter https://github.com/cknd/stackprinter
- Gibbon1 7y ago
  Old school neckbeard way a) Crashes dump the program state to a file and then you load that in the debugger. b) If the process is still running but broken, attach the debugger. c) In my firmware I log bus fault addresses and restart. That allows me to see what code or memory access caused the error. 75% of the time it's fairly obvious what happened.
  - int_19h 7y ago
    Crash dump debugging is tricky to implement Python, because its normal debugging APIs - sys.settrace, frame objects etc - require being inside the process with a running Python interpreter. You can still do it, but you basically have to redo the whole thing from scratch. For example, instead of asking for a repr() of an object to display its value, you have to access the internal representation of it directly - and then accommodate all the differences in that between various Python versions. Something like this (note several different dicts): https://github.com/Microsoft/PTVS/tree/bcdfec4f211488e373fa253583fefe3dbb320c87/Python/Product/Debugger.Concord/Proxies/Structs https://github.com/Microsoft/PTVS/tree/bcdfec4f211488e373fa2...
- yoz 7y ago
  Python's standard lib has had something like this for years, but it's been criminally under-promoted (probably because of the terrible name): https://docs.python.org/3/library/cgitb.html https://docs.python.org/3/library/cgitb.html However, yours provides much nicer output. Thank you!
nitrogen 7y ago
Also someone should make a debugger that supports the equivalent of print statements, e.g. set print breakpoint on a variable to print its value every time it’s run, instead of typing print everywhere. Traditonal debuggers like gdb for compiled languages support this (breakpoint actions, memory write breakpoints, and variable displays). If something similar isn't already in your language's debugger, that might be a source of ideas for adding it.
spc476 7y ago
Until such a day comes, `printf()` debugging will still be useful. Heck, even after that day comes, it may still find uses. The other day I was writing LPEG [1]. I had a rule that wasn't firing for some reason and I wanted to know why LPEG was skipping that part of the input. It was not a simple matter to fire up a debugger and put a breakpoint on the rule: -- look for 1 or more ASCII characters minus the colon local hdr_name = (R"\0\127" - P":")^1 -- [2] I mean, I could put a breakpoint there, but that would trigger when the expression is being compiled, not run. LPEG has its own VM geared for parsing (it really is its own language) so yes, it is sometimes difficult to debug (especially when you end up with a large parsing expression to parse a document---I only found the bug when I was creating another document similar to my initial testing document that was just different enough). Fortunately, there is a way to hook into the parsing done by LPEG and I was able to insert literal print statements during parsing that showed what exactly was going on (my rule was a bit too promiscuous). [1] Lua Parsing Expression Grammar http://www.inf.puc-rio.br/~roberto/lpeg/ http://www.inf.puc-rio.br/~roberto/lpeg/ [2] Yes, a regular expression for that is smaller than what I typed, but with LPEG, I can reuse that rule in other expressions. Also, that wasn't the actual rule used, but the effective rule I was using.
nijave 7y ago
Or just use a logger with proper log levels
int_19h 7y ago
I'm not sure what you mean by "debugger equivalent of print statements". If what you want is a breakpoint that just prints something out when it's hit, then these already exist, e.g.: https://code.visualstudio.com/docs/editor/debugging#_logpoints https://code.visualstudio.com/docs/editor/debugging#_logpoin... But "set breakpoint on a variable" kinda sounds more like a data breakpoint (i.e. break/log when value changes)? These also exist in some places: https://docs.microsoft.com/en-us/visualstudio/debugger/using-breakpoints?view=vs-2019#BKMK_set_a_data_breakpoint_managed https://docs.microsoft.com/en-us/visualstudio/debugger/using... but usually with many limitations. The problem with these is that it's hard to implement it in such a way that there's zero overhead when not debugging. If I remember correctly, native data breakpoints have some hardware support on Intel, but for high-level languages it can be difficult to map their data model to something like that.
- badatshipping 7y ago
  Oh wow, I never knew about that. Thanks.
AtHeartEngineer 7y ago
True...but inheriting code, not having time to refactor, and just getting a basic version out the door...pysnooper is pretty useful.

josteink 7y ago

Yeah. Python is dead easy to debug via pdb. If you don’t want to go CLI-debugging both VSCode and Emacs have more integrated, in-editor options. Many bugs can be isolated and reproduced in unit-tests. That’s just a click away from being debuggable inside a real debugger. Why use anything but that? To me, not using a debugger to debug seems kinda crazy.

JHer 7y ago
Debugging python processes running in containers unfortunately requires a bit of setup in VS Code (no idea about Emacs).

strunz 7y ago

I love using PDB, but even still this has a different use case. Say you have a bug in a function and it depends on timing or it's a race condition, or it has a lot of statements. Being able to see the state throughout the function in your output may be able to catch things or speed up debugging.

BenFrantzDale 7y ago
Agreed. My go-to for debugging is `import epdb;epdb.st()` which is a quick and dirty breakpoint. There may be more elegant solutions but that has served me very well.
askvictor 7y ago
True, though pysnooper (and print statements for that matter) will change the execution speed, so won't necessarily catch timing bugs (or introduce new ones).

OrgNet 7y ago

With python being a language that tries to only have one way of doing each thing, maybe they should have figured that out at the beginning stages of the language's design phase.

cknd 7y ago

One situation where I would have wanted a real debugger and couldn't use one was in crash reporting on remote systems (like, machine learning code running in some container somewhere, or on a headless box far from the network). Since Python's built-in tracebacks are pretty minimal, the default crash logs don't offer much help other than a line number. I ended up writing a tool that prints tracebacks along with code context and local variables [i], sort of a souped up version of the built-in crash message. It's surprising how much that already helped in a few situations, makes me wonder why it isn't the default. So, yes, more debuggers, but also abundant logging everywhere! [i] https://github.com/cknd/stackprinter https://github.com/cknd/stackprinter

Gibbon1 7y ago
Old school neckbeard way a) Crashes dump the program state to a file and then you load that in the debugger. b) If the process is still running but broken, attach the debugger. c) In my firmware I log bus fault addresses and restart. That allows me to see what code or memory access caused the error. 75% of the time it's fairly obvious what happened.
- int_19h 7y ago
  Crash dump debugging is tricky to implement Python, because its normal debugging APIs - sys.settrace, frame objects etc - require being inside the process with a running Python interpreter. You can still do it, but you basically have to redo the whole thing from scratch. For example, instead of asking for a repr() of an object to display its value, you have to access the internal representation of it directly - and then accommodate all the differences in that between various Python versions. Something like this (note several different dicts): https://github.com/Microsoft/PTVS/tree/bcdfec4f211488e373fa253583fefe3dbb320c87/Python/Product/Debugger.Concord/Proxies/Structs https://github.com/Microsoft/PTVS/tree/bcdfec4f211488e373fa2...
yoz 7y ago
Python's standard lib has had something like this for years, but it's been criminally under-promoted (probably because of the terrible name): https://docs.python.org/3/library/cgitb.html https://docs.python.org/3/library/cgitb.html However, yours provides much nicer output. Thank you!

nitrogen 7y ago

Also someone should make a debugger that supports the equivalent of print statements, e.g. set print breakpoint on a variable to print its value every time it’s run, instead of typing print everywhere. Traditonal debuggers like gdb for compiled languages support this (breakpoint actions, memory write breakpoints, and variable displays). If something similar isn't already in your language's debugger, that might be a source of ideas for adding it.

spc476 7y ago

Until such a day comes, `printf()` debugging will still be useful. Heck, even after that day comes, it may still find uses. The other day I was writing LPEG [1]. I had a rule that wasn't firing for some reason and I wanted to know why LPEG was skipping that part of the input. It was not a simple matter to fire up a debugger and put a breakpoint on the rule: -- look for 1 or more ASCII characters minus the colon local hdr_name = (R"\0\127" - P":")^1 -- [2] I mean, I could put a breakpoint there, but that would trigger when the expression is being compiled, not run. LPEG has its own VM geared for parsing (it really is its own language) so yes, it is sometimes difficult to debug (especially when you end up with a large parsing expression to parse a document---I only found the bug when I was creating another document similar to my initial testing document that was just different enough). Fortunately, there is a way to hook into the parsing done by LPEG and I was able to insert literal print statements during parsing that showed what exactly was going on (my rule was a bit too promiscuous). [1] Lua Parsing Expression Grammar http://www.inf.puc-rio.br/~roberto/lpeg/ http://www.inf.puc-rio.br/~roberto/lpeg/ [2] Yes, a regular expression for that is smaller than what I typed, but with LPEG, I can reuse that rule in other expressions. Also, that wasn't the actual rule used, but the effective rule I was using.

nijave 7y ago

Or just use a logger with proper log levels

int_19h 7y ago

I'm not sure what you mean by "debugger equivalent of print statements". If what you want is a breakpoint that just prints something out when it's hit, then these already exist, e.g.: https://code.visualstudio.com/docs/editor/debugging#_logpoints https://code.visualstudio.com/docs/editor/debugging#_logpoin... But "set breakpoint on a variable" kinda sounds more like a data breakpoint (i.e. break/log when value changes)? These also exist in some places: https://docs.microsoft.com/en-us/visualstudio/debugger/using-breakpoints?view=vs-2019#BKMK_set_a_data_breakpoint_managed https://docs.microsoft.com/en-us/visualstudio/debugger/using... but usually with many limitations. The problem with these is that it's hard to implement it in such a way that there's zero overhead when not debugging. If I remember correctly, native data breakpoints have some hardware support on Intel, but for high-level languages it can be difficult to map their data model to something like that.

badatshipping 7y ago
Oh wow, I never knew about that. Thanks.

AtHeartEngineer 7y ago

True...but inheriting code, not having time to refactor, and just getting a basic version out the door...pysnooper is pretty useful.