14 ms·

This is a bit of a weird article. It spends most of the time talking about logging, which is somewhat useful for debugging but not really. pdb gets a few lines

This is a bit of a weird article. It spends most of the time talking about logging, which is somewhat useful for debugging but not really. pdb gets a few lines of description, and that is about it. Personally, I can't live without Pycharm when working with Python purely because of how fantastic the debugging experience is. The integration with the interactive IPython shell is simply fantastic, and the live variable viewer is also really handy. Between being able to prototype code in the interactive IPython REPL and then graduate it seamlessly to scripts/functions/classes, and then having an amazing debugging UX to fallback on for those bugs that slip through, I find myself to be orders of magnitude more efficient than when coding in something like vscode (which I absolutely love, and want to transition to for Python coding, but it is extremely limited in comparison to Pycharm, especially when it comes to the REPL experience + debugging).

vonseel 6y ago
Yeah I agree entirely on Pychaem debugger being awesome. But it’s no substitute for logging in production. Somethings just need to be logged, either for proof or those situations where you just can’t reasonably attach a Pycharm debugger.
- underdeserver 6y ago
  Yes, logging is crucial for knowing what happened, but if you don't understand how it could have happened, step by step debugging is immensely useful. Assuming, of course, you can reproduce in your dev environment.
- jerzyt 6y ago
  Production logging with accurate and precise timestamp is an absolute time saver. That means like 1ms timestamp and NNTP synchronized servers.
- _coveredInBees 6y ago
  Oh yes, absolutely. I didn't mean to say logging isn't useful. It is vital to have good logging for production because like you said, you aren't just going to be able to attach a debugger. But for an article titled "Guide to Python Debugging", logging should be a short paragraph, not the core of the article imo. Debugging in production on the other hand... then logging makes a lot of sense to focus on.
0x008 6y ago
I tried pycharm once and every time i started it up it had to index files for hours. Anybody got around that problem? I really like IntelliJ so I would love to give pycharm a try.
- eadan 6y ago
  You want to make sure PyCharm is only indexing files within your project. Depending on your setup, it can try to index all files within virtual environments etc.
- BerislavLopac 6y ago
  Everything you can do with PyCharm you can do with IntelliJ with the Python plugin; in fact, the plugin and PyCharm share the underlying code base.
jedberg 6y ago
> Between being able to prototype code in the interactive IPython REPL and then graduate it seamlessly to scripts/functions/classes, and then having an amazing debugging UX to fallback on for those bugs that slip through, Do you have a good tutorial on how this all ties together? It sounds like heaven but it's not how I use PyCharm because I don't know any better.
- _coveredInBees 6y ago
  No link to any tutorial, it's just how I've always coded and found myself to be the most productive and produce the least bugs. A brief description is: 1. Open up the Python interpreter (highly recommend installing iPython in your project venv so you can use it as the default interpreter in Pycharm) 2. Setup toy data to prototype the thing you are trying to do (or just load in some data or a pickled file, etc.) 3. Start prototyping the functionality of your code one step at a time. If things need to be in a loop in your main script: eg: `for thing in things:` , I just set `thing = things[0]` in the REPL and then continue prototyping the logic of what goes in the inner loop. 4. Use the variable viewer, printing variables in the REPL, etc to make sure things are doing what you want 5. As sequences of statements are verified as doing what you want, you just "graduate" them to your script so you in effect know that they are somewhat tested and do what you wanted 6. Get to a point where you get the right result you were looking for 7. Now set `thing = things[4]` or some other edge-case and just run it a few times to make sure you still get the right result. 8. You're good to go. Wrap up the code in whatever function or class and you are ready to give it a shot for real This sounds like more work, but it is super easy to execute code in the interpreter that you have in your editor (select code in script window, ALT + Shift + E to execute in interpreter) Now the thing people might say is - Well, it is hard to always setup stuff with toy data, and that's true. In that case, you just set a debugger at the point where you would be writing your new function/method, and when you pause there, launch the interactive python interpretter within the debugger and you can start prototyping code from scratch in exactly the same way above, but in the debugger, with actual data you care about. I find that you eliminate all the dumb/idiotic bugs in code this way and the only ones that I need to actually debug at times are certain edge cases I did not account for at the time of prototyping.
jerzyt 6y ago
I agree. After criticizing print statements, more than half of the article discusses logging, which is a little more organized print. And I'm not disagreeing - it's invaluable in production systems. I wish there was more to how to actually debug in Jupyter or Spyder, and I'm relatively new to both, and was hoping for some tips on that.
- perculis 6y ago
  Try pycharm. It allows debugging Jupyter notebooks. Not perfect but better than nothing.
david_draco 6y ago
There is a divide: Some people love debuggers, some people think placing print statements is the best way to debug code. You are reading an article from the second camp. Log files have the benefit of being able to adjust the signal-to-noise afterwards, I guess that is why it is highlighted.
- neves 6y ago
  I was impressed that almost everyone in the book Programmers at Work used print statements to debug. It changed my mind about using it for debugging code. The book: https://www.goodreads.com/book/show/2092682.Programmers_at_Work https://www.goodreads.com/book/show/2092682.Programmers_at_W... I have a newer edition, but I'm not at home now.
- crimsonalucard1 6y ago
  There really is no logical argument. People have their preferences for a bike or a car but the car is faster than the bike just like how the debugger is generally better then printing. There is very little logical argument for the alternative. Taking a snapshot of program state using log is isomorphic to freezing the program at that same state. The difference is that with the debugger you have the option to step into the next state or increase your resolution and look at state within a lower level module/function. With the debugger you have access to ALL state at the point where you froze it, not just statements you happened to log. With logs, the way you debug is you log some state, reset the program with more logs and iterate until you find an issue. With a debugger you freeze the program examine all the state .... advance the program however far you want.... then the issue is found (unless you advanced the state too far ... but with logging the state ALWAYS advances too far until the end of execution, so with logging you always need to reset). Logically, logging is just debugging with less features so people who prefer logging just in general don't want to use the extra features. Additionally logging has the added downside of constantly polluting your pristine code with your log annotations which you have to remove later (can introduce more bugs). While debugging with console PDB has the same effect, a GUI largely alleviates this problem. The only argument for logging/printing is that you can use logging to debug an application in production.
TuringNYC 6y ago
I'm curious - how do you use pdb when debugging, say, controllers on web functions? I guess you can try and debug underlying functions using pdb, but i'm not sure how it happens (short of logging) once you're means of interaction is an HTTP call, or a response (e.g., to a trigger/message)
- tummybug 6y ago
  Run the application locally in a terminal and just use pdb normally. To do it to an application running in a container is slightly more involved but still possible.
abiogenesis 6y ago
I second PyCharm, not only from a debugging perspective but as a whole. I don't understand people struggling with generic text editors, development oriented text tools (sublime etc), or even vscode just because they are free when there is a much better tool money can buy (and PyCharm has a free, community version as well).
- raziel2p 6y ago
  Anecdotal, but I have never seen any of my colleagues with PyCharm work more quickly than me in Sublime Text. I think if you "struggle" in an IDE or editor, it's not really the IDE or editor that's the problem. A "jump to definiton" key and plaintext search tools (I use grep or ag in the command line) tend to be enough for me 99% of the time. I also think there's something to it being forced to think about your code rather than just relying on your IDE. In the past I saw random functions getting extracted to random modules and imported everywhere because PyCharm makes it so easy to do so, with no real thought as to where it belongs, which led to some weird circular import error happening pretty much every month.
  - soVeryTired 6y ago
    I work with data scientists and ML people more than hardcore developers, but I've noticed that a lot of people who use sublime text tend to resort to print() debugging. IMO python's lack of explicit typing makes it difficult to reason about by inspection alone ("does foo() return a dataframe or a numpy array?!"). For me at least, I need to get into the guts of a system and watch it execute to really understand it. The print() debug crowd tend to be much better than me at reasoning about code by inspection alone, but when you work on something complex that you didn't write yourself, that only gets you so far.
    - naniwaduni 6y ago
      Part of it is that the print() debug crowd tends to prefer to externalize more of the reasoning into the code itself (rather than into their tooling, or into their brains—though sometimes it is the latter, and that can be a disaster coming back a few months later). That doesn't work great on other people's code, obviously.
jordic 6y ago
For me pdbpp it's a must, always installed on every venv

vonseel 6y ago

Yeah I agree entirely on Pychaem debugger being awesome. But it’s no substitute for logging in production. Somethings just need to be logged, either for proof or those situations where you just can’t reasonably attach a Pycharm debugger.

underdeserver 6y ago
Yes, logging is crucial for knowing what happened, but if you don't understand how it could have happened, step by step debugging is immensely useful. Assuming, of course, you can reproduce in your dev environment.
jerzyt 6y ago
Production logging with accurate and precise timestamp is an absolute time saver. That means like 1ms timestamp and NNTP synchronized servers.
_coveredInBees 6y ago
Oh yes, absolutely. I didn't mean to say logging isn't useful. It is vital to have good logging for production because like you said, you aren't just going to be able to attach a debugger. But for an article titled "Guide to Python Debugging", logging should be a short paragraph, not the core of the article imo. Debugging in production on the other hand... then logging makes a lot of sense to focus on.

0x008 6y ago

I tried pycharm once and every time i started it up it had to index files for hours. Anybody got around that problem? I really like IntelliJ so I would love to give pycharm a try.

eadan 6y ago
You want to make sure PyCharm is only indexing files within your project. Depending on your setup, it can try to index all files within virtual environments etc.
BerislavLopac 6y ago
Everything you can do with PyCharm you can do with IntelliJ with the Python plugin; in fact, the plugin and PyCharm share the underlying code base.

jedberg 6y ago

> Between being able to prototype code in the interactive IPython REPL and then graduate it seamlessly to scripts/functions/classes, and then having an amazing debugging UX to fallback on for those bugs that slip through, Do you have a good tutorial on how this all ties together? It sounds like heaven but it's not how I use PyCharm because I don't know any better.

_coveredInBees 6y ago
No link to any tutorial, it's just how I've always coded and found myself to be the most productive and produce the least bugs. A brief description is: 1. Open up the Python interpreter (highly recommend installing iPython in your project venv so you can use it as the default interpreter in Pycharm) 2. Setup toy data to prototype the thing you are trying to do (or just load in some data or a pickled file, etc.) 3. Start prototyping the functionality of your code one step at a time. If things need to be in a loop in your main script: eg: `for thing in things:` , I just set `thing = things[0]` in the REPL and then continue prototyping the logic of what goes in the inner loop. 4. Use the variable viewer, printing variables in the REPL, etc to make sure things are doing what you want 5. As sequences of statements are verified as doing what you want, you just "graduate" them to your script so you in effect know that they are somewhat tested and do what you wanted 6. Get to a point where you get the right result you were looking for 7. Now set `thing = things[4]` or some other edge-case and just run it a few times to make sure you still get the right result. 8. You're good to go. Wrap up the code in whatever function or class and you are ready to give it a shot for real This sounds like more work, but it is super easy to execute code in the interpreter that you have in your editor (select code in script window, ALT + Shift + E to execute in interpreter) Now the thing people might say is - Well, it is hard to always setup stuff with toy data, and that's true. In that case, you just set a debugger at the point where you would be writing your new function/method, and when you pause there, launch the interactive python interpretter within the debugger and you can start prototyping code from scratch in exactly the same way above, but in the debugger, with actual data you care about. I find that you eliminate all the dumb/idiotic bugs in code this way and the only ones that I need to actually debug at times are certain edge cases I did not account for at the time of prototyping.

jerzyt 6y ago

I agree. After criticizing print statements, more than half of the article discusses logging, which is a little more organized print. And I'm not disagreeing - it's invaluable in production systems. I wish there was more to how to actually debug in Jupyter or Spyder, and I'm relatively new to both, and was hoping for some tips on that.

perculis 6y ago
Try pycharm. It allows debugging Jupyter notebooks. Not perfect but better than nothing.

david_draco 6y ago

There is a divide: Some people love debuggers, some people think placing print statements is the best way to debug code. You are reading an article from the second camp. Log files have the benefit of being able to adjust the signal-to-noise afterwards, I guess that is why it is highlighted.

neves 6y ago
I was impressed that almost everyone in the book Programmers at Work used print statements to debug. It changed my mind about using it for debugging code. The book: https://www.goodreads.com/book/show/2092682.Programmers_at_Work https://www.goodreads.com/book/show/2092682.Programmers_at_W... I have a newer edition, but I'm not at home now.
crimsonalucard1 6y ago
There really is no logical argument. People have their preferences for a bike or a car but the car is faster than the bike just like how the debugger is generally better then printing. There is very little logical argument for the alternative. Taking a snapshot of program state using log is isomorphic to freezing the program at that same state. The difference is that with the debugger you have the option to step into the next state or increase your resolution and look at state within a lower level module/function. With the debugger you have access to ALL state at the point where you froze it, not just statements you happened to log. With logs, the way you debug is you log some state, reset the program with more logs and iterate until you find an issue. With a debugger you freeze the program examine all the state .... advance the program however far you want.... then the issue is found (unless you advanced the state too far ... but with logging the state ALWAYS advances too far until the end of execution, so with logging you always need to reset). Logically, logging is just debugging with less features so people who prefer logging just in general don't want to use the extra features. Additionally logging has the added downside of constantly polluting your pristine code with your log annotations which you have to remove later (can introduce more bugs). While debugging with console PDB has the same effect, a GUI largely alleviates this problem. The only argument for logging/printing is that you can use logging to debug an application in production.

TuringNYC 6y ago

I'm curious - how do you use pdb when debugging, say, controllers on web functions? I guess you can try and debug underlying functions using pdb, but i'm not sure how it happens (short of logging) once you're means of interaction is an HTTP call, or a response (e.g., to a trigger/message)

tummybug 6y ago
Run the application locally in a terminal and just use pdb normally. To do it to an application running in a container is slightly more involved but still possible.

abiogenesis 6y ago

I second PyCharm, not only from a debugging perspective but as a whole. I don't understand people struggling with generic text editors, development oriented text tools (sublime etc), or even vscode just because they are free when there is a much better tool money can buy (and PyCharm has a free, community version as well).

raziel2p 6y ago
Anecdotal, but I have never seen any of my colleagues with PyCharm work more quickly than me in Sublime Text. I think if you "struggle" in an IDE or editor, it's not really the IDE or editor that's the problem. A "jump to definiton" key and plaintext search tools (I use grep or ag in the command line) tend to be enough for me 99% of the time. I also think there's something to it being forced to think about your code rather than just relying on your IDE. In the past I saw random functions getting extracted to random modules and imported everywhere because PyCharm makes it so easy to do so, with no real thought as to where it belongs, which led to some weird circular import error happening pretty much every month.
- soVeryTired 6y ago
  I work with data scientists and ML people more than hardcore developers, but I've noticed that a lot of people who use sublime text tend to resort to print() debugging. IMO python's lack of explicit typing makes it difficult to reason about by inspection alone ("does foo() return a dataframe or a numpy array?!"). For me at least, I need to get into the guts of a system and watch it execute to really understand it. The print() debug crowd tend to be much better than me at reasoning about code by inspection alone, but when you work on something complex that you didn't write yourself, that only gets you so far.
  - naniwaduni 6y ago
    Part of it is that the print() debug crowd tends to prefer to externalize more of the reasoning into the code itself (rather than into their tooling, or into their brains—though sometimes it is the latter, and that can be a disaster coming back a few months later). That doesn't work great on other people's code, obviously.

jordic 6y ago

For me pdbpp it's a must, always installed on every venv