16 ms·
PySnooper: Never use print for debugging again
- operatorequals 7y agoThis looks pretty useful. I love these packages that seem like forgotten stdlib features.
- v3ss0n 7y agothat gonna be huge chunk of log to search
- cool-RR 7y agoNo, because by default it only logs what's happening directly in your function, not what happens deeper in the stack. (i.e. functions that your function calls.) You can set depth=2 or depth=3 and then you'll get a huge chunk of log. The deeper levels will be indented.
- scaryclam 7y agoInteresting project, though I'd suggest losing the line about "can't be bothered to set one up right now" regarding a full debugger. (i)pdb is built in and is simple to use. Perhaps focus on what this can add rather than framing the project as something like a lazy alternative (especially when this may actually be harder to set up than throwing in "import pdb; pdb.set_trace()")? edit: spelling
- cool-RR 7y ago(i)pdb is built in and is simple to use pdb and its variants (my favorite is PuDB) are generally difficult to use in complex, corporate projects. If you've got a multi-process, multi-thread Python project running on a remote host, you'll need a really full-featured debugger to work with it effectively. I recommend Wing IDE or PyCharm for that.
- maxmcd 7y agoThere's this: https://pypi.org/project/rpdb/ https://pypi.org/project/rpdb/ for remote debugging. Celery also ships with its own rdb: http://docs.celeryproject.org/en/v2.3.3/tutorials/debugging.html http://docs.celeryproject.org/en/v2.3.3/tutorials/debugging.... Certainly not a counter, as I'm less familiar with Wing and Pycharm's debugging features, but both of these have been helpful to me in multi-process, multi-thread python environments.
- whitehouse3 7y agoI’d also lose the profanity. > You can use it in your shitty, sprawling enterprise codebase without having to do any setup. Debugging is a sensitive subject, particularly given how frustrating it can be. There’s a place for vulgarity somewhere, but I’d rather see your README provide authoritative info than crack jokes.
- cool-RR 7y agoI wrote a lot of shitty, sprawling enterprise code over the years, and I feel there's no shame in admitting that's what it is :)
- abaldwin7302 7y agoI appreciated the README as written. I immediately understood what kind of environment the author intended this to be used in. A more formal explanation would have degraded the message. Vulgarity doesn't have to taken as judgemental, it can also be a good way to express empathy. The way I read it, the author's intent was the latter.
- 7y ago
- fifnir 7y agoWill it work with multiprocessing?
- cool-RR 7y agoGood question. If the function is launched in a spawned process, it'll work, though you might have trouble getting the stderr, so you better include a log file as the first argument, like this `@snoop('/var/log/snoop.log')` If the function launches new processes internally... I'm not sure. If you try it and it doesn't work, open an issue: https://github.com/cool-RR/PySnooper/issues https://github.com/cool-RR/PySnooper/issues
- sametmax 7y agoThere is a race condition when writting on the same log file from several processes at the same time, which is a typical use case for wsgi frameworks such as django or flask.
- tgbugs 7y agoI have to say sometimes I do find it much easier to read logs than to muck about in an interactive interpreter. That said I did just add `export PYTHONBREAKPOINT=pudb.set_trace` to my bashrc and am slowly going through and removing all my old `from IPython import embed` lines. A much simpler workflow that doesn't incur major runtime costs.
- cool-RR 7y agoPuDB is amazing. Best command-line debugger ever, plus you get feelings of nostalgia for the Borland days.
- StavrosK 7y agoFor those, like me, who don't know about the new breakpoint(): https://hackernoon.com/python-3-7s-new-builtin-breakpoint-a-quick-tour-4f1aebc444c https://hackernoon.com/python-3-7s-new-builtin-breakpoint-a-...
- hguhghuff 7y agoI love this concept and will definitely be using it. Presumably it works with Django? If yes then thrice thanks. This project would justify an additional dedicated screen just for dumping function debug logs.
- cool-RR 7y agoI confirmed it worked with Django before releasing :)
- StavrosK 7y agoThis looks great, good job! It strikes a great balance between PuDB (my favorite, but can't easily run in Docker/remotely) and q (very simple to use but you need too many print() calls everywhere). PySnooper seems great to just drop in and get the full context of what I want to debug. Can it be used as a context manager (`with pysnooper.snoop():`) for even finer targeting?
- lee 7y agoUsing print is sometimes superior to pdb. If you want to quickly see how the program runs without stepping through each line print is justified. This looks like an evolved print. Nice tool!
- collyw 7y agoSometimes but not always, especially on large unfamiliar code bases.
- zerkten 7y agoThis might be useful as a teaching aid. You can run some code and then get the line-by-line breakdown for newbie programmers.
- Quiark 7y agoSimilar principle to my project https://github.com/Quiark/overlog https://github.com/Quiark/overlog which has some more interactive data exploration.
- digitalsushi 7y agoAs a very mediocre ruby programmer, is there an equivalent in ruby to this? If you have a little pattern that is more clever than 'puts' everywhere, please share, it would be well received by at least one person out there. Thanks!
- johnernaut 7y agoYes, check out Pry: https://pryrepl.org/ https://pryrepl.org/
- alexisnorman 7y agoSecond that, pry rules!
- petercooper 7y agoSomething that prints out the state after running each line of code? No, I don't think so, but I believe it wouldn't be too hard to rig up something similar with TracePoint - https://ruby-doc.org/core-2.2.0/TracePoint.html https://ruby-doc.org/core-2.2.0/TracePoint.html - which is a built in Ruby way to trace code execution. For example, dump this into a file and run it: TracePoint.trace(:line) do |tp| STDERR.puts "Ran: #{File.readlines(tp.path)[tp.lineno - 1]}" end a = 10 b = 5 puts a a += b puts a puts b It'll print out each line of code as it runs it. You could then parse each line for identifiers and print them out, etc. It'd be a project, but it's doable.
- raldu 7y agoCheck out byebug: https://github.com/deivid-rodriguez/byebug https://github.com/deivid-rodriguez/byebug
- butuzov 7y agoAgain, Ram - it's a really nice project. Good luck with it. But just in case if you from print cult, you will like https://github.com/gruns/icecream https://github.com/gruns/icecream
- StavrosK 7y agoThis is great, thanks! I've been wanting this from the q library for ages, but the author hasn't responded in a few years. I'll switch to this right away, thanks again!
- lsh 7y ago"Ram"? I went looking for a project called Ram but couldn't find one. Link?
- AtHeartEngineer 7y agoI looked at icecream and I really like pysnooper more. Especially for big complicated programs.
- amelius 7y agoDoes it work in multithreaded code? And is debug output nicely separated based on the thread?
- jcroll 7y agoI come from the php world now coding in python. There used to be a very nice library from Symfony called VarDumper: https://symfony.com/doc/current/components/var_dumper.html https://symfony.com/doc/current/components/var_dumper.html that would just pretty print variables you dumped to the browser in an easy to consume form. Is there anything like this in python?
- brootstrap 7y agoI think there is a difference because python doesnt require a web browser to run. People are just running programs with no web UI or webserver etc... If you really wanted, you could pipe your debug prints to a file and load in a browser if that is what you want :p
- guitarbill 7y agoThere is pretty-print [0], which you could dump inside pre tags, or print to stdout. But honestly, depending on the framework, quicker options exist than (basically) printf debugging. [0] https://docs.python.org/3/library/pprint.html#example https://docs.python.org/3/library/pprint.html#example
- jcroll 7y agoFor Flask what would be the better option?
- guitarbill 7y agoPretty much the stuff already mentioned elsewhere on HN for this article. I often find Werkzeug's excellent debugger is enough: https://news.ycombinator.com/item?id=19718869 https://news.ycombinator.com/item?id=19718869 . pdb/ipdb/pudb et al (pick your fav) can help for really tricky stuff. And sufficient logging, so you know what's going on at all times even without a debugger attached. (occasionally, the low effort of print-debugging works, but if you keep having to print in more/different locations... it's a blunt tool IMO)
- thanatropism 7y ago
- StopHammoTime 7y agoThis is great, I'm definitely going to be using this in my next project.
- dhbradshaw 7y agoWe have some fairly intense functions that needs to be profiled with realistic data and io conditions. I've been doing that by hand, line by line in the shell. I just tried wrapping this around one of these functions in the django shell (actually shell_plus, but same thing). Just imported pysnooper, created a new function using `new_f = pysnooper.snoop()(original_f)` and called new_f on the realistic data and got a nice printout that included values and times. Very useful.
- chaitanya 7y agoIs there an equivalent of Common Lisp's TRACE in the Python world? IMO it is the most value for money (i.e. time and convenience) debugging tool I've used till now. Simply write (TRACE function1 function2 ...) in the REPL and you will get a nicely formatted output of arguments passed and value(s) returned for each invocation of the given functions. Another nice feature is that a deeper an invocation is in the stack the more it is indented -- so recursive functions are fairly easy to debug too. You can't use it for everything but its sufficient most of the time. PySnooper looks good, but it is inconvenient in a couple of ways: 1. It prints every line of the traced function -- most of the time this is overkill and not what one needs. 2. To snoop on a function you need to modify the source file. Not a deal breaker but you still have to remember to revert this.
- yingw787 7y agoI use this snippet: ``` import ipdb ipdb.set_trace() ``` It drops me into IPython, an interactive Python shell, with the interactive Python debugger. You can type variables and use `pdb` primitives like (c)ontinue, (u)p call stack, (n)ext line, etc. I really like it, but ofc YMMV.
- jmuhlich 7y agoTry pdb++ (pdbpp) instead — it's like a much improved ipdb. It monkeypatches itself into pdb rather than defining a new package, so it works from any context that drops into the debugger. Its “sticky mode” alone is worth the switch.
- chaitanya 7y agoYeah but the idea is that, with TRACE, most of the time you shouldn't need to drop into the debugger (all CLs come with an interactive debugger too). If TRACE is not sufficient, only then do I go down the debugger route (or strategic print statements).
- spc476 7y agoI've done something similar (but not exact) for Lua [1]. I was testing code to trace Lua execution and found some code that was executed more than I expected [2]. Adding a trace like you described should be easy as well in Lua. [1] http://boston.conman.org/2015/02/12.1 http://boston.conman.org/2015/02/12.1 [2] http://boston.conman.org/2015/02/13.1 http://boston.conman.org/2015/02/13.1
- tylerwince 7y agoSuper neat project. I am a print debugger myself and will definitely use this at some point in the future. For scenarios where PySnooper might be overkill and you just want to see the value of specific variables, I wrote a port of the Rust `dbg` macro for both Python and Go that are pretty nifty when looking at values really quickly: https://github.com/tylerwince/pydbg https://github.com/tylerwince/pydbg https://github.com/tylerwince/godbg https://github.com/tylerwince/godbg
- scwoodal 7y agoFor my Django projects the development server is werkzeug [1] and anywhere a break point is needed I'll add 1/0 in the code then hit refresh in the browser which pulls up an interactive debugger [2]. [1] https://django-extensions.readthedocs.io/en/latest/runserver_plus.html https://django-extensions.readthedocs.io/en/latest/runserver... [2] https://werkzeug.palletsprojects.com/en/0.15.x/debug/#using-the-debugger https://werkzeug.palletsprojects.com/en/0.15.x/debug/#using-...
- iamgopal 7y agoThis is what I'm going to try to use in every language from now on.
- Xavdidtheshadow 7y agoExcept for JS, which will execute `1/0` happily and without complaint. For that, I usually do `undefined.foo`.
- TazeTSchnitzel 7y agoJS has a debugger keyword!
- CaptainMarvel 7y agoPython has one too now!
- someguy1010 7y agothanks for showing me this! I use the more I learn about django-extensions the more impressed I am with its features. Being able to interact with the django shell with a jupyter notebook is awesome too.
- vraivroo 7y agoThis works in recent versions of Rails as well.
- 7y ago
- gigatexal 7y agooh man, that example code is way too verbose! I like the idea though as I have never really liked the logging in python
- a_c 7y agoBefore clicking I was like: why would one not use pdb. With vim, python-mode, and 'find . -name "*.py"| entr python testcases.py', setting break point and re-running is painless. I was wrong. Upon skimming, it seems a huge plus PySnooper have over pdb is auto inspecting states, sparing whole lot of manual typing
- bkyan 7y agoHave you considered providing an option to pipe the output to a Slack channel or something similar to that?
- cool-RR 7y agoHaha. I don't know if this is satire or not, but in case it's not: You can pass any writable stream as the first argument and PySnooper will use it. So it should be easy to integrate with Slack or anything else.
- mikepurvis 7y agoUnsure if serious, but if you're looking for a crash reporter tool that integrates with Slack, you might be interested in something more like Sentry.
- bkyan 7y agoYes, I was serious. I'm curious why you thought I was joking? cool-RR thought I was joking, as well, so I must be missing something... ¯\_(ツ)_/¯ Thanks for the tip on Sentry -- I'll take a look!
- 7y ago
- freeappdown 7y agocan download the top 20 relate apps to slove the problem from https://www.freeappdown.com https://www.freeappdown.com
- MyBrew 7y agoIs there anything like this for java?
- nexuist 7y agoTo add on: Is there anything like this for Node.js? Would save lots of headaches.
- rossdavidh 7y agoWhile I sometimes use one debug tool or another, I have never understood the aversion to print statements reflected in the title. Sometimes, often even, a print statement is just fine, and anything else is overabstracting it. Not saying other options aren't nice to have available, just that there is nothing wrong with using a simple print statement in many situations.
- craftyguy 7y agoOnce you go down the path, it's easy to start adding more and more, and it's easy to forget about them. They sometimes blend in well with patches you generate, since it's just a 'print()' and nothing obvious like 'import pdb; pdb.set_trace()'
- _jal 7y agoThat's a matter of being organized and competent, not a fault of print(). People create bugs by forgetting things all the time. If you believe their spin, FB snarfed millions of contact lists by accident because of that. I've troubleshot countless things that happened because of code that fell between the cracks. And on the other side of that, I could tell you about the time we went a month without logs in production because someone didn't properly test a library change, so everyone's carefully manicured logging broke. There's nothing wrong with print(), at least that isn't wrong with a lot of other things.
- oblio 7y ago> That's a matter of being organized and competent, not a fault of print(). Or you can be organized and competent at a higher level and just use tools. We're using computers for a reason :)
- maw 7y agoWhen the language I'm using permits it -- that is, when I'm not using Python -- I deliberately misindent them. They're very visible in diffs then. And even if accidentally committed they're still easy to spot and eliminate later.
- makmanalp 7y agoI think this is a very neat tool for educational purposes - sort of like python tutor!
- yason 7y agoHappy to see new tools for debugging. Yet it's the good old print that I most often end up using. Debugging is very context-sensitive: adding prints makes sense because I know exactly what I'm interested in, and I can drill down on that without being disturbed by anything else. I can print low-level stuff or high-level indicators, depending where my debugging takes me. There's no ready-made recipe for that. Some codebases have built-in logging or tracing functionality: you just flick a switch in a module and begin to get ready-made prints from all the interesting parts. But I've found myself never using those: they are not my prints, they do not come from the context I'm in and they don't have a meaning. Use what you want but please don't underestimate prints.
- joshmarlow 7y agoAssuming you use Python, you might like this library - https://github.com/robdmc/behold https://github.com/robdmc/behold It gives you contextual debugging - so you can put fancy prints throughout a function but they are silent unless some context is true. It's useful for when you have a hot path that is executed a lot but you only want debug prints for one of those invocations.
- anitil 7y agoI've definitely had the experience of using custom_logger.log("something"), then checking the system log. Nope, no there, must be in /var/log/app ... nope. Hmm oh I know! I'll turn up the logging in the config file, wherever that is. Still nothing. Did I need to compile in some flag? Screw it. print, run directly from shell, done.
- thatoneuser 7y agoFuck var/log/app.
- lsh 7y agohonestly, fuck the whole standard logging library in Python. It is the most infuriating thing to use. log4j has a lot to answer for
- scriptkiddy 7y agoThis is great! I'm currently working on a large Django project that has itself and all of it's services running in Docker containers via docker-compose. In order to use a traditional debugger, I would need to set up remote debugging and the integration with VS code for that is really not great. Not to mention that getting a remote debugger to work with Django's monkey-patched imports is a little wonky as well. With this package, it seems like I can just get my debugging via stderr.
- _verandaguy 7y agoThis is super neat, and definitely a great tool for early debugging -- but for anything more in-depth, there's built-in pdb and third-party ipdb, which gives PDB an IPython frontend. Both use gdb semantics which is great if that's what you're used to.
- badatshipping 7y agoThis is a thoughtful hack but the real solution here is a) make debuggers easier to set up and b) make your project easily debuggable with a debugger from the beginning. Like, debugging should be considered part of programming, and a local dev environment that can’t be debugged should be viewed to be as broken as a codebase without e.g. a way to run the server in watch mode. Also someone should make a debugger that supports the equivalent of print statements, e.g. set print breakpoint on a variable to print its value every time it’s run, instead of typing print everywhere.
- josteink 7y agoYeah. Python is dead easy to debug via pdb. If you don’t want to go CLI-debugging both VSCode and Emacs have more integrated, in-editor options. Many bugs can be isolated and reproduced in unit-tests. That’s just a click away from being debuggable inside a real debugger. Why use anything but that? To me, not using a debugger to debug seems kinda crazy.
- JHer 7y agoDebugging python processes running in containers unfortunately requires a bit of setup in VS Code (no idea about Emacs).
- strunz 7y agoI love using PDB, but even still this has a different use case. Say you have a bug in a function and it depends on timing or it's a race condition, or it has a lot of statements. Being able to see the state throughout the function in your output may be able to catch things or speed up debugging.
- BenFrantzDale 7y agoAgreed. My go-to for debugging is `import epdb;epdb.st()` which is a quick and dirty breakpoint. There may be more elegant solutions but that has served me very well.
- askvictor 7y ago
- josteink 7y ago> You'd love to use a full-fledged debugger with breakpoints and watches, but you can't be bothered to set one up right now. In VSCode: built in. Press the play icon. In Emacs: M-x realgud:pdb Is that really more effort than this? Why invent inferior solutions to solved problems?
- deleted 7y ago[deleted]
- ben509 7y agoInstallation for a module like this: cd project pip install pysnooper # or pipenv or poetry And it works. That's far superior to poking around in the dark trying to make an IDE see my project correctly. If IDE authors ever figure out how to implement a test button that invokes `python -c "some stuff"` and shows me the results, I'd consider using them.
- josteink 7y ago> If IDE authors ever figure out how to implement a test button that invokes `python -c "some stuff"` You mean like both Emacs and VSCode already does?
- bluejay2387 7y agoDoes this work in Jupyter? That would be a good use case for it. In pretty much everything else you really should be setting up a debugger...
- ToBeBannedSoon 7y agoThis is not a substitute for debug-level logging when you need it.
- tezka 7y agoWhat’s so hard about putting this at the beginning of the function? import pdb; pdb.set_trace()
- jasonhansel 7y agoIMO the real advantage of print() over a full-fledged debugger comes when you're testing. Just replace print() calls with assert(). (Also debuggers, especially on the front end, always seem incredibly laggy and slow.) Using debuggers tends to encourage people to fix problems without writing regression tests. Honestly, I'd prefer "better support for print-line debugging" to "better debugger that you can set up" in most cases.
- zephyrfalcon 7y agoAnother (lightweight) tool for debugging is the q library: https://pypi.org/project/q/ https://pypi.org/project/q/
- vemv 7y agoGithub star count looks fake for a project so recent.
- jdormit 7y agoI mean, it's been on the front page of HN for 8 hours...
- euske 7y agoNever use print for debugging again... If you're creating a reasonably complex project, spend some time to set up a nice and robust logging facility and always use it instead of print. That's the first thing you should do. You will not regret that decision. I sometimes use a debugger to tackle with unfamiliar code, but I always prefer using trace/logging whenever possible, because 1) you can see the context and whole process that reached that point, and 2) the history of debugging can be checked into a VCS. I'd write an one-liner to scan the log file rather than setting up a conditional breakpoint. I particularly like doing this for a GUI application. A regression testing can be done by comparing logs.
- fourier_mode 7y agoWould this run in Cython. I am trying to understand a Cython code and something like this would be helpful.
- vaylian 7y agoThat looks really nice! I normally use pudb which is a curses-based debugger but pudb falls short when the debugged program runs with several threads or if it uses an asynchronous event loop. Having a non-interactive debugger like PySnooper could definitely help in such situations!
- deleted 7y ago[deleted]
- maximleon 7y agoDont think it works, tried few examples, it throws NotImplementedError at me all the time. Any hint? ~/anaconda3/lib/python3.7/site-packages/pysnooper/tracer.py in get_source_from_frame(frame) 75 pass 76 if source is None: ---> 77 raise NotImplementedError 78 79 # If we just read the source from a file, or if the loader did not NotImplementedError:
- kapitanjack 7y agoCan someone explain to me why to use this instead of `import pdb; pdb.set_trace()` ? I am new to python and am confused someone told me not to use print and use pdb, so how is this different ?
- nojvek 7y agoReally love this. Seems like it captures a whole bunch of interesting information when a function is invoked. I love auto-loggers like this where you can selectively capture interesting bits. This is the basis of reverse debugging I.e capturing chronological snapshots. Would love to see a Vscode extension that allows to step forward/backwards through time when an interesting thing happens.
- AtHeartEngineer 7y agoI missed this on HN last week but I just found it on Google. THANK YOU! I've been debugging weird serial errors for the last day, and this is helping a ton!