5 ms·

Interactive shell to a running Python process

146 points by daenz 15y ago

Peaker 15y ago
You can also run an RPyC* server in your process, and connect to it from any Python shell on any computer to control it. * http://rpyc.sourceforge.net/ http://rpyc.sourceforge.net/
sidharth 15y ago
This is great I thought Ipython was great. But this is awesome.
cmsj 15y ago
My most excellent friend Thomas Hurst implemented a remote python shell a few years ago, for use in debugging running instances of Terminator. You can see the code in question here: http://bazaar.launchpad.net/~gnome-terminator/terminator/trunk/view/head:/terminatorlib/debugserver.py http://bazaar.launchpad.net/~gnome-terminator/terminator/tru... and how it gets set up, starting at line 88, here: http://bazaar.launchpad.net/~gnome-terminator/terminator/trunk/view/head:/terminator#L88 http://bazaar.launchpad.net/~gnome-terminator/terminator/tru...
thelastnode 15y ago
What are the consequences of running this in production? What's the performance hit like, is it unsafe, etc.?
- anonymoushn 15y ago
  We've been using rfoo to do this. It doesn't seem to impact performance when you're not using it. When you are using it, it's unsafe if you do unsafe things. We primarily use it for debugging by passively inspecting the messed-up portion of the server's state.
- masklinn 15y ago
  > is it unsafe, etc.? As the readme notes, it's completely unsafe if you perform any mutation operation. As to the performance hit, I'd expect almost none when not in use for long-running processes: it spawns a thread, and that thread will then wait on a socket accept, no connection, no resources spent. You'll still be paying in memory for the server thread, but that's it (the server thread only creates two objects, an rlcompleter.Completer instance and a function).
storborg 15y ago
Would it be possible to use gdb or something similar to inject this into a running process?
- jeremya 15y ago
  Yes, pyrasite does just that with a python wrapper around the gdb commands. https://fedorahosted.org/pyrasite/ https://fedorahosted.org/pyrasite/ It was discussed here: http://news.ycombinator.com/item?id=3014484 http://news.ycombinator.com/item?id=3014484
- mixmastamyk 15y ago
  See the albertzeyer comment.
albertzeyer 15y ago
If you want to do the same without modifying the script (or if you have the script already running without the needed import for the remote debugging possibility), i.e. inject into a running process: https://github.com/albertz/pydbattach https://github.com/albertz/pydbattach
politician 15y ago
I wanted something like this back in the day to be able to poke around in the EVE Online client. Nice job!
jphackworth 15y ago
Very interesting. I would love to see this for other programming languages, especially Ruby (probably fairly easy) or Objective-C (may be quite hard).
- albertzeyer 15y ago
  For ObjC: https://github.com/albertz/Pyjector https://github.com/albertz/Pyjector (and then access the ObjC runtime via the Python ObjC bridge) https://github.com/albertz/FScriptAnywhereSIMBL https://github.com/albertz/FScriptAnywhereSIMBL
  - saurik 15y ago
    Or Cycript, which all of us iOS hackers use. ;P http://www.cycript.org/ http://www.cycript.org/ http://iphonedevwiki.net/index.php/Cycript http://iphonedevwiki.net/index.php/Cycript
- Xuzz 15y ago
  Objective-C also has Cycript at http://cycript.org/ http://cycript.org/, which has a hybrid syntax of JavaScript and Objective-C and supports injecting into any running process.b
- mcantelon 15y ago
  It's dead easy in Node using the REPL module (http://nodejs.org/docs/v0.6.5/api/repl.html http://nodejs.org/docs/v0.6.5/api/repl.html). I've used this to debug weird async issues.
- rue 15y ago
  Ruby's had the much more featureful Pry for some time now: http://github.com/pry/pry http://github.com/pry/pry. (And there have been various projects to provide some sub-capabilities for years.)
- Maven911 15y ago
  Id like to see it in bash, all i know now is to use the +x option but you need to do it before its run
swdunlop 15y ago
I use ipython's "ipdb" or plain old Python "pdb" for the same functionality, and a bit more -- people don't seem to realize that ipdb gives them a full-on REPL within the calling context. I usually wrap it into our error logging functions, so an environment variable enables debugging where I would normally have a catch-all -- very handy when looking for heisenbugs in AndBug. AntiLog -- https://gist.github.com/a256ed1295619fad7cfc https://gist.github.com/a256ed1295619fad7cfc AndBug -- https://github.com/swdunlop/andbug https://github.com/swdunlop/andbug
teyc 15y ago
Zope has this facility and boy it was useful for troubleshooting live systems.
- kra 15y ago
  And Twisted has Manhole, but for some reason we were never allowed to enable it on production deployments...

146 points by daenz 15y ago

You can also run an RPyC* server in your process, and connect to it from any Python shell on any computer to control it. * http://rpyc.sourceforge.net/ http://rpyc.sourceforge.net/

sidharth 15y ago

This is great I thought Ipython was great. But this is awesome.

cmsj 15y ago

My most excellent friend Thomas Hurst implemented a remote python shell a few years ago, for use in debugging running instances of Terminator. You can see the code in question here: http://bazaar.launchpad.net/~gnome-terminator/terminator/trunk/view/head:/terminatorlib/debugserver.py http://bazaar.launchpad.net/~gnome-terminator/terminator/tru... and how it gets set up, starting at line 88, here: http://bazaar.launchpad.net/~gnome-terminator/terminator/trunk/view/head:/terminator#L88 http://bazaar.launchpad.net/~gnome-terminator/terminator/tru...

thelastnode 15y ago

What are the consequences of running this in production? What's the performance hit like, is it unsafe, etc.?

anonymoushn 15y ago
We've been using rfoo to do this. It doesn't seem to impact performance when you're not using it. When you are using it, it's unsafe if you do unsafe things. We primarily use it for debugging by passively inspecting the messed-up portion of the server's state.
masklinn 15y ago
> is it unsafe, etc.? As the readme notes, it's completely unsafe if you perform any mutation operation. As to the performance hit, I'd expect almost none when not in use for long-running processes: it spawns a thread, and that thread will then wait on a socket accept, no connection, no resources spent. You'll still be paying in memory for the server thread, but that's it (the server thread only creates two objects, an rlcompleter.Completer instance and a function).

storborg 15y ago

Would it be possible to use gdb or something similar to inject this into a running process?

jeremya 15y ago
Yes, pyrasite does just that with a python wrapper around the gdb commands. https://fedorahosted.org/pyrasite/ https://fedorahosted.org/pyrasite/ It was discussed here: http://news.ycombinator.com/item?id=3014484 http://news.ycombinator.com/item?id=3014484
mixmastamyk 15y ago
See the albertzeyer comment.

albertzeyer 15y ago

If you want to do the same without modifying the script (or if you have the script already running without the needed import for the remote debugging possibility), i.e. inject into a running process: https://github.com/albertz/pydbattach https://github.com/albertz/pydbattach

politician 15y ago

I wanted something like this back in the day to be able to poke around in the EVE Online client. Nice job!

jphackworth 15y ago

Very interesting. I would love to see this for other programming languages, especially Ruby (probably fairly easy) or Objective-C (may be quite hard).

albertzeyer 15y ago
For ObjC: https://github.com/albertz/Pyjector https://github.com/albertz/Pyjector (and then access the ObjC runtime via the Python ObjC bridge) https://github.com/albertz/FScriptAnywhereSIMBL https://github.com/albertz/FScriptAnywhereSIMBL
- saurik 15y ago
  Or Cycript, which all of us iOS hackers use. ;P http://www.cycript.org/ http://www.cycript.org/ http://iphonedevwiki.net/index.php/Cycript http://iphonedevwiki.net/index.php/Cycript
Xuzz 15y ago
Objective-C also has Cycript at http://cycript.org/ http://cycript.org/, which has a hybrid syntax of JavaScript and Objective-C and supports injecting into any running process.b
mcantelon 15y ago
It's dead easy in Node using the REPL module (http://nodejs.org/docs/v0.6.5/api/repl.html http://nodejs.org/docs/v0.6.5/api/repl.html). I've used this to debug weird async issues.
rue 15y ago
Ruby's had the much more featureful Pry for some time now: http://github.com/pry/pry http://github.com/pry/pry. (And there have been various projects to provide some sub-capabilities for years.)
Maven911 15y ago
Id like to see it in bash, all i know now is to use the +x option but you need to do it before its run

swdunlop 15y ago

I use ipython's "ipdb" or plain old Python "pdb" for the same functionality, and a bit more -- people don't seem to realize that ipdb gives them a full-on REPL within the calling context. I usually wrap it into our error logging functions, so an environment variable enables debugging where I would normally have a catch-all -- very handy when looking for heisenbugs in AndBug. AntiLog -- https://gist.github.com/a256ed1295619fad7cfc https://gist.github.com/a256ed1295619fad7cfc AndBug -- https://github.com/swdunlop/andbug https://github.com/swdunlop/andbug

teyc 15y ago

Zope has this facility and boy it was useful for troubleshooting live systems.

kra 15y ago
And Twisted has Manhole, but for some reason we were never allowed to enable it on production deployments...