3 ms·

Yes. Good things here in the big list [1]. Fish style auto completion and traceback improvements are especially welcome. I find myself reaching for IPython more

Yes. Good things here in the big list [1]. Fish style auto completion and traceback improvements are especially welcome. I find myself reaching for IPython more readily than browser Jupyter. REPL just feels better for control, though you can't beat browser for visuals. In that vein I have probably somewhat obscure question, but since OP is here I thought I'd give it a shot. I'd like to use Unix shell in concert with IPython. I'd send data to IPython kernel from zsh terminal sessions and call functions get data back. This data I could then send to Visidata or browser for bespoke visualization. Or whatever else is available in the shell. I think Jupyter's messaging protocol kind of allows this, but I haven't managed to grasp the fine details enough to get anywhere. I can get to shell from IPython, but from the outside this REPL isn't accessible from the Unix "REPL". [1] https://ipython.readthedocs.io/en/latest/whatsnew/version8.html#ipython-8-0 https://ipython.readthedocs.io/en/latest/whatsnew/version8.h...

carreau 5y ago
You likely want to use something like https://github.com/jupyter/jupyter_console https://github.com/jupyter/jupyter_console, or ipykernel directly to have a persistent python process. One issue is that shells are text based, so you have to do a lot of serialisation/deserialisation. But honestly at that point I would just look into https://xon.sh/ https://xon.sh/ that blends Python and Shell together. IPython and Xonsh devs are friends, so if you need anything from one into the other it's likely doable.
- jarpineh 5y ago
  Thing is, I like my zsh. Sometimes I use Nu shell as well, which does structured data better. Serialization I think needs to happen anyway to get data into IPython from outside. I have tried Xonsh once or twice, but it was too large change. Unix shell does some things very well, Python I like to use for other things. It is the crossover I’d like to smooth out. I am confused on what having a persistent Python process means in this context. Isn’t IPython already that? Jupyter console states it’s a single process IPython terminal. That does leave me wondering what is different when I start IPython vs Jupyter console. I might have assumed years ago that they are mutually exclusive…
  - carreau 5y ago
    Well, jupyter_console itself is a frontend/CLI, it starts another process (ipykernel) and communicate with it using ZMQ. IPykernel itself use IPython as a library to execute code. It might be easier to send code to an ipykernel started with jupyter_console, as it already has an eventloop and is listening to sockets, that trying to talk to an IPython. Plus using ZMQ you'll have richer messages than just text.
    - jarpineh 5y ago
      Ok. Thank you for the hints. And your work on IPython. It is very much appreciated.
jeroenjanssens 5y ago
I have written a bit about combining Jupyter console with the command line [0]. Not exactly the workflow you describe, but perhaps still useful. [0] https://datascienceatthecommandline.com/2e/chapter-10-polyglot-data-science.html#jupyter https://datascienceatthecommandline.com/2e/chapter-10-polygl...
- jarpineh 5y ago
  Thank you. Calling to shell isn’t really what I’m thinking of doing. But I have used it many times in the past for great effect. I think I have your book’s first edition! Thank you for the link. And the updated edition! I’ll have to check what’s changed.

carreau 5y ago

You likely want to use something like https://github.com/jupyter/jupyter_console https://github.com/jupyter/jupyter_console, or ipykernel directly to have a persistent python process. One issue is that shells are text based, so you have to do a lot of serialisation/deserialisation. But honestly at that point I would just look into https://xon.sh/ https://xon.sh/ that blends Python and Shell together. IPython and Xonsh devs are friends, so if you need anything from one into the other it's likely doable.

jarpineh 5y ago
Thing is, I like my zsh. Sometimes I use Nu shell as well, which does structured data better. Serialization I think needs to happen anyway to get data into IPython from outside. I have tried Xonsh once or twice, but it was too large change. Unix shell does some things very well, Python I like to use for other things. It is the crossover I’d like to smooth out. I am confused on what having a persistent Python process means in this context. Isn’t IPython already that? Jupyter console states it’s a single process IPython terminal. That does leave me wondering what is different when I start IPython vs Jupyter console. I might have assumed years ago that they are mutually exclusive…
- carreau 5y ago
  Well, jupyter_console itself is a frontend/CLI, it starts another process (ipykernel) and communicate with it using ZMQ. IPykernel itself use IPython as a library to execute code. It might be easier to send code to an ipykernel started with jupyter_console, as it already has an eventloop and is listening to sockets, that trying to talk to an IPython. Plus using ZMQ you'll have richer messages than just text.
  - jarpineh 5y ago
    Ok. Thank you for the hints. And your work on IPython. It is very much appreciated.

jeroenjanssens 5y ago

I have written a bit about combining Jupyter console with the command line [0]. Not exactly the workflow you describe, but perhaps still useful. [0] https://datascienceatthecommandline.com/2e/chapter-10-polyglot-data-science.html#jupyter https://datascienceatthecommandline.com/2e/chapter-10-polygl...

jarpineh 5y ago
Thank you. Calling to shell isn’t really what I’m thinking of doing. But I have used it many times in the past for great effect. I think I have your book’s first edition! Thank you for the link. And the updated edition! I’ll have to check what’s changed.