7 ms·

A Haskell kernel for IPython

208 points by lelf 13y ago

darkxanthos 13y ago
I'd love to see something like this but as a magic method for iPython Notebooks. Using those you can integrate bash, R, Ruby, SQL, etc. into the same notebook and pass data between them all. EDIT: Here's a link to most known magic extensions: https://github.com/ipython/ipython/wiki/Extensions-Index https://github.com/ipython/ipython/wiki/Extensions-Index
- wcbeard10 13y ago
  Can't find a link, but I believe I've seen at least one demo passing data back and forth between Julia and Python in a notebook that looks similar to what you're referring to. Edit: here's an example [0] [0] http://nbviewer.ipython.org/urls/raw.github.com/JuliaLang/IJulia.jl/master/python/doc/JuliaMagic.ipynb http://nbviewer.ipython.org/urls/raw.github.com/JuliaLang/IJ...
  - StefanKarpinski 13y ago
    Excellent work with IHaskell; I don't want to hijack this thread, but since it came up organically: The PyCall.jl [1] package allows transparent calling of Python from Julia. It's quite stable and exceptionally well done (it was written by Steven G. Johnson of FFTW [2] fame – his software is uniformly impressive). The IJulia package [3], which primarily provides a pure Julia IPython kernel so that you can interact with Julia via the various IPython front-ends, also includes [4] a Python package for calling Julia from Python plus IPython "magics" for calling Julia from a normal IPython process. This is rather more experimental than IJulia, but should be usable. [1] https://github.com/stevengj/PyCall.jl https://github.com/stevengj/PyCall.jl [2] http://fftw.org http://fftw.org [3] https://github.com/JuliaLang/IJulia.jl https://github.com/JuliaLang/IJulia.jl [4] https://github.com/JuliaLang/IJulia.jl/tree/master/python https://github.com/JuliaLang/IJulia.jl/tree/master/python
Ixiaus 13y ago
This is rad and I had no idea IPython was flexible enough that you could do this. Installing immediately!
arocks 13y ago
I think this is exciting and has a valid use case. The other day we were explaining Python list comprehensions which was heavily inspired by Haskell's list comprehensions. But it was no where as concise or expressive as Haskell. In many cases Haskell is very concise and it helps to have an option on the interactive interpreter to enter a Haskell one-liner to prototype a solution.
codemac 13y ago
It's like org-mode, but without all the emacs! :) I actually was just discussing with coworkers how I wish tools around org-mode could actually create an "editable" final state that didn't require emacs. I've been introducing iPython as a way around this, but I've been disappointed with forcing everyone into using python for our analysis work. Only one of my coworkers really uses emacs, and so sharing perf data and analysis quickly has mostly become iPython. This gives me hope!
- terhechte 13y ago
  It would be awesome to have a org mode -> ipython export that also supports all the languages that org mode supports (or rather, the main subset, I guess), though I think that that's a long way of. Julia lang also supports an iPython notebook implementation, but if I remember correctly, they replaced python with julia in there. It would probably be a lot of work to support all different languages within ipython. Actually, just porting Babel over might be easier.
  - zokier 13y ago
    There is a ipython emacs frontend, but I'm not familiar enough with emacs to say if it would help at all.
    - terhechte 13y ago
      I'm pretty sure that it only exports the structure, but it doesn't allow editing and running of the code in ipython since that would require having support for the plethora of languages. Example: There's a c extension to babel/org-mode that allows you to write a document like this in normal text, and then type: +begin_src c int test = 42; printf("%i", test); +end_src and then run and execute this code block in the document, and have the results appear underneath it. That is already pretty cool. But even cooler is that you can have a python block, and a R block and a ruby block, and then define a variable in python, and have ruby do something with that variable (i.e hand it over) and then create a new variable based on the ruby transformations, and hand it over to R to do some statistics processing with the data. Finally, You take the R output, display it as a table, and generate a plot for it. All in one text file. Truly awesome.
PieSquared 13y ago
Author here - didn't expect this to end up on Hacker News, given the niche interest :) Happy to answer any questions you guys have. (And if anyone has an interest in contributing, please get in touch!)
- roberto 13y ago
  Can you give more details on the development? Do you use any IPython code, or do you just follow the specification? What would be necessary if I wanted to implement this for another language? Thanks!
  - PieSquared 13y ago
    In addition to publishing the code, the IHaskell page also links to a blog post about how to go about implementing IPython kernels [0]. The implementation is entirely in Haskell and doesn't use any IPython code, but simply communicates with the frontends via ZeroMQ networking library. [0] http://andrew.gibiansky.com/blog/ipython/ipython-kernels/ http://andrew.gibiansky.com/blog/ipython/ipython-kernels/
    - roberto 13y ago
      This is amazing, thanks!
- mvermaat 13y ago
  This is really cool, well done! Be sure to send a message to the IPython-dev mailing list if you're happy to recieve contributions and/or feedback (also from IPython developers).
- fperez_org 13y ago
  IPython creator here, just to say I'm thrilled to see this! We've wanted more languages to follow this route, and while we have only had time to work directly with the Julia team, it's fabulous to see others following up with the idea. Please don't hesitate to ask us on the dev mailing list if you run into any problems. For one thing, we're planning on making some updates to the protocol spec soon, based mostly on the experience of the IJulia work. It would be great to have your input there as well, in case you spot something that wouldn't fit well the patterns you saw while doing the Haskell port. And besides, this means I now have one less excuse to get off my butt and learn Haskell! :)
StefanKarpinski 13y ago
This is great to see. The IPython project has grown to be much more than just a collection of nice interactive interfaces to Python. It is a well-designed JSON-based protocol between interactive interfaces and execution kernels – which can execute any language, not just Python. The fact that you can do things like IHaskell and IJulia is a testament to how well the IPython team has designed the protocol. The best part about this project is that it stands to reduce much of the duplicated / wasted effort that goes into recreating the same interface functionality for every single programming language. This is already allowing Julia (and now Haskell) to reap the benefits of the amazing browser UI work that's gone into the IPython Notebook – all for the very low cost of implementing a rather simple ZMQ message server.
- PieSquared 13y ago
  Thanks for the encouraging comments! I started working on this by looking at IJulia (huge fan of Julia!) and reading the source code and using that to figure out exactly what I should implement.
robrenaud 13y ago
IPython really needs a new name. It's too successful/cross language to just be associated with Python. The guide to learning Julia actually encourages using IPython Julia.
houshuang 13y ago
Very cool. Agree about the rebranding of IPython, but I'm sure that'll come as the community matures. IHaskell specific: As a beginning Haskell learner, is there a simple graphing framework that can make graphs shown inline in IPython (of course bindings to D3 etc to make web native graphs would be awesome, but even just images are fine) ... I'm currently learning maths/stats by implementing basic functions and experimenting with them in R, Julia, possibly Haskell etc. Being able to easily do simple graphs is crucial.
- PieSquared 13y ago
  IHaskell in its current state is the very first step towards that experience. I'm hoping that graphing and numeric support will come in the next few weeks to months. I'm also not sure that the Haskell community really has something as powerful and standard and nice interface-wise as NumPy, and if not, I'd be interested in creating it. (Currently playing around with building a nicer interface on top of hmatrix. REPA is pretty good, but doesn't use LAPACK/ATLAS/etc in the backend.)
hyperion2010 13y ago
I use ipython for all my (mostly scientific) python interpreter needs and have been looking for a functional programming language to learn. Between all the recent posts on haskell and now this, I think I know which one it will be.
616c 13y ago
Wow such great Haskell stuff comin to this site these days. Haskell on IPython? HELL YES. I am definitely taking my Haskell education more seriously this week.
qa5278217 13y ago
http://www.odnoklassniki.ru/dk?cmd=PopLayerPhoto&st.layer.cmd=PopLayerViewFriendPhotoStickyOuter&st.layer.revNav=off&st.layer.showNav=on&st.layer.type=FRIEND&st.layer.navStartPhotoId=528342796904&st.layer.photoId=528342796904&st.layer.happening=off&st.layer.se=off&st.layer.limitedUi=false&st.cmd=friendMain&st.friendId=otkmvwecjzzevueyzhc0qbtoqvhlrcklfhmdhct http://www.odnoklassniki.ru/dk?cmd=PopLayerPhoto&st.layer.cm...

208 points by lelf 13y ago

I'd love to see something like this but as a magic method for iPython Notebooks. Using those you can integrate bash, R, Ruby, SQL, etc. into the same notebook and pass data between them all. EDIT: Here's a link to most known magic extensions: https://github.com/ipython/ipython/wiki/Extensions-Index https://github.com/ipython/ipython/wiki/Extensions-Index

wcbeard10 13y ago
Can't find a link, but I believe I've seen at least one demo passing data back and forth between Julia and Python in a notebook that looks similar to what you're referring to. Edit: here's an example [0] [0] http://nbviewer.ipython.org/urls/raw.github.com/JuliaLang/IJulia.jl/master/python/doc/JuliaMagic.ipynb http://nbviewer.ipython.org/urls/raw.github.com/JuliaLang/IJ...
- StefanKarpinski 13y ago
  Excellent work with IHaskell; I don't want to hijack this thread, but since it came up organically: The PyCall.jl [1] package allows transparent calling of Python from Julia. It's quite stable and exceptionally well done (it was written by Steven G. Johnson of FFTW [2] fame – his software is uniformly impressive). The IJulia package [3], which primarily provides a pure Julia IPython kernel so that you can interact with Julia via the various IPython front-ends, also includes [4] a Python package for calling Julia from Python plus IPython "magics" for calling Julia from a normal IPython process. This is rather more experimental than IJulia, but should be usable. [1] https://github.com/stevengj/PyCall.jl https://github.com/stevengj/PyCall.jl [2] http://fftw.org http://fftw.org [3] https://github.com/JuliaLang/IJulia.jl https://github.com/JuliaLang/IJulia.jl [4] https://github.com/JuliaLang/IJulia.jl/tree/master/python https://github.com/JuliaLang/IJulia.jl/tree/master/python

Ixiaus 13y ago

This is rad and I had no idea IPython was flexible enough that you could do this. Installing immediately!

arocks 13y ago

I think this is exciting and has a valid use case. The other day we were explaining Python list comprehensions which was heavily inspired by Haskell's list comprehensions. But it was no where as concise or expressive as Haskell. In many cases Haskell is very concise and it helps to have an option on the interactive interpreter to enter a Haskell one-liner to prototype a solution.

codemac 13y ago

It's like org-mode, but without all the emacs! :) I actually was just discussing with coworkers how I wish tools around org-mode could actually create an "editable" final state that didn't require emacs. I've been introducing iPython as a way around this, but I've been disappointed with forcing everyone into using python for our analysis work. Only one of my coworkers really uses emacs, and so sharing perf data and analysis quickly has mostly become iPython. This gives me hope!

terhechte 13y ago
It would be awesome to have a org mode -> ipython export that also supports all the languages that org mode supports (or rather, the main subset, I guess), though I think that that's a long way of. Julia lang also supports an iPython notebook implementation, but if I remember correctly, they replaced python with julia in there. It would probably be a lot of work to support all different languages within ipython. Actually, just porting Babel over might be easier.
- zokier 13y ago
  There is a ipython emacs frontend, but I'm not familiar enough with emacs to say if it would help at all.
  - terhechte 13y ago
    I'm pretty sure that it only exports the structure, but it doesn't allow editing and running of the code in ipython since that would require having support for the plethora of languages. Example: There's a c extension to babel/org-mode that allows you to write a document like this in normal text, and then type: +begin_src c int test = 42; printf("%i", test); +end_src and then run and execute this code block in the document, and have the results appear underneath it. That is already pretty cool. But even cooler is that you can have a python block, and a R block and a ruby block, and then define a variable in python, and have ruby do something with that variable (i.e hand it over) and then create a new variable based on the ruby transformations, and hand it over to R to do some statistics processing with the data. Finally, You take the R output, display it as a table, and generate a plot for it. All in one text file. Truly awesome.

PieSquared 13y ago

Author here - didn't expect this to end up on Hacker News, given the niche interest :) Happy to answer any questions you guys have. (And if anyone has an interest in contributing, please get in touch!)

roberto 13y ago
Can you give more details on the development? Do you use any IPython code, or do you just follow the specification? What would be necessary if I wanted to implement this for another language? Thanks!
- PieSquared 13y ago
  In addition to publishing the code, the IHaskell page also links to a blog post about how to go about implementing IPython kernels [0]. The implementation is entirely in Haskell and doesn't use any IPython code, but simply communicates with the frontends via ZeroMQ networking library. [0] http://andrew.gibiansky.com/blog/ipython/ipython-kernels/ http://andrew.gibiansky.com/blog/ipython/ipython-kernels/
  - roberto 13y ago
    This is amazing, thanks!
mvermaat 13y ago
This is really cool, well done! Be sure to send a message to the IPython-dev mailing list if you're happy to recieve contributions and/or feedback (also from IPython developers).
fperez_org 13y ago
IPython creator here, just to say I'm thrilled to see this! We've wanted more languages to follow this route, and while we have only had time to work directly with the Julia team, it's fabulous to see others following up with the idea. Please don't hesitate to ask us on the dev mailing list if you run into any problems. For one thing, we're planning on making some updates to the protocol spec soon, based mostly on the experience of the IJulia work. It would be great to have your input there as well, in case you spot something that wouldn't fit well the patterns you saw while doing the Haskell port. And besides, this means I now have one less excuse to get off my butt and learn Haskell! :)

StefanKarpinski 13y ago

This is great to see. The IPython project has grown to be much more than just a collection of nice interactive interfaces to Python. It is a well-designed JSON-based protocol between interactive interfaces and execution kernels – which can execute any language, not just Python. The fact that you can do things like IHaskell and IJulia is a testament to how well the IPython team has designed the protocol. The best part about this project is that it stands to reduce much of the duplicated / wasted effort that goes into recreating the same interface functionality for every single programming language. This is already allowing Julia (and now Haskell) to reap the benefits of the amazing browser UI work that's gone into the IPython Notebook – all for the very low cost of implementing a rather simple ZMQ message server.

PieSquared 13y ago
Thanks for the encouraging comments! I started working on this by looking at IJulia (huge fan of Julia!) and reading the source code and using that to figure out exactly what I should implement.

robrenaud 13y ago

IPython really needs a new name. It's too successful/cross language to just be associated with Python. The guide to learning Julia actually encourages using IPython Julia.

houshuang 13y ago

Very cool. Agree about the rebranding of IPython, but I'm sure that'll come as the community matures. IHaskell specific: As a beginning Haskell learner, is there a simple graphing framework that can make graphs shown inline in IPython (of course bindings to D3 etc to make web native graphs would be awesome, but even just images are fine) ... I'm currently learning maths/stats by implementing basic functions and experimenting with them in R, Julia, possibly Haskell etc. Being able to easily do simple graphs is crucial.

PieSquared 13y ago
IHaskell in its current state is the very first step towards that experience. I'm hoping that graphing and numeric support will come in the next few weeks to months. I'm also not sure that the Haskell community really has something as powerful and standard and nice interface-wise as NumPy, and if not, I'd be interested in creating it. (Currently playing around with building a nicer interface on top of hmatrix. REPA is pretty good, but doesn't use LAPACK/ATLAS/etc in the backend.)

hyperion2010 13y ago

I use ipython for all my (mostly scientific) python interpreter needs and have been looking for a functional programming language to learn. Between all the recent posts on haskell and now this, I think I know which one it will be.

616c 13y ago

Wow such great Haskell stuff comin to this site these days. Haskell on IPython? HELL YES. I am definitely taking my Haskell education more seriously this week.

qa5278217 13y ago

http://www.odnoklassniki.ru/dk?cmd=PopLayerPhoto&st.layer.cmd=PopLayerViewFriendPhotoStickyOuter&st.layer.revNav=off&st.layer.showNav=on&st.layer.type=FRIEND&st.layer.navStartPhotoId=528342796904&st.layer.photoId=528342796904&st.layer.happening=off&st.layer.se=off&st.layer.limitedUi=false&st.cmd=friendMain&st.friendId=otkmvwecjzzevueyzhc0qbtoqvhlrcklfhmdhct http://www.odnoklassniki.ru/dk?cmd=PopLayerPhoto&st.layer.cm...