7 ms·

Am I the only person in the universe who doesn't like Jupyter? I much prefer tools like Rmarkdown and Sweave.

by gbrown 9y ago

tincholio 9y ago
Nope... you're not the only one
jasongrout 9y ago
We would love to hear what you like about Rmarkdown and Sweave. Jupyter tooling is always improving, and we are very interested in engaging with users about their needs, and helping grow the ecosystem to be able to address those.
- goerz 9y ago
  The number one feature request by far at this point would be collaborative features in the notebook. Not really because I actually want to collaboratively edit a notebook with others, but because I would like to open the same notebook in two separate browser windows (side-by-side), and edit a cell at the top of the notebook in one, and a cell at the bottom in the other, and have things get merged automatically. Or, not having to worry that I left a (remote) notebook open on my work computer when I connect to the same server from home, causing parts of the notebook to be accidentally overwritten.
  - targafarian 8y ago
    JupyterLab has the ability to open multiple "views" on the same file. http://jupyterlab.readthedocs.io/en/latest/user/notebook.html?highlight=views http://jupyterlab.readthedocs.io/en/latest/user/notebook.htm...
    - goerz 8y ago
      That's amazing! I'll have to try out JupyterLab sometime very soon. I was holding out because I'm using a whole bunch of plugins for the classic notebook from https://github.com/ipython-contrib/jupyter_contrib_nbextensions https://github.com/ipython-contrib/jupyter_contrib_nbextensi... that I think have't been ported yet
- gbrown 9y ago
  Using a web browser as an IDE just seems like a solution in search of a problem. Also, maybe I just never spent enough time working with Jupyter, but it seemed to me that it encourages a sort of exploratory workflow, really well suited to teaching programming and data science. It was less clear to me how it could be used well in production environments. I use Rmarkdown and Sweave to write homeworks for my students in a very Jupyter way. I also use them to generate data driven static webpages, procedurally generate production quality and easily formatted PDF and HTML reports. I also use them as a templating system for auto-generated model diagnostic emails. Perhaps I need to return to Jupyter to see what I'm missing, but I don't really know what purpose it would serve, or what kind of work it would make easier.
  - takluyver 9y ago
    Doing the interface in the web browser has its ups and downs: it makes some things more awkward locally, but it's easy to deliver the same interface remotely - e.g. a university can run a JupyterHub instance for a course, and students visit a URL and login. nteract is an attempt to make a notebook interface as a local application.
    - gbrown 9y ago
      For sure, I totally see how it's useful as a teaching tool. I don't see how it fits into a proper production system.
      - carreau 9y ago
        Different system and requirement need different tools, if you prefer text editor. Depending on your preference look at Emacs IPython Notebook, the Jupyter VS-Code extension, or Atom Hydrogen. That will basically let you select chunk of code and execute in a kernel. You do not have to use the notebook format, or the browser based editor. JupyterLab also allow Rmarkdown-like workflow where code-blocks in a markdown document can be executed to display graph. I believe the important part is to allow interoperability between different ways people want to work. You can't have 1 size fits all, and there are still a lot of work that can be done to cover some use case.
- asdlllkasdasd 9y ago
  Rmarkdown has the better version control story.
- nerdponx 9y ago
  My biggest frustrations with Jupyter are (see #4 for comments on Sweave etc): 1. The default front-end is a weak platform for getting work done. It's a JavaScript code editor. It will never be as good as my personal text editor configuration. It will never be as good as an IDE like RStudio, Spyder, or Pycharm. It's good that there are keyboard shortcuts for doing things like adding cells, and extensions for things like folding cells and adding a table of contents. But it still isn't terribly comfortable to use all day. Also I personally hate doing everything in a browser. Apart from some useful notebook extensions, there are no viable alternative front ends yet. 2. Running a remote kernel is a pain in the ass (cat a config file then manually tunnel 4 ports over SSH), and I can't seem to get it to work on Windows at all. This is an issue at my company because we do a lot of work on remote servers that can be accessed only through SSH or JupyterHub. Individual users do not have control over the latter, so we are stuck with the inadequate default experience I just described above. 3. No kernel other than Ipython is mature. IRKernel is getting there. Everything else is at best a beta-quality product. 4. Notebooks are not a plain text file format. Hand editing a notebook is messy. They do not play well with version control systems and diff tools. RMarkdown and Knitr/Sweave are just preprocessors for established plain text formats (Markdown and Latex with some extra syntax). With those formats you can take advantage of a wealth of existing tooling, as well as having the freedom to edit the file in a normal text editor without having to rely on a special front end. Ironically having everything formatted as JSON should make it easier to write those special front ends, but I have not seen any good ones yet.
  - goerz 9y ago
    To address “hand editing” problem I wrote a little vim plugin that wraps around notedown (https://github.com/aaren/notedown https://github.com/aaren/notedown) to edit notebooks on the fly in a markdown format. It’s not perfect, but it goes a long way for quickly editing notebooks “as a whole”: https://github.com/goerz/ipynb_notedown.vim https://github.com/goerz/ipynb_notedown.vim Other than that, I try to put any lengthy code in functions that are in a module alongside the notebook, so that the notebook mostly contains one-line commands to do kick off a calculation or to generate a plot. I also have a shortcut that copies the content of the current browser text field (notebook cell) into MacVim, and pastes it back automatically as soon as I close the editor.
- mschuetz 9y ago
  I guess it would be nice to make it run within an electron environment, without extra console and browser tabs to run it in. Just jupyter running as an electron app with maybe tabs for each notebook to give it a more native desktop app feel.
  - po84 9y ago
    Check out https://nteract.io/ https://nteract.io/. It's based on Jupyter standards (protocol, kernels, notebook format), written in JS, and runs on your desktop.
ILikeConemowk 9y ago
>Am I the only person in the universe who doesn't like Jupyter? Yes ;]
chubot 9y ago
I've been using Python for 15 years, and I also haven't gotten into Jupyter. I worked with some research scientists who used it, but even they ported their programs to plain text files in order to get version control with textual diffs (which is IMO better for collaboration.) One reason I don't use it is that I started doing data science before the Python ecosystem was viable -- before Pandas existed. I use R for data science (which I generally find superior due to Hadley Wickham's libraries). I know Jupyter supports R now, but I already had a terminal/web-based workflow by the time that happened. ----- More importantly, I think this recent blog post finally crystallized why I don't program in REPLs: Because they encourage global variables! I naturally structure my code into functions from the outset. I don't like the persistence because it can lead to "wrong" programs. I prefer to test my programs with a "clean slate", i.e. by starting a new process. http://willcrichton.net/notes/programming-in-the-debugger/ http://willcrichton.net/notes/programming-in-the-debugger/ Jupyter’s structure of delimited code cells enables a programming style where each can be treated like an atomic unit, where if it completes, then its effects are persisted in memory for other code cells to process. However, this style of programming with Jupyter has its limits. For example, Jupyter penalizes abstraction by removing this interactive debuggability. In other words, if you put all your code in functions like I do, then Jupyter doesn't add anything. It doesn't let you "step through" the function like a debugger does. Though, I think that I should somehow try to get over this because there are a lot of benefits to something like Jupyter, like having graphics inline. Or maybe Jupyter just needs an integrated debugger? And maybe the ability to clear state or tree-walk definitions? I don't like having unused definitions laying around in my workspace. Also, does Jupyter have any notion of data flow? I don't think it does, because Python doesn't. I think Observable might address some of my gripes, but I haven't tried it yet: https://beta.observablehq.com/ https://beta.observablehq.com/
- celias 9y ago
  Take a look at Cauldron, aka the unnotebook -http://www.unnotebook.com http://www.unnotebook.com The author discussed it on pocastinit episode 111 - https://www.podcastinit.com/episode-111-cauldron-notebook-with-scott-ernst/ https://www.podcastinit.com/episode-111-cauldron-notebook-wi...
pmarin 9y ago
I would like something like make for notebook cells inside a repl. I would write the notebook in my text editor (not in a web form) execute "reload" in the repl, execute the cells that have been changed and render the webpage. The output should be a complete self contained html file with everything else embeded and of course not a single line of javascript.
- chubot 9y ago
  BTW I'm pretty sure that this is exactly what Sweave is. You write in a text editor, and then invoke a batch process to generate an HTML file with mixed code and data. Wikipedia says it does incremental rebuilding, which was news to me. I've never used it, but I know a lot of R users who use it. https://en.wikipedia.org/wiki/Sweave https://en.wikipedia.org/wiki/Sweave
- po84 9y ago
  If I understand you correctly, a combination of http://ipython.readthedocs.io/en/stable/interactive/magics.html#magic-notebook http://ipython.readthedocs.io/en/stable/interactive/magics.h... and http://nbconvert.readthedocs.io/en/latest/execute_api.html http://nbconvert.readthedocs.io/en/latest/execute_api.html might get you there. Post if you build something like this!
jncraton 9y ago
You are definitely not alone. I personally use pweave, which is essentially a thin Markdown/LaTeX parser wrapping Jupyter kernels for injecting the output of code chunks. I like being able to use my existing tools (text editor, make, version control, etc). I also like being able to write clean functions and run unit tests while still ultimately being able generate a clean final document.
andrewla 9y ago
I'm with you (although you should check out knitr on the R side). I see two use cases for this sort of notebook thing. One is reproducible research, where I find that a woven solution is far preferable over a notebook. I can use version control to assist in iteration, and use an editor of my choice. The other is exploratory analysis, which Rmarkdown/Sweave/knitr is not really a good substitute for. For exploratory analysis, I've found that I use the terminal integration in Vim together with the REPL works well for this. It lets me save half-working stuff and record some of my thought process; in a Jupyter notebook the only things I can leave in are working code snippets.
- your-nanny 9y ago
  I use the reply for exploration, and my notebook for final product. Do same in Matlab, but just with code.
yiyus 9y ago
I don't use it because I'm used to other things, but I really love the concept and I am sure I would be a user if it had existed some years ago. More importantly, I know it is doing a great job helping many scientists. Other tools like the ones you mention may improve certain aspects or fit better in some workflows, but Jupyter is the clear leader in the field, it is the one that changed the state of the art, even although some similar tools already existed and new (and maybe better) ones will come. I think the award is well deserved.
- gbrown 9y ago
  I think the workflow is great for the web tutorials I see all over the place, and I'm sure it's a pretty good fit for exploratory data analysis and model building. On the other hand, I see a lot of people pushing it as the must have tool for data science broadly, and I just don't see it fitting into most production workflows. When a model and the associated data pipelines hit production, they need to be version controlled - plain text files can't be beat for that. The idea that the same tool should be both IDE and report is also very strange to me - I can see how it lowers the barrier to entry in some cases, but it doesn't seem optimal for most uses. I do agree that Jupyter has helped a lot to get reproducibility on people's radars, and that's a positive thing.
deleted 9y ago
[deleted]
nicodjimenez 9y ago
I agree. My big issue with Jupiter is that it tries to replace the command line instead of augmenting it. I want a web app that is a full fledged Linux command line plus some API's for visualizing information, don't think that exists yet. I just cannot work in any environment that doesn't support VIM + tmux + linux command line utilities.

Am I the only person in the universe who doesn't like Jupyter? I much prefer tools like Rmarkdown and Sweave.

by gbrown 9y ago

Am I the only person in the universe who doesn't like Jupyter? I much prefer tools like Rmarkdown and Sweave.

tincholio 9y ago

Nope... you're not the only one

jasongrout 9y ago

We would love to hear what you like about Rmarkdown and Sweave. Jupyter tooling is always improving, and we are very interested in engaging with users about their needs, and helping grow the ecosystem to be able to address those.

goerz 9y ago
The number one feature request by far at this point would be collaborative features in the notebook. Not really because I actually want to collaboratively edit a notebook with others, but because I would like to open the same notebook in two separate browser windows (side-by-side), and edit a cell at the top of the notebook in one, and a cell at the bottom in the other, and have things get merged automatically. Or, not having to worry that I left a (remote) notebook open on my work computer when I connect to the same server from home, causing parts of the notebook to be accidentally overwritten.
- targafarian 8y ago
  JupyterLab has the ability to open multiple "views" on the same file. http://jupyterlab.readthedocs.io/en/latest/user/notebook.html?highlight=views http://jupyterlab.readthedocs.io/en/latest/user/notebook.htm...
  - goerz 8y ago
    That's amazing! I'll have to try out JupyterLab sometime very soon. I was holding out because I'm using a whole bunch of plugins for the classic notebook from https://github.com/ipython-contrib/jupyter_contrib_nbextensions https://github.com/ipython-contrib/jupyter_contrib_nbextensi... that I think have't been ported yet
gbrown 9y ago
Using a web browser as an IDE just seems like a solution in search of a problem. Also, maybe I just never spent enough time working with Jupyter, but it seemed to me that it encourages a sort of exploratory workflow, really well suited to teaching programming and data science. It was less clear to me how it could be used well in production environments. I use Rmarkdown and Sweave to write homeworks for my students in a very Jupyter way. I also use them to generate data driven static webpages, procedurally generate production quality and easily formatted PDF and HTML reports. I also use them as a templating system for auto-generated model diagnostic emails. Perhaps I need to return to Jupyter to see what I'm missing, but I don't really know what purpose it would serve, or what kind of work it would make easier.
- takluyver 9y ago
  Doing the interface in the web browser has its ups and downs: it makes some things more awkward locally, but it's easy to deliver the same interface remotely - e.g. a university can run a JupyterHub instance for a course, and students visit a URL and login. nteract is an attempt to make a notebook interface as a local application.
  - gbrown 9y ago
    For sure, I totally see how it's useful as a teaching tool. I don't see how it fits into a proper production system.
    - carreau 9y ago
      Different system and requirement need different tools, if you prefer text editor. Depending on your preference look at Emacs IPython Notebook, the Jupyter VS-Code extension, or Atom Hydrogen. That will basically let you select chunk of code and execute in a kernel. You do not have to use the notebook format, or the browser based editor. JupyterLab also allow Rmarkdown-like workflow where code-blocks in a markdown document can be executed to display graph. I believe the important part is to allow interoperability between different ways people want to work. You can't have 1 size fits all, and there are still a lot of work that can be done to cover some use case.
asdlllkasdasd 9y ago
Rmarkdown has the better version control story.
nerdponx 9y ago
My biggest frustrations with Jupyter are (see #4 for comments on Sweave etc): 1. The default front-end is a weak platform for getting work done. It's a JavaScript code editor. It will never be as good as my personal text editor configuration. It will never be as good as an IDE like RStudio, Spyder, or Pycharm. It's good that there are keyboard shortcuts for doing things like adding cells, and extensions for things like folding cells and adding a table of contents. But it still isn't terribly comfortable to use all day. Also I personally hate doing everything in a browser. Apart from some useful notebook extensions, there are no viable alternative front ends yet. 2. Running a remote kernel is a pain in the ass (cat a config file then manually tunnel 4 ports over SSH), and I can't seem to get it to work on Windows at all. This is an issue at my company because we do a lot of work on remote servers that can be accessed only through SSH or JupyterHub. Individual users do not have control over the latter, so we are stuck with the inadequate default experience I just described above. 3. No kernel other than Ipython is mature. IRKernel is getting there. Everything else is at best a beta-quality product. 4. Notebooks are not a plain text file format. Hand editing a notebook is messy. They do not play well with version control systems and diff tools. RMarkdown and Knitr/Sweave are just preprocessors for established plain text formats (Markdown and Latex with some extra syntax). With those formats you can take advantage of a wealth of existing tooling, as well as having the freedom to edit the file in a normal text editor without having to rely on a special front end. Ironically having everything formatted as JSON should make it easier to write those special front ends, but I have not seen any good ones yet.
- goerz 9y ago
  To address “hand editing” problem I wrote a little vim plugin that wraps around notedown (https://github.com/aaren/notedown https://github.com/aaren/notedown) to edit notebooks on the fly in a markdown format. It’s not perfect, but it goes a long way for quickly editing notebooks “as a whole”: https://github.com/goerz/ipynb_notedown.vim https://github.com/goerz/ipynb_notedown.vim Other than that, I try to put any lengthy code in functions that are in a module alongside the notebook, so that the notebook mostly contains one-line commands to do kick off a calculation or to generate a plot. I also have a shortcut that copies the content of the current browser text field (notebook cell) into MacVim, and pastes it back automatically as soon as I close the editor.
mschuetz 9y ago
I guess it would be nice to make it run within an electron environment, without extra console and browser tabs to run it in. Just jupyter running as an electron app with maybe tabs for each notebook to give it a more native desktop app feel.
- po84 9y ago
  Check out https://nteract.io/ https://nteract.io/. It's based on Jupyter standards (protocol, kernels, notebook format), written in JS, and runs on your desktop.

ILikeConemowk 9y ago

>Am I the only person in the universe who doesn't like Jupyter? Yes ;]

chubot 9y ago

I've been using Python for 15 years, and I also haven't gotten into Jupyter. I worked with some research scientists who used it, but even they ported their programs to plain text files in order to get version control with textual diffs (which is IMO better for collaboration.) One reason I don't use it is that I started doing data science before the Python ecosystem was viable -- before Pandas existed. I use R for data science (which I generally find superior due to Hadley Wickham's libraries). I know Jupyter supports R now, but I already had a terminal/web-based workflow by the time that happened. ----- More importantly, I think this recent blog post finally crystallized why I don't program in REPLs: Because they encourage global variables! I naturally structure my code into functions from the outset. I don't like the persistence because it can lead to "wrong" programs. I prefer to test my programs with a "clean slate", i.e. by starting a new process. http://willcrichton.net/notes/programming-in-the-debugger/ http://willcrichton.net/notes/programming-in-the-debugger/ Jupyter’s structure of delimited code cells enables a programming style where each can be treated like an atomic unit, where if it completes, then its effects are persisted in memory for other code cells to process. However, this style of programming with Jupyter has its limits. For example, Jupyter penalizes abstraction by removing this interactive debuggability. In other words, if you put all your code in functions like I do, then Jupyter doesn't add anything. It doesn't let you "step through" the function like a debugger does. Though, I think that I should somehow try to get over this because there are a lot of benefits to something like Jupyter, like having graphics inline. Or maybe Jupyter just needs an integrated debugger? And maybe the ability to clear state or tree-walk definitions? I don't like having unused definitions laying around in my workspace. Also, does Jupyter have any notion of data flow? I don't think it does, because Python doesn't. I think Observable might address some of my gripes, but I haven't tried it yet: https://beta.observablehq.com/ https://beta.observablehq.com/

celias 9y ago
Take a look at Cauldron, aka the unnotebook -http://www.unnotebook.com http://www.unnotebook.com The author discussed it on pocastinit episode 111 - https://www.podcastinit.com/episode-111-cauldron-notebook-with-scott-ernst/ https://www.podcastinit.com/episode-111-cauldron-notebook-wi...

pmarin 9y ago

I would like something like make for notebook cells inside a repl. I would write the notebook in my text editor (not in a web form) execute "reload" in the repl, execute the cells that have been changed and render the webpage. The output should be a complete self contained html file with everything else embeded and of course not a single line of javascript.

chubot 9y ago
BTW I'm pretty sure that this is exactly what Sweave is. You write in a text editor, and then invoke a batch process to generate an HTML file with mixed code and data. Wikipedia says it does incremental rebuilding, which was news to me. I've never used it, but I know a lot of R users who use it. https://en.wikipedia.org/wiki/Sweave https://en.wikipedia.org/wiki/Sweave
po84 9y ago
If I understand you correctly, a combination of http://ipython.readthedocs.io/en/stable/interactive/magics.html#magic-notebook http://ipython.readthedocs.io/en/stable/interactive/magics.h... and http://nbconvert.readthedocs.io/en/latest/execute_api.html http://nbconvert.readthedocs.io/en/latest/execute_api.html might get you there. Post if you build something like this!

jncraton 9y ago

You are definitely not alone. I personally use pweave, which is essentially a thin Markdown/LaTeX parser wrapping Jupyter kernels for injecting the output of code chunks. I like being able to use my existing tools (text editor, make, version control, etc). I also like being able to write clean functions and run unit tests while still ultimately being able generate a clean final document.

andrewla 9y ago

I'm with you (although you should check out knitr on the R side). I see two use cases for this sort of notebook thing. One is reproducible research, where I find that a woven solution is far preferable over a notebook. I can use version control to assist in iteration, and use an editor of my choice. The other is exploratory analysis, which Rmarkdown/Sweave/knitr is not really a good substitute for. For exploratory analysis, I've found that I use the terminal integration in Vim together with the REPL works well for this. It lets me save half-working stuff and record some of my thought process; in a Jupyter notebook the only things I can leave in are working code snippets.

your-nanny 9y ago
I use the reply for exploration, and my notebook for final product. Do same in Matlab, but just with code.

yiyus 9y ago

I don't use it because I'm used to other things, but I really love the concept and I am sure I would be a user if it had existed some years ago. More importantly, I know it is doing a great job helping many scientists. Other tools like the ones you mention may improve certain aspects or fit better in some workflows, but Jupyter is the clear leader in the field, it is the one that changed the state of the art, even although some similar tools already existed and new (and maybe better) ones will come. I think the award is well deserved.

gbrown 9y ago
I think the workflow is great for the web tutorials I see all over the place, and I'm sure it's a pretty good fit for exploratory data analysis and model building. On the other hand, I see a lot of people pushing it as the must have tool for data science broadly, and I just don't see it fitting into most production workflows. When a model and the associated data pipelines hit production, they need to be version controlled - plain text files can't be beat for that. The idea that the same tool should be both IDE and report is also very strange to me - I can see how it lowers the barrier to entry in some cases, but it doesn't seem optimal for most uses. I do agree that Jupyter has helped a lot to get reproducibility on people's radars, and that's a positive thing.

deleted 9y ago

[deleted]

nicodjimenez 9y ago

I agree. My big issue with Jupiter is that it tries to replace the command line instead of augmenting it. I want a web app that is a full fledged Linux command line plus some API's for visualizing information, don't think that exists yet. I just cannot work in any environment that doesn't support VIM + tmux + linux command line utilities.