6 ms·

> JupyterLab adapts easily to multiple workflow needs, letting you move from a Notebook/narrative focus to a script/console one. I'm not sure I like where that

by bicubic 10y ago

> JupyterLab adapts easily to multiple workflow needs, letting you move from a Notebook/narrative focus to a script/console one. I'm not sure I like where that design is going. It's starting to look an awful lot like RStudio and Matlab and I moved away from those tools for a reason. My favourite thing about Jupyter is that it is focused on notebooks and narrative. It brought about a revolution of sorts; now we have people blogging and writing papers in Jupyter, github is full of random useful notebooks. This design almost seems like a step backwards in that regard.

denfromufa 10y ago
Notebook is still primary interface for final version of your analysis. But during iteration and experimentation of analysis, the other parts of jupyterlab are really helpful! Before all of these was not so nicely integrated.
- gcr 10y ago
  But from what I can tell, it still doesn't look integrated. It looks like separate tools all glued on the same webpage. Now I have yet a separate terminal, a separate file manager, a separate Python REPL, ... How is this any better than just keeping those applications open and switching between them? I really hope this isn't just a poorly-implemented window manager inside a web browser.
  - travisoliphant 10y ago
    For some plugins integration just means visual integration (i.e. window-manager integration). And, to be clear, this one is not poorly-implemented. It's based on a very powerful web-application toolkit called PhosphorJS written by very experienced UI developers. For other plugins there is very nice integration so that you can have interactions in one plugin update outputs in another and vice-versa. The ease of creating a connected ecosystem of web-applications connected to your workflow is one of the powerful aspects of the new platform. What would great integration look like to you?
    - gcr 10y ago
      I would want integration into my existing workflow, not integration that replaces my workflow with something completely different. For me, this means using emacs-ipython-notebook. This brings the ability to interact with a running IPython Kernel into the text editor that I use daily. It's the best of both worlds: interactive plotting+notebooks+rapid development assistance from IPython, and thanks to Emacs, all the keybindings are what I expect and I'm already familiar with how to extend the environment to suit my needs. EIN has been transformative for me. Perhaps my needs don't match the needs of others. For most people, I imagine the Jupyter-lab style of environment that's self-contained in a browser would in fact be a step up from the ad-hoc notepad++ windows and PuTTY sessions used previously.
  - denfromufa 10y ago
    The layout is customizable - you keep what is needed
p4wnc6 10y ago
I agree, but would phrase it more like this: Jupyter has succeeded because each of the different major modes of interaction have been decoupled. If you just want to use it in a shell, you don't need to involve the browser at all. If you want a narrative format for sharing, presenting, or converting to slides, you can easily launch that environment. This feels like a big step backwards for me too. It's effectively like replicating the MATLAB / Octave / PyDev (Eclipse) sort of IDE-with-extras-plus-console that is so, so cripplingly bad, but acting like it's great and new just because it's all in the browser. If you're a fan of productivity, you shouldn't want to do that kind of stuff in a browser. Heck, I even disable all of the dropdown menus in Emacs because even that is too much of a productivity hindrance / inefficient use of monitor space when I am writing, reading, and thinking about code. This is one of those things where I feel that it doesn't actually solve practical use cases, doesn't make people more productive, but because there is a big hype engine behind it, it gets adopted and talked about anyway, and eventually becomes the sort of thing that an Office Space kind of manager starts to force you to use ... which really scares me. Stay off my lawn.
- denfromufa 10y ago
  Ipython terminal is still there, but you can't expect ma$$$ or Excel users to start working in vim or emacs based workflow. Besides browser provides rich output, cross-platform support. The big thing for me are widgets.
  - p4wnc6 10y ago
    The returns to widgets in these kinds of things are soo diminishing. It looks cool the first 5 times, but then your boss says, "Great. Productionize it" and it's a world of shit. The widgets are good for throwaway demos and presentations, but if you are designing some functions for a business API of some kind, and your form of reporting is to show people something with a widget, now they want your internal API to include the widget, which is a dangerous game. I like super boring static charts, such that I can completely decouple the presentation from whatever business API was used behind the scenes for the data that's displayed. Take a super severe Occam's Razor approach to absolutely avoiding any kind of animation or interactive plot for as long as possible, and only if there is some kind of outrageously severe business case that absolutely demands it (which there almost never is) will I resort to designing a way to productionize the widget aspect of the report visualization as well. For this reason, I now see widgets like from IPython as a huge minefield. It looks pretty, but it's a big trap, and it doesn't add nearly the value everyone thinks it does. (Just to be clear, I'm only talking about the non-demoware case, which is the kind of case I have always worked with.)
nl 10y ago
Context: We use Jupyter heavily (mostly against Spark). In my experience there is a set of things that "traditional" Jupyter notebooks does really well. Anytime you have a linear flow of steps the notebook metaphor works really well. However, if you are doing things approaching traditional development, where you have multiple sources of data, or loops that require debugging, or basically anything that isn't linear in nature it doesn't work so well. I wouldn't want to lose traditional notebooks, but I'd love to be able to offer people something like this that offers better debugging and some development tool support, rather than jumping to a full desktop IDE.
- bicubic 10y ago
  My experience is in line with yours, debugging loops and functions is a big pain point. However, I think there's a much better solution to be had here, which is to add more powerful debugging capabilities to Notebook. I think Notebook has potential for new debugging paradigms, imagine for example being able to break anywhere in a cell and get a new 'forked cell' which operates in the context of the code that you just broke into. I think that's the better direction to go instead of reverting to the very interfaces and paradigms that we moved away from.
  - nl 10y ago
    I'm not sure (And by that I don't mean I disagree: I'm genuinely unsure). To me, the traditional IDEs do work well for debugging and software development. Notebooks are great for explanatory examples and interactive experiments. I think these are different to the type of software development I do when I use an IDE. For example, I find notebooks great for rapid iteration of parameters when I'm doing "data science", or indeed most of the feature extraction->modelling->prediction data science pipeline. What I don't find them good for is developing new algorithms. It isn't clear to me if this is an inherit limitation of the notebook format, or just something where it needs new developments. (To be clear, I've also used both Zeppelin and Beaker notebooks and don't see any particular advantages. I've also used R Studio, but I don't really know enough R to comment sensibly on that)
  - radiowave 10y ago

by bicubic 10y ago

denfromufa 10y ago

Notebook is still primary interface for final version of your analysis. But during iteration and experimentation of analysis, the other parts of jupyterlab are really helpful! Before all of these was not so nicely integrated.

gcr 10y ago
But from what I can tell, it still doesn't look integrated. It looks like separate tools all glued on the same webpage. Now I have yet a separate terminal, a separate file manager, a separate Python REPL, ... How is this any better than just keeping those applications open and switching between them? I really hope this isn't just a poorly-implemented window manager inside a web browser.
- travisoliphant 10y ago
  For some plugins integration just means visual integration (i.e. window-manager integration). And, to be clear, this one is not poorly-implemented. It's based on a very powerful web-application toolkit called PhosphorJS written by very experienced UI developers. For other plugins there is very nice integration so that you can have interactions in one plugin update outputs in another and vice-versa. The ease of creating a connected ecosystem of web-applications connected to your workflow is one of the powerful aspects of the new platform. What would great integration look like to you?
  - gcr 10y ago
    I would want integration into my existing workflow, not integration that replaces my workflow with something completely different. For me, this means using emacs-ipython-notebook. This brings the ability to interact with a running IPython Kernel into the text editor that I use daily. It's the best of both worlds: interactive plotting+notebooks+rapid development assistance from IPython, and thanks to Emacs, all the keybindings are what I expect and I'm already familiar with how to extend the environment to suit my needs. EIN has been transformative for me. Perhaps my needs don't match the needs of others. For most people, I imagine the Jupyter-lab style of environment that's self-contained in a browser would in fact be a step up from the ad-hoc notepad++ windows and PuTTY sessions used previously.
- denfromufa 10y ago
  The layout is customizable - you keep what is needed

p4wnc6 10y ago

I agree, but would phrase it more like this: Jupyter has succeeded because each of the different major modes of interaction have been decoupled. If you just want to use it in a shell, you don't need to involve the browser at all. If you want a narrative format for sharing, presenting, or converting to slides, you can easily launch that environment. This feels like a big step backwards for me too. It's effectively like replicating the MATLAB / Octave / PyDev (Eclipse) sort of IDE-with-extras-plus-console that is so, so cripplingly bad, but acting like it's great and new just because it's all in the browser. If you're a fan of productivity, you shouldn't want to do that kind of stuff in a browser. Heck, I even disable all of the dropdown menus in Emacs because even that is too much of a productivity hindrance / inefficient use of monitor space when I am writing, reading, and thinking about code. This is one of those things where I feel that it doesn't actually solve practical use cases, doesn't make people more productive, but because there is a big hype engine behind it, it gets adopted and talked about anyway, and eventually becomes the sort of thing that an Office Space kind of manager starts to force you to use ... which really scares me. Stay off my lawn.

denfromufa 10y ago
Ipython terminal is still there, but you can't expect ma$$$ or Excel users to start working in vim or emacs based workflow. Besides browser provides rich output, cross-platform support. The big thing for me are widgets.
- p4wnc6 10y ago
  The returns to widgets in these kinds of things are soo diminishing. It looks cool the first 5 times, but then your boss says, "Great. Productionize it" and it's a world of shit. The widgets are good for throwaway demos and presentations, but if you are designing some functions for a business API of some kind, and your form of reporting is to show people something with a widget, now they want your internal API to include the widget, which is a dangerous game. I like super boring static charts, such that I can completely decouple the presentation from whatever business API was used behind the scenes for the data that's displayed. Take a super severe Occam's Razor approach to absolutely avoiding any kind of animation or interactive plot for as long as possible, and only if there is some kind of outrageously severe business case that absolutely demands it (which there almost never is) will I resort to designing a way to productionize the widget aspect of the report visualization as well. For this reason, I now see widgets like from IPython as a huge minefield. It looks pretty, but it's a big trap, and it doesn't add nearly the value everyone thinks it does. (Just to be clear, I'm only talking about the non-demoware case, which is the kind of case I have always worked with.)

nl 10y ago

Context: We use Jupyter heavily (mostly against Spark). In my experience there is a set of things that "traditional" Jupyter notebooks does really well. Anytime you have a linear flow of steps the notebook metaphor works really well. However, if you are doing things approaching traditional development, where you have multiple sources of data, or loops that require debugging, or basically anything that isn't linear in nature it doesn't work so well. I wouldn't want to lose traditional notebooks, but I'd love to be able to offer people something like this that offers better debugging and some development tool support, rather than jumping to a full desktop IDE.

bicubic 10y ago
My experience is in line with yours, debugging loops and functions is a big pain point. However, I think there's a much better solution to be had here, which is to add more powerful debugging capabilities to Notebook. I think Notebook has potential for new debugging paradigms, imagine for example being able to break anywhere in a cell and get a new 'forked cell' which operates in the context of the code that you just broke into. I think that's the better direction to go instead of reverting to the very interfaces and paradigms that we moved away from.
- nl 10y ago
  I'm not sure (And by that I don't mean I disagree: I'm genuinely unsure). To me, the traditional IDEs do work well for debugging and software development. Notebooks are great for explanatory examples and interactive experiments. I think these are different to the type of software development I do when I use an IDE. For example, I find notebooks great for rapid iteration of parameters when I'm doing "data science", or indeed most of the feature extraction->modelling->prediction data science pipeline. What I don't find them good for is developing new algorithms. It isn't clear to me if this is an inherit limitation of the notebook format, or just something where it needs new developments. (To be clear, I've also used both Zeppelin and Beaker notebooks and don't see any particular advantages. I've also used R Studio, but I don't really know enough R to comment sensibly on that)
- radiowave 10y ago