5 ms·

As much I try to like this project, still, something feels clunky. I am an avid user of Jupyter Notebooks for prototyping and code sharing in data science / dee

by stared 7y ago

As much I try to like this project, still, something feels clunky. I am an avid user of Jupyter Notebooks for prototyping and code sharing in data science / deep learning. Fir JavaScript (and D3.js in particular), still - I don't have a default go-to solutig. Gist -> blocks? GitHub and and a standalone page? JSFiddle or CodePen? Observable? (I would love to be convinced to one or the other approach.) My main concerns with observable are: 1. Unlike Jupyter notebook, the code is non-linear. Very often it refers to some pieces. 2. It takes some clicking to reveal the code. (Also, maybe a Jupyter habits, but I am used to results below code.) 3. I am still not sure what is the full syntax besides JavaScript and Markdown. 4. I am not sure if it is easy (or possible) to backup / self-host the code. (Pretty much a requirement to make sure a project won't disappear in a few years.) BTW: Once I gave Obserbavle a try, https://observablehq.com/@stared/tree-of-reddit-sex-life https://observablehq.com/@stared/tree-of-reddit-sex-life (and it got really popular here: https://news.ycombinator.com/item?id=19640562 https://news.ycombinator.com/item?id=19640562).

tolmasky 7y ago
Have you had a chance to try our project, https://runkit.com https://runkit.com? The two projects intersect a bit, but given your specific concerns it may be more along the lines of what you're looking for: 1. Specifically, we have gone through great pain-staking measures to make it just JavaScript and linear (while avoiding some of the pitfalls of Jupiter notebooks, like "losing state in cells"). You can read more about this specific issue here: https://blog.runkit.com/2015/09/10/time-traveling-in-node-js-notebooks/ https://blog.runkit.com/2015/09/10/time-traveling-in-node-js... 2. We are also huge believers in being able to own your code -- as such, every notebook has a download button that downloads a package with the entire dependency tree in a lock file so you can get exactly equivalent behavior. Again, this is only possible since we've started with the premise that RunKit notebooks just are Node modules. 3. Since it is just Node, every single module just works, including binary ones. In fact, every package on npm is built in -- no install, just require it.
shezi 7y ago
I haven't tried it myself, but there is a Javascript kernel for jupyter[1]. In fact, there is a huge number of kernels for jupyter now[2]. [1] https://github.com/n-riesco/ijavascript https://github.com/n-riesco/ijavascript [2] https://github.com/jupyter/jupyter/wiki/Jupyter-kernels https://github.com/jupyter/jupyter/wiki/Jupyter-kernels
jacobolus 7y ago
My own opinions: (1) Non-linear cell order (and in particular re-running all dependent cells anytime anything gets changed) is a huge improvement vs. mathematica/matlab/jupyter notebooks. Once you get used to it you will feel crippled when you try to go back. Being non-linear allows the author to optimize for whatever flow best matches the narrative, rather than ordering things based on arbitrary implementation-specific requirements of the programming environment. It is also possible to e.g. put simplified or brute-force implementations inline in the prose, and then move tricky optimized versions to technical appendices at the end of the document. Being non-linear also allows for showing some placeholder output right away and letting expensive-to-compute output (or slow-to-fetch data) execute after initial notebook load and progressively replace the placeholders. This lets readers get started reading the prose right away without needing to wait for slow operations. (2a) Being able to pin the source of a cell to be shown / hidden at the author’s discretion makes for an improved reader experience in the typical case (assuming a conscientious author). Nobody wants to read e.g. big blobs of redundant LaTeX source in addition to their mathematical formula output. Needing to sometimes click to show code that an author left hidden in the initial view is easy after just a tiny bit of experience as a reader. As an author I tend to mostly leave prose markup pinned closed and code cells pinned open (unless they contain too much distracting or overly technical detail) when publishing notebooks. (1 & 2a) These two features both change incentives for notebook authors: since code can be hidden in the default view authors are not afraid to use a better implementation which includes distracting technical details; in other notebook environments I have noticed authors intentionally use worse-performing but simpler implementations, and technical details often get moved to some external library, hiding the code much more deeply than just putting it in a cell with hidden source / moving the cell to the bottom of the document. (2b) I also prefer output below code, but the reason for the order here is so that showing/hiding the source won’t make the output jump up and down the page excessively; with a bit of experience the other order is not really an obstacle in practice. (3) It’s Javascript (with a few minor additions as shown in TFA), but you can use whatever javascript tagged template you want, including html, markdown, or anything else you can dream up. https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_literals#Tagged_templates https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe... (4) It is fairly easy, but it’s a fair complaint that it takes some trust / involves some risk to build work on top of a closed-source platform, especially for folks with strong free software ideology; even if it is possible to mirror code outside observablehq.com, it is at least mildly inconveient, and necessarily loses access to some great platform features. I personally have a lot of faith in the Observable team’s commitment to not breaking documents published on their platform or letting them disappear. But when business is involved, there’s no absolute guarantee.
billconan 7y ago
I'm making an alternative called epiphany, focusing on tech blogging, https://epiphany.pub/post?refId=2684bc94f9fcb9ffe637ebfbeba2af8c797c6ad9a66181026ee4bd3806b6f211 https://epiphany.pub/post?refId=2684bc94f9fcb9ffe637ebfbeba2... for 1. epiphany is also non-linear, because I'd like to reveal the results and then explain pieces. for 2. one click to the edit mode. for 3. epiphany uses vanilla js with some apis to print results and create ui. for 4. I try to use a plain text format, I also implemented pullrequest and versions like that of github. you could download all your data.
jashkenas 7y ago
With the gentle caveat that this list of concerns feels fairly low effort — surely, with a little elbow grease, you can think of much more damning complaints! ;) — I'll try to respond in good faith: 1. The non-linearity of the code is really the whole point here. Reactivity means that you wan work on any little piece of the code at any time, and you don't have to reload the page, or re-evaluate the entire notebook. It closes the feedback loop, and speeds up the process of iterating on your ideas immensely. 2. We’ve added more keyboard shortcuts, so you can now "Shift-A" to select every cell, and then press "P" to pin every cell open at once (and then to close them again). 3. The full syntax is just the "Observable JavaScript" described in the OP. The Markdown tagged template literal is part of the standard library, and you can define your own that work just like it. In that sense, the syntax is extensible. For example, here’s a different "pl" tagged template literal that runs Prolog programs: https://observablehq.com/@mbostock/hello-tau-prolog https://observablehq.com/@mbostock/hello-tau-prolog 4. Every notebook is available for download and self-hosting. You can download the code as an ES Module, or as a tarball that you can install with npm, or load and render on a web page: https://observablehq.com/@observablehq/downloading-and-embedding-notebooks https://observablehq.com/@observablehq/downloading-and-embed...
- fourthark 7y ago
  What's the rationale for results above code?
  - jashkenas 7y ago
    Great question! On Observable, each cell is a unit that can be evaluated independently, with two sides to it — you have the source code editor, and you have the rendered display (either a chunk of DOM or canvas that you’ve drawn, or an interactive inspector for JS values). Now, the important thing is that the code half of the cell is often collapsed. You navigate around the notebook, toggling open and closed the code editors, and typing into them. If the code is on top, the entire cell jumps when it opens and closes, pushing the rendered display up and down. If the code is on the bottom, it emerges from the half of the cell that is always there, and the display is stable. Ultimately, that’s why each cell has content before source code — the content is always there, and the code may or may not be visible, so working with the notebook as a whole feels much more stable and less jumpy with the rendered value as primary and the source code as secondary.
ww520 7y ago
1. Non-linear is a plus. Why force an order. Spreadsheet is non-linear and it works pretty well. 2. Clicking to reveal can be simply addressed with a global Expand-All toggle.

by stared 7y ago

tolmasky 7y ago

Have you had a chance to try our project, https://runkit.com https://runkit.com? The two projects intersect a bit, but given your specific concerns it may be more along the lines of what you're looking for: 1. Specifically, we have gone through great pain-staking measures to make it just JavaScript and linear (while avoiding some of the pitfalls of Jupiter notebooks, like "losing state in cells"). You can read more about this specific issue here: https://blog.runkit.com/2015/09/10/time-traveling-in-node-js-notebooks/ https://blog.runkit.com/2015/09/10/time-traveling-in-node-js... 2. We are also huge believers in being able to own your code -- as such, every notebook has a download button that downloads a package with the entire dependency tree in a lock file so you can get exactly equivalent behavior. Again, this is only possible since we've started with the premise that RunKit notebooks just are Node modules. 3. Since it is just Node, every single module just works, including binary ones. In fact, every package on npm is built in -- no install, just require it.

shezi 7y ago

I haven't tried it myself, but there is a Javascript kernel for jupyter[1]. In fact, there is a huge number of kernels for jupyter now[2]. [1] https://github.com/n-riesco/ijavascript https://github.com/n-riesco/ijavascript [2] https://github.com/jupyter/jupyter/wiki/Jupyter-kernels https://github.com/jupyter/jupyter/wiki/Jupyter-kernels

jacobolus 7y ago

My own opinions: (1) Non-linear cell order (and in particular re-running all dependent cells anytime anything gets changed) is a huge improvement vs. mathematica/matlab/jupyter notebooks. Once you get used to it you will feel crippled when you try to go back. Being non-linear allows the author to optimize for whatever flow best matches the narrative, rather than ordering things based on arbitrary implementation-specific requirements of the programming environment. It is also possible to e.g. put simplified or brute-force implementations inline in the prose, and then move tricky optimized versions to technical appendices at the end of the document. Being non-linear also allows for showing some placeholder output right away and letting expensive-to-compute output (or slow-to-fetch data) execute after initial notebook load and progressively replace the placeholders. This lets readers get started reading the prose right away without needing to wait for slow operations. (2a) Being able to pin the source of a cell to be shown / hidden at the author’s discretion makes for an improved reader experience in the typical case (assuming a conscientious author). Nobody wants to read e.g. big blobs of redundant LaTeX source in addition to their mathematical formula output. Needing to sometimes click to show code that an author left hidden in the initial view is easy after just a tiny bit of experience as a reader. As an author I tend to mostly leave prose markup pinned closed and code cells pinned open (unless they contain too much distracting or overly technical detail) when publishing notebooks. (1 & 2a) These two features both change incentives for notebook authors: since code can be hidden in the default view authors are not afraid to use a better implementation which includes distracting technical details; in other notebook environments I have noticed authors intentionally use worse-performing but simpler implementations, and technical details often get moved to some external library, hiding the code much more deeply than just putting it in a cell with hidden source / moving the cell to the bottom of the document. (2b) I also prefer output below code, but the reason for the order here is so that showing/hiding the source won’t make the output jump up and down the page excessively; with a bit of experience the other order is not really an obstacle in practice. (3) It’s Javascript (with a few minor additions as shown in TFA), but you can use whatever javascript tagged template you want, including html, markdown, or anything else you can dream up. https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_literals#Tagged_templates https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe... (4) It is fairly easy, but it’s a fair complaint that it takes some trust / involves some risk to build work on top of a closed-source platform, especially for folks with strong free software ideology; even if it is possible to mirror code outside observablehq.com, it is at least mildly inconveient, and necessarily loses access to some great platform features. I personally have a lot of faith in the Observable team’s commitment to not breaking documents published on their platform or letting them disappear. But when business is involved, there’s no absolute guarantee.

billconan 7y ago

I'm making an alternative called epiphany, focusing on tech blogging, https://epiphany.pub/post?refId=2684bc94f9fcb9ffe637ebfbeba2af8c797c6ad9a66181026ee4bd3806b6f211 https://epiphany.pub/post?refId=2684bc94f9fcb9ffe637ebfbeba2... for 1. epiphany is also non-linear, because I'd like to reveal the results and then explain pieces. for 2. one click to the edit mode. for 3. epiphany uses vanilla js with some apis to print results and create ui. for 4. I try to use a plain text format, I also implemented pullrequest and versions like that of github. you could download all your data.

jashkenas 7y ago

With the gentle caveat that this list of concerns feels fairly low effort — surely, with a little elbow grease, you can think of much more damning complaints! ;) — I'll try to respond in good faith: 1. The non-linearity of the code is really the whole point here. Reactivity means that you wan work on any little piece of the code at any time, and you don't have to reload the page, or re-evaluate the entire notebook. It closes the feedback loop, and speeds up the process of iterating on your ideas immensely. 2. We’ve added more keyboard shortcuts, so you can now "Shift-A" to select every cell, and then press "P" to pin every cell open at once (and then to close them again). 3. The full syntax is just the "Observable JavaScript" described in the OP. The Markdown tagged template literal is part of the standard library, and you can define your own that work just like it. In that sense, the syntax is extensible. For example, here’s a different "pl" tagged template literal that runs Prolog programs: https://observablehq.com/@mbostock/hello-tau-prolog https://observablehq.com/@mbostock/hello-tau-prolog 4. Every notebook is available for download and self-hosting. You can download the code as an ES Module, or as a tarball that you can install with npm, or load and render on a web page: https://observablehq.com/@observablehq/downloading-and-embedding-notebooks https://observablehq.com/@observablehq/downloading-and-embed...

fourthark 7y ago
What's the rationale for results above code?
- jashkenas 7y ago
  Great question! On Observable, each cell is a unit that can be evaluated independently, with two sides to it — you have the source code editor, and you have the rendered display (either a chunk of DOM or canvas that you’ve drawn, or an interactive inspector for JS values). Now, the important thing is that the code half of the cell is often collapsed. You navigate around the notebook, toggling open and closed the code editors, and typing into them. If the code is on top, the entire cell jumps when it opens and closes, pushing the rendered display up and down. If the code is on the bottom, it emerges from the half of the cell that is always there, and the display is stable. Ultimately, that’s why each cell has content before source code — the content is always there, and the code may or may not be visible, so working with the notebook as a whole feels much more stable and less jumpy with the rendered value as primary and the source code as secondary.

ww520 7y ago

1. Non-linear is a plus. Why force an order. Spreadsheet is non-linear and it works pretty well. 2. Clicking to reveal can be simply addressed with a global Expand-All toggle.