5 ms·
I think they fundamentally json is just the wrong format for these files. Speaking from (ancient and limited) experience I made a little notebook-style interpre
by benjaminjackman 8y ago
I think they fundamentally json is just the wrong format for these files. Speaking from (ancient and limited) experience I made a little notebook-style interpreter for learning scala back in 2009 or so called scalide. It saved its files ("scalapads") to XML. XML actually worked better in some ways since most of the code could live between the tags unescaped (sans < > &) so it merged / diffed the user code well. The meta-level stuff (cell boundaries etc) needed by the notebook ... not so much.
In json the code has to be escaped into strings, and json is really finicky about syntax (e.g. no trailing commas). So it doesn't work well.
I never got the chance to redo it, however the solution I was leaning to for my post "I won the lottery, I can work on fun stuff" attempt was to store the meta-code in a version of the host language(s), with some simple syntax that could live comfortably in the comments of various different languages to do things like encode the cell divisions and so on.
Basically something like:
#notebook[lang=python]
#cell[lang=python]
def add(x,y):
return x + y
#endcell
//notebook[lang=scala]
//cell[lang=python]
def add(x: Int, y: Int) = x + y
//endcell
This I think would be beneficial for a couple of reasons.
1. Better diffing / merging.
2. One click toggle between show source and view as notebook mode, which would really allow this to work in an IDE like vscode pretty seamlessly. The cells become something akin to //#regions in the IDE. But at the end of the day you are still editing a source code file, so you can edit the whole file easily.
3. The keyboard shortcuts for executing and jumping between cells would generally work in raw code mode, so you could just edit there continuously and manually writing out //cell //endcell. Also the executing results could appear in block comments inline in the editor, off to the side, or in a popup above, the code you are editing.
4. The IDEs could uprender the comment-syntax into cells as they gained better support for the paradigm (similar to how they do for code folding / syntax higlighting already).
5. Eventually, perhaps a cross language, metasyntax could be established to make things a bit more concrete than magic comments (get ready for some serious bikeshed painting though!)
The closest I have seen anything come in this regard is Quokka however it's not quite all the way there.
- TeMPOraL 8y agoThe closest thing I've seen to what you described would be... Emacs. It actually uses the "metadata in file-specific comments" paradigm. You can put file-local values for Emacs variables in comments at the top or bottom of your file, like described in [0]. Your example could be rewritten as: # -*- notebook-lang: python -*- or // -*- notebook-lang: scala -*- Still, the usual way of using Emacs for "interactive notebooks" is via org-mode, which is a better Markdown with support for (among other things) executing code blocks straight in the org document you're writing. This way, Emacs support all your points 1 to 5, and is generally more powerful than Jupyter or other similar things, but it also means you can kiss any kind of collaboration goodbye. For some weird reason, the more powerful a tool, the less likely it is other people will be using it. -- [0] - https://www.gnu.org/software/emacs/manual/html_node/emacs/Specifying-File-Variables.html#Specifying-File-Variables https://www.gnu.org/software/emacs/manual/html_node/emacs/Sp...
- nemoniac 8y agoWell, the reason is not so weird. Generally speaking, the more powerful the tool, the higher the bar. More time and effort is required to learn it and become proficient with it. When it comes to the very powerful tools, few will have the aptitude or be prepared to put in the effort to learn them. For those who don't, less powerful tools take their place and proliferate. As you say, emacs checks all the boxes but the majority is not prepared to learn it and prefers to program throught their browser.
- btown 8y agoAlong the same lines, I would love to see a syntax something like this: mynotebook.py ### (cell boundary) """Top-level unused strings (docstring-esque) rendered as markdown""" def add(x, y): return x + y # jupyter-output-hash: 0123abc (which would link to some external key-value storage for the project) Anything in something other than the primary language could be in something like `execute_scala(""" scala code """)` - which would execute properly given proper globals. As long as the output-hash storage is treated as append-only and is highly available (output cells could even be encrypted for security if this was a public cloud service, or you could even use a local or shared filesystem), then this file would not only parse and run as a perfectly valid Python file, but it would also hold references to outputs in a source-control friendly way. IDEs could show the cell outputs inline. If you rerun your notebook and get different outputs for some reason, `git diff` tells you exactly where things changed without being too messy. Basically, put outputs in off-chain storage, and just be a literate code file. I feel like this would address most people's needs, no?