12 ms·

I think this is the Alan Kay future of computing. Right now we're in this weird hybrid state where we still work with digital documents primarily using the phys

I think this is the Alan Kay future of computing. Right now we're in this weird hybrid state where we still work with digital documents primarily using the physical paper interface. Imagine digital academic "papers" in STEM fields that natively ran the simulations the paper was describing. Jupyter sort of delivers that, but it still feels like early days for interactive digital-first documents (or as Steve Jobs has been credited for saying, "bicycles of the mind").

asdff 6y ago
Why compute twice? Waste of resources. Some simulations also demand some serious hardware requirements that might not even be possible to run locally.
- karlicoss 6y ago
  While a good point, at the moment the balance is much more shifted towards dead media rather than wasted resources. At best, the document doesn't get as much engagement as it could. At worst you get non-reproducible research papers, when you're really lucky if you can find the code in open access and compile it, let alone get the same results. And sure, some simulations are very heavy, but they are more of exceptions. Also possible to have the best of both worlds, and have both a simulation, and a static snapshot available.
  - dllthomas 6y ago
    > At worst you get non-reproducible research papers Ability to rerun programs is great, but we should be careful to remember that it's a different thing than reproducibility.
    - colechristensen 6y ago
      Often it’s the first step to reproducibility though. Am enormous amount of scientific effort is figuring out how a researcher did something they published.
      - dmix 6y ago
        Basically adding another whole project on top of the other project this way. Imagine trying to figure out some 2001 JS paper thing for ex. But applied to every generation of technical development. There’s always standards of course but we’ve seen those go sideways enough time to make one cringe at the thought of ‘dynamic papers’ via some new medium. The kind of thing that sounds amazing on the surface then you remember the sort of crazy IT depts that thousands of universities run and forget the whole thing.
        deleted 6y ago
        [deleted]
      - fourthark 6y ago
        Today was the first time I encountered a paper with a Docker image. Fantastic to be able to try it out with no effort. I suppose this only works in a few fields though.
      - dllthomas 6y ago
        > Often it’s the first step to reproducibility though. It shouldn't be! Reproduction needs to involve the interaction of human brain meats with a human level description of the solution. This is how we make sure that people aren't talking about something different than what was actually done, and how we make sure our conclusions are robust against the things we've failed to specify. Imagine saying the same thing for physics: I start replication by running a time machine and using the same apparatus as the original experiment under the same conditions. Impracticality aside, this would be potentially useful to suss out fraud and certain kinds of errors, but what successful replication tells is is manifestly less powerful than successful replication on a new apparatus in a new location at a new time, with new values for everything we've failed to control.
- mlyle 6y ago
  Not everything is resource constrained, though. Imagine being able to easily make interactive content that illustrates what you're trying to convey and allows the user to "play with it." For things that are heavily resource constrained, it still could be a boon to have interactive access to the data that comes out of it.
- elihu 6y ago
  Even if it's not practical to re-run all of the computation, in many cases it would be nice to have the output data stored in the document in a form where you can interact with it rather than just having static pixels.
- Siira 6y ago
  It’s possible to also include the results, so no dilemma there. (I think current notebook formats already do this.)
karlicoss 6y ago
Even for non-academic reporting: imagine if instead of 'dead' news articles or some tax reforms, or climate change, or whatever, you had an interactive model you could play with (and for example, plug in your own numbers if you disagree with some of the inputs).
- tasogare 6y ago
  Good news: the software you describe exists since 1985.
  - mlyle 6y ago
    I think now we have a lot of things like this-- we have Jupyter, Matlab, etc, to create engineer-centric general purpose interactive documents. We have labor-heavy ways to make end-user focused ones in the browser. We have spreadsheets. But-- wouldn't it be cool if there was a way ordinary people could create interactive content to interact with data in a rich, intuitive way?
    - localhost 6y ago
      Why can’t ordinary people use Jupyter? Or put another way, what’s missing from Jupyter that would get ordinary people to use it?
      - BlueTemplar 6y ago
        How can I send a Jupyter page as a standalone, offline document ?
      - indigochill 6y ago
        My feelings about this aspect of Jupyter are two-fold: 1. On the creation side, it requires someone be comfortable with Python (or other Jupyter language) to some degree. Right now, programming is still considered a career skill rather than something "ordinary people" should be expected to know. Perhaps layering a graphical programming interface on top of this, which UE4 seems to have had some success with with their Blueprint system, would get "ordinary people" over the mental hurdle of being intimidated by code-as-text. Just look at the mental gymnastics people will engage with in Excel while thinking it's not programming. I see this as more of a social problem than a technical one, at any rate. 2. Once you build an interactive Jupyter document (especially if you use interactive widgets), it's not necessarily that easy to share in its original state without requiring the reader also have a Jupyter environment set up or access a server running Jupyter. I would like to be able to share the document in a way that can be accessed offline by someone without them needing to set up the whole environment. Maybe an "Adobe Reader"-like application for Jupyter notebooks that "ordinary people" can just install with a click?
mizzao 6y ago
This already exists, with a focus on machine learning: https://distill.pub/ https://distill.pub/
BlueTemplar 6y ago
I would already settle for non-obsolete animation support : - GIF is obsolete (~100x heavier than MP4 in my use-case, so out of the question) - MP4 has poor support in PDF readers (- Besides, PDF is not appropriate for electronic documents.) - EPUB doesn't seem to support MP4 at all (- EPUB does support PNG, not sure about APNG, will have to try it out...) - MHTML=EML support has been dropped from browsers, which is completely baffling to me. There are alternatives like SingleFile, but they feel like dirty hacks : https://addons.mozilla.org/en-US/firefox/addon/single-file/ https://addons.mozilla.org/en-US/firefox/addon/single-file/ - What future for AV1 support ?
- scrollaway 6y ago
  MHTML is a neat format, it's unfortunate it never got much steam. I think it could have been more popular if web browsers had defaulted to it when saving pages, rather than this weird html + _files/ directory (which on Windows is mysteriously linked so that when you delete one, you delete the other - no idea how they do that!). What I've read of EPUB is also pretty disappointing. Seeing as it's a compiled format, once again, instead of going the zipfile + bunch of html inside + specific layout, we could have had a subset of html in .mhtml.gz with, like, metadata in a <script type="application/json" id="x-epub-metadata">. And then, guess what, web browsers could have been able to read it natively…
  - iggldiggl 6y ago
    > (which on Windows is mysteriously linked so that when you delete one, you delete the other - no idea how they do that!) Probably just some special code in Windows Explorer watching for the combo of .htm(l) file plus simlarly-named folder – via the command line I can delete just the HTML file or the folder separately without problems.
- rwbaskette 6y ago
  I’ve been surprised with what you can accomplish with data URIs. Embedding a MP4 can work great, but your text editor will likely hate it.

asdff 6y ago

Why compute twice? Waste of resources. Some simulations also demand some serious hardware requirements that might not even be possible to run locally.

karlicoss 6y ago
While a good point, at the moment the balance is much more shifted towards dead media rather than wasted resources. At best, the document doesn't get as much engagement as it could. At worst you get non-reproducible research papers, when you're really lucky if you can find the code in open access and compile it, let alone get the same results. And sure, some simulations are very heavy, but they are more of exceptions. Also possible to have the best of both worlds, and have both a simulation, and a static snapshot available.
- dllthomas 6y ago
  > At worst you get non-reproducible research papers Ability to rerun programs is great, but we should be careful to remember that it's a different thing than reproducibility.
  - colechristensen 6y ago
    Often it’s the first step to reproducibility though. Am enormous amount of scientific effort is figuring out how a researcher did something they published.
    - dmix 6y ago
      Basically adding another whole project on top of the other project this way. Imagine trying to figure out some 2001 JS paper thing for ex. But applied to every generation of technical development. There’s always standards of course but we’ve seen those go sideways enough time to make one cringe at the thought of ‘dynamic papers’ via some new medium. The kind of thing that sounds amazing on the surface then you remember the sort of crazy IT depts that thousands of universities run and forget the whole thing.
      - deleted 6y ago
        [deleted]
    - fourthark 6y ago
      Today was the first time I encountered a paper with a Docker image. Fantastic to be able to try it out with no effort. I suppose this only works in a few fields though.
    - dllthomas 6y ago
      > Often it’s the first step to reproducibility though. It shouldn't be! Reproduction needs to involve the interaction of human brain meats with a human level description of the solution. This is how we make sure that people aren't talking about something different than what was actually done, and how we make sure our conclusions are robust against the things we've failed to specify. Imagine saying the same thing for physics: I start replication by running a time machine and using the same apparatus as the original experiment under the same conditions. Impracticality aside, this would be potentially useful to suss out fraud and certain kinds of errors, but what successful replication tells is is manifestly less powerful than successful replication on a new apparatus in a new location at a new time, with new values for everything we've failed to control.
mlyle 6y ago
Not everything is resource constrained, though. Imagine being able to easily make interactive content that illustrates what you're trying to convey and allows the user to "play with it." For things that are heavily resource constrained, it still could be a boon to have interactive access to the data that comes out of it.
elihu 6y ago
Even if it's not practical to re-run all of the computation, in many cases it would be nice to have the output data stored in the document in a form where you can interact with it rather than just having static pixels.
Siira 6y ago
It’s possible to also include the results, so no dilemma there. (I think current notebook formats already do this.)

karlicoss 6y ago

Even for non-academic reporting: imagine if instead of 'dead' news articles or some tax reforms, or climate change, or whatever, you had an interactive model you could play with (and for example, plug in your own numbers if you disagree with some of the inputs).

tasogare 6y ago
Good news: the software you describe exists since 1985.
- mlyle 6y ago
  I think now we have a lot of things like this-- we have Jupyter, Matlab, etc, to create engineer-centric general purpose interactive documents. We have labor-heavy ways to make end-user focused ones in the browser. We have spreadsheets. But-- wouldn't it be cool if there was a way ordinary people could create interactive content to interact with data in a rich, intuitive way?
  - localhost 6y ago
    Why can’t ordinary people use Jupyter? Or put another way, what’s missing from Jupyter that would get ordinary people to use it?
    - BlueTemplar 6y ago
      How can I send a Jupyter page as a standalone, offline document ?
    - indigochill 6y ago
      My feelings about this aspect of Jupyter are two-fold: 1. On the creation side, it requires someone be comfortable with Python (or other Jupyter language) to some degree. Right now, programming is still considered a career skill rather than something "ordinary people" should be expected to know. Perhaps layering a graphical programming interface on top of this, which UE4 seems to have had some success with with their Blueprint system, would get "ordinary people" over the mental hurdle of being intimidated by code-as-text. Just look at the mental gymnastics people will engage with in Excel while thinking it's not programming. I see this as more of a social problem than a technical one, at any rate. 2. Once you build an interactive Jupyter document (especially if you use interactive widgets), it's not necessarily that easy to share in its original state without requiring the reader also have a Jupyter environment set up or access a server running Jupyter. I would like to be able to share the document in a way that can be accessed offline by someone without them needing to set up the whole environment. Maybe an "Adobe Reader"-like application for Jupyter notebooks that "ordinary people" can just install with a click?

mizzao 6y ago

This already exists, with a focus on machine learning: https://distill.pub/ https://distill.pub/

BlueTemplar 6y ago

I would already settle for non-obsolete animation support : - GIF is obsolete (~100x heavier than MP4 in my use-case, so out of the question) - MP4 has poor support in PDF readers (- Besides, PDF is not appropriate for electronic documents.) - EPUB doesn't seem to support MP4 at all (- EPUB does support PNG, not sure about APNG, will have to try it out...) - MHTML=EML support has been dropped from browsers, which is completely baffling to me. There are alternatives like SingleFile, but they feel like dirty hacks : https://addons.mozilla.org/en-US/firefox/addon/single-file/ https://addons.mozilla.org/en-US/firefox/addon/single-file/ - What future for AV1 support ?

scrollaway 6y ago
MHTML is a neat format, it's unfortunate it never got much steam. I think it could have been more popular if web browsers had defaulted to it when saving pages, rather than this weird html + _files/ directory (which on Windows is mysteriously linked so that when you delete one, you delete the other - no idea how they do that!). What I've read of EPUB is also pretty disappointing. Seeing as it's a compiled format, once again, instead of going the zipfile + bunch of html inside + specific layout, we could have had a subset of html in .mhtml.gz with, like, metadata in a <script type="application/json" id="x-epub-metadata">. And then, guess what, web browsers could have been able to read it natively…
- iggldiggl 6y ago
  > (which on Windows is mysteriously linked so that when you delete one, you delete the other - no idea how they do that!) Probably just some special code in Windows Explorer watching for the combo of .htm(l) file plus simlarly-named folder – via the command line I can delete just the HTML file or the folder separately without problems.
rwbaskette 6y ago
I’ve been surprised with what you can accomplish with data URIs. Embedding a MP4 can work great, but your text editor will likely hate it.