3 ms·

Jupyter system has three different subsystems that communicate via message passing: browser state, ZeroMQ and kernel process. Each system has its own leaky abst

Jupyter system has three different subsystems that communicate via message passing: browser state, ZeroMQ and kernel process. Each system has its own leaky abstraction. That is why the version controls and time travel debugging in the traditional sense become impossible. We shoud embrace this challenge and think beyond document based IDEs. For example, you can think of rendered notebooks as caching. I have even played with the concept of using rendered notebook as the input/data source of another notebook. This looks like a radical idea. But it is exactly how https://nbviewer.jupyter.org/ https://nbviewer.jupyter.org/ works.

TuringTest 8y ago
> the concept of using rendered notebook as the input/data source of another notebook Isn't that essentially what a spreadsheet does? Each table contains the results of a previous computation, intertwined with input and configuration data, and you just keep chaining more and more programs through a single unifying metaphor (the cell coordinate system plus function expressions). This model also gets you reactive recomputation for free. It's a powerful system, but quite old; I'd say the only radical thing is convincing classic developers that there's a better model than their beloved imperative, one-step-at-a-time idea of what running a program means.
- ontouchstart 8y ago
  Not exactly the same model as spreadsheets. In spreadsheets, a cell is a value in memory. The sheet is a graph of states. In Jupyter Notebook, the cells are nothing but messages. And the saved notebook files are JSON serialized byte stream of the message and metadata. It doesn’t keep the states, just data, a digital artifact.
  - TuringTest 8y ago
    Yet spreadsheet cells may also be formulas, which are akin to method calls / message passing / function application. The value of a cell with a formula is the memoized result of its latest function call execution. I don't see an essential difference with a notebook cell, other than the layout - grid vs linear. (Although I might be missing something; I don't have broad experience with Jupyter's computation model). Plus, spreadsheets may also be distributed.
    - ontouchstart 8y ago
      Good points. I don't care much about the visual layout of Jupyter notebook, so in this sense, it is not much different than spreadsheets. However, when I say Jupyter cells are nothing but messages to/from the kernel, it is profoundly different than spread sheets where cells can be message as you described. The same with distributed runtime states. Spreadsheets may also be distributed, but Jupyter notebook runtime is always distributed. You might have a half system like nbviewer that doesn't have a kernel running, but that is not a full Jupyter system. (I am distinguishing Jupyter runtime from the JSON byte stream artifact that we call a notebook file)
      - TuringTest 8y ago
        So, if I understand it right, Jupyter follows an agent-based model of computation, right? As opposed to spreadsheets, which follow a functional reactive paradigm. Yes, these are different computation models; although formally it is possible to transform mathematically one into the other, and vice-versa (in the same way that you can transform any computation model to a Turing machine or lambda calculus). It doesn't have any practical concerns, but it means that you can translate insights gained from one model to the other.
        ontouchstart 8y ago
        This thread is getting too deep and too narrow (visually) to read on HN. So I will summarize and stop here. 1. It is a good observation that Jupyter runtime follows an agent-based computation model instead of the MVC of smalltalk, or functional reactive Spreadsheets. Theoretically they are equivalent in the Turing sense. But this model opens huge opportunities that allow it to mimic how human brain and human societies work. It is all about messages, context containers and just-in-time computation. 2. This paradigm change also affects how we grow (instead of building) software architecture, which is the topic of the root of this HN submission. In the build/develop paradigm, software engineers focused on requirements, specification, foundation, frameworks, ..., i.e., the house building metaphor (https://en.wikipedia.org/wiki/Design_Patterns https://en.wikipedia.org/wiki/Design_Patterns ) But if you are growing a software project, there is no architecture. Then you should use agriculture metaphors: ecosystem, environment, energy cycle, water cycle, etc. It is about seeding, weeding, environment control, harvesting and sustainability. Push it a little bit further, programmers in this paradigm are more like software farmers than data plumbers in a data warehouse or code assemblers in a factory. It was a nice discussion in a virtual public square. I enjoyed it a lot. I hope you did as well.
- ontouchstart 8y ago
  Another difference is spreadsheet states are local, Jupyter Notebook states are distributed.

TuringTest 8y ago

> the concept of using rendered notebook as the input/data source of another notebook Isn't that essentially what a spreadsheet does? Each table contains the results of a previous computation, intertwined with input and configuration data, and you just keep chaining more and more programs through a single unifying metaphor (the cell coordinate system plus function expressions). This model also gets you reactive recomputation for free. It's a powerful system, but quite old; I'd say the only radical thing is convincing classic developers that there's a better model than their beloved imperative, one-step-at-a-time idea of what running a program means.

ontouchstart 8y ago
Not exactly the same model as spreadsheets. In spreadsheets, a cell is a value in memory. The sheet is a graph of states. In Jupyter Notebook, the cells are nothing but messages. And the saved notebook files are JSON serialized byte stream of the message and metadata. It doesn’t keep the states, just data, a digital artifact.
- TuringTest 8y ago
  Yet spreadsheet cells may also be formulas, which are akin to method calls / message passing / function application. The value of a cell with a formula is the memoized result of its latest function call execution. I don't see an essential difference with a notebook cell, other than the layout - grid vs linear. (Although I might be missing something; I don't have broad experience with Jupyter's computation model). Plus, spreadsheets may also be distributed.
  - ontouchstart 8y ago
    Good points. I don't care much about the visual layout of Jupyter notebook, so in this sense, it is not much different than spreadsheets. However, when I say Jupyter cells are nothing but messages to/from the kernel, it is profoundly different than spread sheets where cells can be message as you described. The same with distributed runtime states. Spreadsheets may also be distributed, but Jupyter notebook runtime is always distributed. You might have a half system like nbviewer that doesn't have a kernel running, but that is not a full Jupyter system. (I am distinguishing Jupyter runtime from the JSON byte stream artifact that we call a notebook file)
    - TuringTest 8y ago
      So, if I understand it right, Jupyter follows an agent-based model of computation, right? As opposed to spreadsheets, which follow a functional reactive paradigm. Yes, these are different computation models; although formally it is possible to transform mathematically one into the other, and vice-versa (in the same way that you can transform any computation model to a Turing machine or lambda calculus). It doesn't have any practical concerns, but it means that you can translate insights gained from one model to the other.
      - ontouchstart 8y ago
        This thread is getting too deep and too narrow (visually) to read on HN. So I will summarize and stop here. 1. It is a good observation that Jupyter runtime follows an agent-based computation model instead of the MVC of smalltalk, or functional reactive Spreadsheets. Theoretically they are equivalent in the Turing sense. But this model opens huge opportunities that allow it to mimic how human brain and human societies work. It is all about messages, context containers and just-in-time computation. 2. This paradigm change also affects how we grow (instead of building) software architecture, which is the topic of the root of this HN submission. In the build/develop paradigm, software engineers focused on requirements, specification, foundation, frameworks, ..., i.e., the house building metaphor (https://en.wikipedia.org/wiki/Design_Patterns https://en.wikipedia.org/wiki/Design_Patterns ) But if you are growing a software project, there is no architecture. Then you should use agriculture metaphors: ecosystem, environment, energy cycle, water cycle, etc. It is about seeding, weeding, environment control, harvesting and sustainability. Push it a little bit further, programmers in this paradigm are more like software farmers than data plumbers in a data warehouse or code assemblers in a factory. It was a nice discussion in a virtual public square. I enjoyed it a lot. I hope you did as well.
ontouchstart 8y ago
Another difference is spreadsheet states are local, Jupyter Notebook states are distributed.