3 ms·

> Link blocks to highlight dependencies, Pyflow will then automatically run your blocks in the correct order This is a problem. If you require users to define

> Link blocks to highlight dependencies, Pyflow will then automatically run your blocks in the correct order This is a problem. If you require users to define dependencies for you, they will make mistakes and things will go wrong in ways that are incredibly difficult to debug. This is already a huge pain point with Jupyter notebooks, and they're only one-dimensional! Have you considered going the route of Observable notebooks [1]? In an Observable notebook, each cell can export at most one variable. This means that the notebook can automatically determine dependencies statically, run cells in the correct order, and even show the user a viz of dependencies. [1] https://observablehq.com/ https://observablehq.com/

brrrrrm 5y ago
Isn't this readily solved by a user convention such as only updating one variable each cell? Why limit users based on mistakes they haven't made yet?
- phailhaus 5y ago
  User conventions don't work, because there is no mechanism for enforcement other than "hope they remember!" I would say most users don't even know about this convention, given I work in an environment where notebooks are used heavily and have never heard of this. Because these mistakes are allowed to happen silently, they will happen, and it will almost certainly make the code harder to manage. On top of that, Pyflow seems to rely on the user to define dependencies between blocks as well. This means that the user would have to both perfectly maintain the convention of one-variable-per-block, but also link every block that touches that variable going forward. Without ever making a mistake. It's such a well-defined graph problem that it's practically made for computers to handle, not fleshy people. This is why tools are so important: they can impose constraints that allow you to act more freely elsewhere. When I change a cell in Observable or add a reference, I don't have to think. The engine handles dependencies for me, without fail. I can't make an entire class of errors, which means I can just focus on functionality.
mrtranscendence 5y ago
Wait, what is it about exporting only one variable at a time that makes it possible to determine dependencies statically? Why not two variables? Or three? If you're determining dependencies based on variable names it seems like that would be possible.
- phailhaus 5y ago
  You're right, it's more due to the fact that Observable requires you to explicitly export variables. However, limiting it to one has a couple advantages: 1. You don't recalculate other variables if only one needs to update. For example, if a cell exports two variables, and a dependency of one of them updates, you're forced to re-evaluate the whole cell. 2. I think the UX plays out simpler if you limit it to one variable. It basically reads like `myVar = {...}` at the top of the cell. It's a bit trickier if you try to support exporting multiple variables at once. 3. If you find yourself needing to tie several values together, you can still manage it by exporting a single dictionary value!

brrrrrm 5y ago

Isn't this readily solved by a user convention such as only updating one variable each cell? Why limit users based on mistakes they haven't made yet?

phailhaus 5y ago
User conventions don't work, because there is no mechanism for enforcement other than "hope they remember!" I would say most users don't even know about this convention, given I work in an environment where notebooks are used heavily and have never heard of this. Because these mistakes are allowed to happen silently, they will happen, and it will almost certainly make the code harder to manage. On top of that, Pyflow seems to rely on the user to define dependencies between blocks as well. This means that the user would have to both perfectly maintain the convention of one-variable-per-block, but also link every block that touches that variable going forward. Without ever making a mistake. It's such a well-defined graph problem that it's practically made for computers to handle, not fleshy people. This is why tools are so important: they can impose constraints that allow you to act more freely elsewhere. When I change a cell in Observable or add a reference, I don't have to think. The engine handles dependencies for me, without fail. I can't make an entire class of errors, which means I can just focus on functionality.

mrtranscendence 5y ago

Wait, what is it about exporting only one variable at a time that makes it possible to determine dependencies statically? Why not two variables? Or three? If you're determining dependencies based on variable names it seems like that would be possible.

phailhaus 5y ago
You're right, it's more due to the fact that Observable requires you to explicitly export variables. However, limiting it to one has a couple advantages: 1. You don't recalculate other variables if only one needs to update. For example, if a cell exports two variables, and a dependency of one of them updates, you're forced to re-evaluate the whole cell. 2. I think the UX plays out simpler if you limit it to one variable. It basically reads like `myVar = {...}` at the top of the cell. It's a bit trickier if you try to support exporting multiple variables at once. 3. If you find yourself needing to tie several values together, you can still manage it by exporting a single dictionary value!