4 ms·
Re Dockerfiles: Right now, we need to rebuild your docker images from time to time (e.g. when we make some changes to the kernel). That means that if you create
by Equiet 6y ago
Re Dockerfiles: Right now, we need to rebuild your docker images from time to time (e.g. when we make some changes to the kernel). That means that if you create a docker image with `RUN pip install numpy` and we need to rebuild it in a year, you might get a different version which might break things. The correct solution here is to encourage users to always use `RUN pip install numpy==1.19.3`. We already do this in Python cells (when you run `!pip install numpy` we query PyPI and suggest the last version to you), but we haven't added it to Dockerfiles yet. So to set the expectations we have this notice in the docs.
Regarding other issues: We currently record every execution in project history. That means even if you run cells out of order, you can still get a list of commands that shows how you got to the current state.
The next step for us is to start subtly notifying users when they are doing something that could be an issue later down the road (for example executing cells out of order). We already built this, but decided not to ship it yet because it needed more love. The second thing we are working on is interactive/reacting execution. This is very very very cool and brings the experience from the notebook to the next level (at least for me), but needs much more testing.
Reproducibility vs flexibility (in the sense of letting the user do whatever they want if they know what they're doing) is a difficult problem. In the end, it's going to be a combination of friendly nudges and much better experience if users are following the "reproducible" path. However, we never want to limit users in what they are able to do.
I spent a lot of time thinking about this and would be happy to chat about what you're thinking. Feel free to email me at jakub@deepnote.com.
- setgree 6y agoThanks so much for the thoughtful reply! I will follow up when I've had a deeper dive into the product.