4 ms·
Jupyter notebooks are ultimately just a text file, which can be version controlled, verified (via the VCS or independent hashing), and if necessary - digitally
by dunpeal 8y ago
Jupyter notebooks are ultimately just a text file, which can be version controlled, verified (via the VCS or independent hashing), and if necessary - digitally signed in a cryptographically secure manner.
Given that Git is already secure, I'd say all the researcher needs to know is basic usage of Git. Version controlling the notebook satisfies all the requirements you mentioned: prevents accidental distortion, is verifiable, restorable, etc.
- ashtonbaker 8y agoIt's not just tracability and security that's the issue, though. It's an issue of reproducibility. When I give you a notebook, how do you 'run' it to verify my calculations? You can't just press "run", because I might have executed the cells out of order, or deleted cells which later cells depended on. Version control doesn't solve this problem. Good practice might solve it, but it turns out that researchers are stunningly bad at following a strict top-to-bottom execution of notebooks. And who can blame them? The non-linearity of notebooks is one of their features. But when it comes to reproducibility and just the basic mechanics of sharing code, we definitely need a better solution.
- 21 8y agoYou forget that a typical Python Jupyter notebook will import a dozen Python packages. Versioning that it's still a major problem in the ecosystem.
- dunpeal 8y ago> Versioning that it's still a major problem in the ecosystem. There's actually a perfectly good way to solve this problem: a virtual environment with a requirements.txt file.
- deleted 8y ago[deleted]
- LyndsySimon 8y agoFor research in particular, I'd suggest a Pipfile.lock. That will ensure that the packages installed match exactly, not just by version number.