3 ms·
I'm using Python for lab automation, so my data comes fresh from an experiment. I've found that keeping stuff in memory is extremely convenient until the kernel
by analog31 9y ago
I'm using Python for lab automation, so my data comes fresh from an experiment. I've found that keeping stuff in memory is extremely convenient until the kernel shuts down for some reason (e.g., I inadvertently kill it while forgetting that I've left a notebook open).
Still, I love having my data collection scripts documented right there with the subsequent analysis. So, I've disciplined myself to handle experimental data in one of two ways:
* For "small" data, format it as a Python thing (list, dict, whatever is appropriate), and paste it into the next cell as an input. I haven't found a way to do this automatically, and I'm careful not to make things too automatic lest I run a cell and over-write old data.
* For "big" data, dump it to a file. I just turn the system time into a filename, to avoid over-writing an old file.
I don't think I've come up with the last word, on using Jupyter as a self data collecting lab notebook, nor am I yet 100% certain that it's even a good idea. This is a work in progress, but much better than anything else I've ever tried. For complicated experiments, I still create stand alone Python programs to control things.