4 ms·
Yes, multi-language notebooks solve the "multi-language" but not the "large-scale" problems with bioinformatics (or data science) data analysis. However powerfu
by bpeng2000 9y ago
Yes, multi-language notebooks solve the "multi-language" but not the "large-scale" problems with bioinformatics (or data science) data analysis. However powerful other notebook environments can be, they are rather limited if they can only execute the notebooks on a single machine. However powerful other workflow systems can be, they are counterproductive if they require you to develop workflows in another environment and in another language. Backing up SoS Notebook with the SoS workflow engine provides a single environment for both interactive data analysis and the development and execution of workflows. This topic definitely worths a separate blog post so I will just list a few features that SoS enables here:
1. Extended from Python 3.6 to make SoS an easy and yet powerful workflow language.
2. Embedding workflows in SoS Notebooks allows you to annotate the workflows with detailed descriptions (markdown cells) and results (of demo runs).
3. Supports both forward (sequentially numbered) and makefile style (patten matching) workflows.
4. Execution signatures to avoid re-execution of long steps.
5. Magics to execute workflows in SoS Notebooks so you can, for example, execute cells of a notebook conditionally and repeatedly.
6. A task system that sends parts of workflows to remote host (a more powerful workstation), cluster (PBS/Torque/LFS/Slurm systems), or RQ task queue, even if the remote systems are on different file systems (SoS automatically maps paths and synchronize files).