3 ms·
I recently discovered that joblib can do something similar, both on disk and in memory: https://joblib.readthedocs.io/en/latest/memory.html https://joblib.readt
by bravura 6y ago
I recently discovered that joblib can do something similar, both on disk and in memory: https://joblib.readthedocs.io/en/latest/memory.html https://joblib.readthedocs.io/en/latest/memory.html
- thechao 6y agoThis memoizes closures of functions, and lets them be executed on other processors? Does it support dependency tracking between memoized closures (incremental recomputation), or do I have to roll that, myself?
- trombonechamp 6y agoUnfortunately you need to roll that yourself in joblib. But it is automatic in bionic (https://github.com/square/bionic https://github.com/square/bionic), assuming your workflow can be structured in the way bionic likes.
- thechao 6y agoBionic is what I'm looking for (right on their use-case page): a Make replacement that can work with intermediate computations, and not just files.