3 ms·

This is very interesting! I will have look into this more. Two quick comments: How does this compare to Arvados (http://arvados.org http://arvados.org)? You

by tetron 12y ago

This is very interesting! I will have look into this more. Two quick comments: How does this compare to Arvados (http://arvados.org http://arvados.org)? You might be interested in the common workflow language effort which among other goals is developing a multi-vendor standard for wrapping analysis tools using Docker (https://github.com/common-workflow-language/common-workflow-language https://github.com/common-workflow-language/common-workflow-...)

jdoliner 12y ago
Hi, Just read up on Arvados. I'm totally new to it but it seems really cool. I think pfs and Arvados are very philosophically aligned. Arvados seems to believe as we do that the most important thing missing from data science right now is collaboration. I didn't have the time to fully grok their architecture but it seems like there are some differences there. In particular I don't know if Arvados has git-like semantics. It does on the other hand have a ton of cool features that pfs doesn't have yet such as content addressable storage and support for in memory databases that help speed up computations (I think that's what they do at least.) Definitely a very interesting project and hopefully good ideas can flow between the ecosystems. The common workflow language effort is definitely very interesting as well. It looks like it's bioinformatic specific right now but we're planning to have something similar but more generic in the near future and I'll definitely be looking to cwi for inspiration. Thanks so much for the links they're both very interesting.