11 ms·
Pachyderm was the first thing I thought of. In fact there should be opportunities for collaboration here. Pachyderm looks brilliant for high-scalability parall
by lobster_johnson 9y ago
Pachyderm was the first thing I thought of. In fact there should be opportunities for collaboration here.
Pachyderm looks brilliant for high-scalability parallel data processing, and the versioned data part is a way to not just maintain the history of the data, but also avoid reprocessing of data that hasn't changed since the previous run.
- jaz46 9y agoHi, I'm one of the creators of Pachyderm. We've been talking with the Quilt founders about various ways to work together and think there are some really exciting opportunities!