3 ms·
for a job where that workflow is sufficient, you probably shouldn't. But if you need an always-on, always-ready pipeline that can scale horizontally on an as-n
by claytonjy 8y ago
for a job where that workflow is sufficient, you probably shouldn't.
But if you need an always-on, always-ready pipeline that can scale horizontally on an as-needed basis, and perhaps mixes many different modules/languages, than Pachyderm might be a good fit.
- emkt 8y agoWell you can use https://github.com/argoproj/argo https://github.com/argoproj/argo for workflow needs without need to version everything. For data versioning if you need, you can use https://dvc.org/ https://dvc.org/ As this gives me freedom to choose as if I want data versioning and/or workflow rather than one bug bundle.