4 ms·

Out of curiosity, was Kubeflow ever considered and was there a deliberate decision against Kubeflow? Kubeflow Pipelines under the hood (and not very hidden) us

by verst 5y ago

Out of curiosity, was Kubeflow ever considered and was there a deliberate decision against Kubeflow? Kubeflow Pipelines under the hood (and not very hidden) uses Argo Workflows.

RocketSyntax 5y ago
Or Apache Airflow. I'd expect this and KFP to have better integrations that cater to data science. You can use Elyra extension to design the workflows right from JupyterLab.
- FridgeSeal 5y ago
  Not from said company, but did do a workflow tool comparison at my work and we also went with Argo. Argo had better kubernetes + surrounding ecosystem integration out of the box, it was designed to run containers by default which suited us because we had mixed language workloads. Airflow was mostly Python specific, unless you then ran plugins and extensions, the config/pipeline definition was written in Python which I didn’t want to do after witnessing my teammates write the worst Python I’ve seen in my career, and last time I evaluated it, it depended on a bunch of external, Python specific tools (celery etc) that I had previously found painful to run.
  - verst 5y ago
    > my teammates write the worst Python I’ve seen in my career Isn't that fairly common which is why there are "ML engineers" that then productionize (clean up and optimize) the original code to be plugged into a production workflow / pipeline system?
KidComputer 5y ago
I found Kubeflow Pipelines to be worse than vanilla Argo. Not only is the documentation poor for the python DSL that gets compiled to the Argo workflow spec but some operations are downright impossible to express in the DSL. Vanilla Argo + RBAC works wonderfully especially with Argo Server since v3+.
- verst 5y ago
  I agree. For anything sophisticated you have to use the Kubernetes Python client combined with the KFP DSL. The compiled Argo Workflow spec can become such a mess that you quickly run into a size limitations -- especially if you choose to deploy a Python function directly (which then uses Kaniko to build the containers in cluster and run them via Argo Workflows). At that point I would rather use Argo directly. And KFP is generally quite behind in Argo Workflow versions. And it's not like KFP makes CI/CD (MLOps) any easier than Argo Workflows itself.

by verst 5y ago

Out of curiosity, was Kubeflow ever considered and was there a deliberate decision against Kubeflow? Kubeflow Pipelines under the hood (and not very hidden) uses Argo Workflows.

RocketSyntax 5y ago

Or Apache Airflow. I'd expect this and KFP to have better integrations that cater to data science. You can use Elyra extension to design the workflows right from JupyterLab.

FridgeSeal 5y ago
Not from said company, but did do a workflow tool comparison at my work and we also went with Argo. Argo had better kubernetes + surrounding ecosystem integration out of the box, it was designed to run containers by default which suited us because we had mixed language workloads. Airflow was mostly Python specific, unless you then ran plugins and extensions, the config/pipeline definition was written in Python which I didn’t want to do after witnessing my teammates write the worst Python I’ve seen in my career, and last time I evaluated it, it depended on a bunch of external, Python specific tools (celery etc) that I had previously found painful to run.
- verst 5y ago
  > my teammates write the worst Python I’ve seen in my career Isn't that fairly common which is why there are "ML engineers" that then productionize (clean up and optimize) the original code to be plugged into a production workflow / pipeline system?

KidComputer 5y ago

I found Kubeflow Pipelines to be worse than vanilla Argo. Not only is the documentation poor for the python DSL that gets compiled to the Argo workflow spec but some operations are downright impossible to express in the DSL. Vanilla Argo + RBAC works wonderfully especially with Argo Server since v3+.

verst 5y ago
I agree. For anything sophisticated you have to use the Kubernetes Python client combined with the KFP DSL. The compiled Argo Workflow spec can become such a mess that you quickly run into a size limitations -- especially if you choose to deploy a Python function directly (which then uses Kaniko to build the containers in cluster and run them via Argo Workflows). At that point I would rather use Argo directly. And KFP is generally quite behind in Argo Workflow versions. And it's not like KFP makes CI/CD (MLOps) any easier than Argo Workflows itself.