4 ms·

Hi! I’m David Aronchick, PM on Kubeflow, I’m happy to answer any questions! I was one of the early PMs on Kubernetes, and we very much want to make this a commu

Hi! I’m David Aronchick, PM on Kubeflow, I’m happy to answer any questions! I was one of the early PMs on Kubernetes, and we very much want to make this a community project, so please join us in thinking about what’s next! - GH: https://GitHub.com/Google/Kubeflow https://GitHub.com/Google/Kubeflow - kubeflow-Discuss: https://groups.google.com/forum/m/#!forum/kubeflow-discuss https://groups.google.com/forum/m/#!forum/kubeflow-discuss NOTE: The name "flow" does not refer directly to TensorFlow; if anything it's a nod at all the river themes that pop up in the ML community (eg FBLearner Flow) Disclosure: I work at Google on Kubeflow

pacala 9y ago
Great to hear this not tied to TensorFlow! How would one use a different DL platform, say PyTorch or DyNet?
- TheIronYuppie 9y ago
  The steps would basically be: - Containerize the DL platform - Create a k8s manifest (similar to our CRD if necessary) - Create a service endpoint - integrate all that into the JH deployment This is less hard than it sounds, but we'd love help! We only started with TF because that's what we know. Disclosure: I work at Google on Kubeflow
  - minerals 9y ago
    Interesting, though I don't see how it is better than a plain docker image over kubernetes? Not much of a hassle now too. And how is it different from what DL4J is already doing with Zeppelin and supporting both Keras, TF, MXNet and PyTorch on the way?
    - bonesss 9y ago
      > ...how [is it] better than a plain docker image over kubernetes? Scalability for people with existing on-premise (or cloud based), kubernetes workflows, especially once it comes to training or heavy crunching. That's not to say that Docker Machine/Swarm/Compose couldn't handle the same, but it's an extra step for kubernetes users and pushes people onto a slightly different toolchain than minikube->K8s.
      - TheIronYuppie 9y ago
        Correct! Many folks have more complicated deployments in the cloud, and we're trying to align (as close as humanly possible) your on-prem stack with your cloud stack, to minimize the pain in migration. If you have a single container, and a simple pipeline, this may be a bit more than you need. We've just found that there are normally 5 or more services/systems that people wire together to create an ML stack, and that's what we're trying to solve for/simplify. Disclosure: I work at Google on Kubeflow
ethbro 9y ago
Kudos on the release, and thanks for your and the team's work! Any thoughts on this vs managed ml-engine? Cost-aside, seems like this nibbles on the smaller scale "but ml tooling is too hard" use cases?
- TheIronYuppie 9y ago
  Thank you! We still love Google Cloud ML engine - it's perfect for those who want to run in the cloud and want a layer of abstraction. This is for people who want portable stacks and a bit more control; and/or want to use their Kubernetes deployments (particularly on-premise or multi-purpose). Does that help? Disclosure: I work at Googlr on Kubeflow
  - ecnahc515 9y ago
    Do you foresee some kind of integration between Google Cloud ML engine and Tensorflow on k8s in the future?
    - TheIronYuppie 9y ago
      We're always ready to talk roadmap - anything in particular you'd like to see integration-wise? Disclosure: I work at Google on Kubeflow
      - ethbro 9y ago
        Off the top of my head, maybe a maintained "ml-engine aligned" kubeflow setup, to the extent that's possible. The use case I'm think of is an ml dev team building on kubeflow and proving a system. Then wanting to transfer it to a non-engineering team, yet wash their hands of any ongoing infrastructure ops responsibility. Knowing that a "ml-engine aligned" kubeflow config would transfer cleanly (including associated bells and whistles) would make that a much more attractive option. Caveat: I'll admit I'm not keeping up on what's in the managed offering, but I'm assuming there are a number of value-adds of the type that end users like (visualizations, etc).
colek42 9y ago
Are you at kubecon this week?
- TheIronYuppie 9y ago
  I am! You can reach me at aronchick (at) Google if you'd like to meet or have questions.

pacala 9y ago

Great to hear this not tied to TensorFlow! How would one use a different DL platform, say PyTorch or DyNet?

TheIronYuppie 9y ago
The steps would basically be: - Containerize the DL platform - Create a k8s manifest (similar to our CRD if necessary) - Create a service endpoint - integrate all that into the JH deployment This is less hard than it sounds, but we'd love help! We only started with TF because that's what we know. Disclosure: I work at Google on Kubeflow
- minerals 9y ago
  Interesting, though I don't see how it is better than a plain docker image over kubernetes? Not much of a hassle now too. And how is it different from what DL4J is already doing with Zeppelin and supporting both Keras, TF, MXNet and PyTorch on the way?
  - bonesss 9y ago
    > ...how [is it] better than a plain docker image over kubernetes? Scalability for people with existing on-premise (or cloud based), kubernetes workflows, especially once it comes to training or heavy crunching. That's not to say that Docker Machine/Swarm/Compose couldn't handle the same, but it's an extra step for kubernetes users and pushes people onto a slightly different toolchain than minikube->K8s.
    - TheIronYuppie 9y ago
      Correct! Many folks have more complicated deployments in the cloud, and we're trying to align (as close as humanly possible) your on-prem stack with your cloud stack, to minimize the pain in migration. If you have a single container, and a simple pipeline, this may be a bit more than you need. We've just found that there are normally 5 or more services/systems that people wire together to create an ML stack, and that's what we're trying to solve for/simplify. Disclosure: I work at Google on Kubeflow

ethbro 9y ago

Kudos on the release, and thanks for your and the team's work! Any thoughts on this vs managed ml-engine? Cost-aside, seems like this nibbles on the smaller scale "but ml tooling is too hard" use cases?

TheIronYuppie 9y ago
Thank you! We still love Google Cloud ML engine - it's perfect for those who want to run in the cloud and want a layer of abstraction. This is for people who want portable stacks and a bit more control; and/or want to use their Kubernetes deployments (particularly on-premise or multi-purpose). Does that help? Disclosure: I work at Googlr on Kubeflow
- ecnahc515 9y ago
  Do you foresee some kind of integration between Google Cloud ML engine and Tensorflow on k8s in the future?
  - TheIronYuppie 9y ago
    We're always ready to talk roadmap - anything in particular you'd like to see integration-wise? Disclosure: I work at Google on Kubeflow
    - ethbro 9y ago
      Off the top of my head, maybe a maintained "ml-engine aligned" kubeflow setup, to the extent that's possible. The use case I'm think of is an ml dev team building on kubeflow and proving a system. Then wanting to transfer it to a non-engineering team, yet wash their hands of any ongoing infrastructure ops responsibility. Knowing that a "ml-engine aligned" kubeflow config would transfer cleanly (including associated bells and whistles) would make that a much more attractive option. Caveat: I'll admit I'm not keeping up on what's in the managed offering, but I'm assuming there are a number of value-adds of the type that end users like (visualizations, etc).

colek42 9y ago

Are you at kubecon this week?

TheIronYuppie 9y ago
I am! You can reach me at aronchick (at) Google if you'd like to meet or have questions.