8 ms·
Show HN: ZenML, extensible open-source MLOps framework for reproducible ML
- ipsum2 6y agoThe API looks fairly brittle, e.g. manually defining the loss and activation inside the pipeline instead of in the model itself. Have you or your customers used this in a large production environment?
- benkoller 6y agoHi, one of the co-founders here. We’ve been using the original base of ZenML for a few years in its previous shapes, mostly in predictive analytics projects. We obviously had to switch things up to make the code base suitable to a broader audience and are still working out kinks. Especially feedback like yours is super appreciated, as we can gauge opinions and evaluate them against our roadmap, so thanks :).
- p1esk 6y agoI’m curious, why would you want to define the loss in the model and not in the trainer?
- justicezyx 6y agoAre the team of bringing ZenML funded by VC investment? I ask because it seems the project just launched fully without a long development process in open (it appears, I could be wrong).
- benkoller 6y agoHi! We are fully bootstrapped, and ZenML is the final result of a longer journey. When we started over three years ago we were running predictive analytics projects with industry partners. Over time our focus expanded, until we finally realized: our tech stack is much more valuable. We did a lot of talking to people to find out if there is a niche for our perspective, and have been pushing the code base to where it is today for the last months. So, to answer in short: there is a longer development cycle behind this, but not in the open - and no VC backing.
- olavolav 6y agoSpeaking as a pytorch user, many of the steps in your Readme example remind me of the usual setup except the pipeline.run() handoff which is replaced by eager evaluation in pytorch. Are you seeing something like an eager mode for your library, or perhaps a pytorch plugin that might use your apis?
- benkoller 6y agoDefinitely! It might sound like a sales-thing to say, but PyTorch in general is a huge point on our roadmap, but we’re not 100% sure what the most logical approach is. To keep the APIs consistent will involve some fiddling on our end.
- p1esk 6y agoTF is dying. Does your framework work with pytorch and jax?
- benkoller 6y agoInteresting - out of curiosity, why do you thing the TF ecosystem is dying? ZenML unfortunately does not yet work with PyTorch, but it’s one of the biggest points for our roadmap next year.
- p1esk 6y ago3 reasons: 1. Pytorch is getting better at deployment. The only reason to use TF until recently was its superior deployment capabilities. 2. Google itself finally realized what a clusterfuck TF has been, threw in the towel, and started fresh with Jax. 3. Pytorch is king in research. ML is still very much a research driven field, so whatever researchers choose is going to win sooner or later. Jax is a serious contender though. Google invested a lot in TF development, and the inertia will keep it alive for a couple more years, but the writing on the wall is clear.