5 ms·
What are the current open source alternatives to MLflow?
by denzil_correa 8y ago
What are the current open source alternatives to MLflow?
- agnokapathetic 8y agohttps://www.mlflow.org https://www.mlflow.org is open source.
- denzil_correa 8y agoI understand. However, there could be more than one open source ML platforms. Hence, open source alternatives to MLFlow doesn't imply MLFlow isn't open source.
- deleted 8y ago[deleted]
- mmq 8y agoPolyaxon, https://github.com/polyaxon/polyaxon https://github.com/polyaxon/polyaxon, is an alternative project to MLFlow that is also open source. disclaimer: I am the author of Polyaxon.
- syllogism 8y agoI've been using Polyaxon to train models for spaCy. It uses Kubernetes, so it's easy to set it up to work very efficiently on Google Compute Engine. For instance, you can set it to use pre-emptible instances, which makes the experiments very cheap.
- brylie 8y agoAny plans to support scikit?
- mmq 8y agoYou can use scikit-learn, or any other library, to train your models.
- hcrisp 8y agoSacred (https://github.com/IDSIA/sacred https://github.com/IDSIA/sacred) FGLab (https://kaixhin.github.io/FGLab/ https://kaixhin.github.io/FGLab/) Metricmachine (https://github.com/danielwaterworth/metricmachine https://github.com/danielwaterworth/metricmachine) Non-open source: Neptune (http://neptune.ml http://neptune.ml) Aetros (https://aetros.com/trainer https://aetros.com/trainer)
- __bee 8y agoHow about SageMaker, Can we include it in this list. I played with SageMaker sometime ago and it helps you build a whole pipeline to host your models, in addition to host your notebook and bridge the gap between data scientists and data engineers.
- brylie 8y agoAnecdotally, we considered using the hosted versions of Jupyter and Apache Zeppelin that are part of AWS SageMaker and EMR. We couldn't figure out a simple/familiar workflow for keeping the notebooks under version control. So, we agreed to run the notebooks locally, use a familiar Git-based workflow, and interact with the AWS infrastructure through the local notebook instances.
- garysieling 8y agoDoes Zeppelin work naturally with git? I've been struggling to get the right setup with just Jupyter
- brylie 8y agoWell, good question. The file format for Jupyter is not ideal for 'code craftsmanship', as pointed out by another comment. There are utilities to strip out some of the metadata from the Jupyter files, such as rendered output and run counters, but that is a trade-off to be decided by your team: https://github.com/kynan/nbstripout https://github.com/kynan/nbstripout
- 8y ago
- TheIronYuppie 8y agoI would imagine Kubeflow (https://github.com/kubeflow/kubeflow https://github.com/kubeflow/kubeflow) would be complimentary (e.g. run MLFlow on top of it) - they claim platform neutrality. Disclosure: I work at Google on Kubeflow.
- ddutta 8y agoYes, Kubeflow is a vey promising platform for ml lifecycle management on kubernetes. The combination of kubernetes, istio and kubeflow could enable other higher layer workflow tools (mlflow, h2o etc). This space is early.
- xianghonglee 8y agothe mlflow now suport ml lifecycly management?I think we can find a way to combinate kubeflow and mlflow
- daveguy 8y agohttps://pachyderm.io https://pachyderm.io is a data science infrastructure framework that has both open source and enterprise versions.
- allisterb 8y agoClassifyBot https://github.com/allisterb/ClassifyBot https://github.com/allisterb/ClassifyBot Designed to automate and make repeatable different stages in classification pipelines. Written in .NET but agnostic about language or framework. Embeds a Python interpreter and can interface with Java or R. Disclaimer: I am a walrus
- shakedown1 8y agoA couple more alternatives: H2o: https://www.h2o.ai/h2o/ https://www.h2o.ai/h2o/ Data Robot: https://www.datarobot.com/ https://www.datarobot.com/
- marcuniq 8y agoRiseML seems to be another alternative: https://riseml.com/ https://riseml.com/ Not open source though
- phltech 8y agoAlso not open source, but looks similar to what ParallelM does http://www.parallelm.com http://www.parallelm.com