4 ms·
This is like FB's FBLearner, which I've used a little bit. This is a platform, so this is not at at the SKL/Tensorflow/Caffe level, this is something which say
by Maro 9y ago
This is like FB's FBLearner, which I've used a little bit.
This is a platform, so this is not at at the SKL/Tensorflow/Caffe level, this is something which says: put your data in our data warehouse in this format, tell me what model you want to run on it, go have a coffee, and by the time you're back I'll magically run your model on a big cluster quickly; if you want I can also try out different hyperparams/models, and I'll give you standard plots/metrics to evaluate the thing. You can also use 1000s of existing feature vectors/metrics that other people have put in the DWH. You can also clone an existing model and tweak that. If you're happy with the model, you can deploy, either on my API by querying and I'll tell you the prediction, or I can publish the model in binary form to a datastore and you can just import it in your iOS code in 3 lines. You can also schedule nightly re-trains, I'll notify you if something big changes in the accuracy metrics, etc.
The point is, this sort of ML platform allows engineers at big companies to leverage existing data/metrics/f.vectors, models, infra, etc. and allows them to move 10-100x faster than startups/hobbyists. When I say 10-100x, I actually mean it, I'm also running ML models at home, and the amount of time I waste on this glue stuff is huge (starting from data prep to f---ing around with tensor ranks).
I'm sure there are already startups working on platforms like this for everybody, we'll just have to wait for one of those to become good.
- MasterScrat 9y ago> I'm sure there are already startups working on platforms like this for everybody Does anyone knows of any such open-source product actually usable today? This would solve a real need. (Also I'd love to work on such a thing)
- Kpourdeilami 9y agoI've been working on something very similar to that. It allows you to connect a SQL database, S3 bucket, etc. then write a query for the data that you want to train the model on. It takes a couple minutes and then it'll expose the trained model over a REST endpoint. It also has feedback loops to fine tune the model on real-time data and has an option to store all the predictions made by model into Redshift or S3 to be visualized on a tool like Tableau. The tool itself is not open-source but it can be installed on the user's AWS, Azure, or GCP account to keep their models and data private. It's still in alpha testing (bunch of pilots) and I'll probably do a "Show HN" post when it is ready for a public beta
- Maro 9y agoThings to look at: H2O, Amazon ML Platform, Azure ML, Google Cloud Actually, in this case I'd bet on the big cloud providers to eventually deliver sth nice (or acquire and integrate).
- MasterScrat 9y agoI would prefer self-hosted, or at least without vendor lock-in. Looks like H2O would fit the bill. Anyone has experience with pipeline.ai?