5 ms·
>But in the future, I think ML will look more like a tower of transfer learning. You'll have a sequence of models, each of which specializes the previous model,
by fouc 8y ago
>But in the future, I think ML will look more like a tower of transfer learning. You'll have a sequence of models, each of which specializes the previous model, which was trained on a more general task with more data available.
He's almost describing a future where we might buy/license pre-trained models from Google/Facebook/etc that are trained on huge datasets, and then extend that with more specific training from other sources of data in order to end up with a model suited to the problem being solved.
It also sounds like we can feed the model's learnings back into new models with new architectures as well as we discover better approaches later.
- mlucy 8y ago> He's almost describing a future where we might buy/license pre-trained models from Google/Facebook/etc that are trained on huge datasets, and then extend that with more specific training from other sources of data in order to end up with a model suited to the problem being solved. Yup, that's basically it. (Although I think there might be more than two parties involved; I think probably there will be one giant pretrained image model that everyone in the world starts from, then someone will specialize it for some domain, then someone will specialize that for some subdomain, all the way down to an individual person's problem, which might only have a few thousand data points.)
- gipp 8y agoNot sure if you were just being cheeky, but this is pretty much exactly what GCP's AutoML offerings are.
- XuMiao 8y agoWhat do you think of life-long learning scenario that models are trained incrementally forever? For example, I train a model with 1000 examples, it sucks. The next guy pick it up and train a new one by putting a regularizer over mine. It might still suck. But after maybe 1000 people, the model begins to get significantly better. Now, I will pickup what I left and improve it by leveraging the current best. This continues forever. Imagine that this community is supported by a block chain. We won't be relying on big companies any more eventually.
- jacquesm 8y agoWhat is it with the word 'blockchain' that will make people toss it into otherwise completely unrelated text?
- oehpr 8y agonothing, they're describing a series of content addressable blocks that link back to their ancestors. Which is a good application of a block chain. Think IPFS. It's not cryptocurrency. Though cryptocurrency definitely popularized the technique.
- fwip 8y agoIPFS isn't a blockchain just like git isn't a blockchain. "Blockchain" has semantic meaning that "a chain of blocks" does not.
- deleted 8y ago[deleted]
- SiempreViernes 8y agoThat recent period in time when it was a license to print money?
- Varcht 8y agoYou know, we love our overloaded terms? In this context it means "Decentralized storage". Keeps people on their toes, keeps the AI guessing.
- Terr_ 8y ago> What do you think of life-long learning scenario that models are trained incrementally forever? The same as the "life-long" coding scenario where monoliths are tweaked incrementally forever. They may have niches but they'll kinda suck, because the underlying problem-space evolves too. Code loses value with age.
- deleted 8y ago[deleted]