3 ms·
Maybe even more powerful would be reducing the number of examples needed to learn, eg less than one shot https://www.technologyreview.com/2020/10/16/1010566/ai
by joshribakoff 4y ago
Maybe even more powerful would be reducing the number of examples needed to learn, eg less than one shot
https://www.technologyreview.com/2020/10/16/1010566/ai-machine-learning-with-tiny-data/ https://www.technologyreview.com/2020/10/16/1010566/ai-machi...
updating weights in real time is useless if each update basically does nothing because it takes an insurmountable amount of training, on the other hand if i can give my model a succinct “lesson” i’d then be very willing to wait a while for it to “process”
- dTal 4y agoAs I understand, that's basically what fine-tuning is?
- monkeydust 4y agoWhat sort of performance difference can you expect from in context Vs fine tuning, any relevant papers on this ?
- vintermann 4y agoYeah, that's why just updating the weights on the models such as they are doesn't work. But they're right that it's desirable to have some sort of online learning, whether on top of a frozen language model, or through some not yet invented way to do it end to end.