3 ms·

Being able to learn within context, without updating weights is amazing. Imagine how much more efficient and/or powerful it could be if we found a way to update

Being able to learn within context, without updating weights is amazing. Imagine how much more efficient and/or powerful it could be if we found a way to update the weights in real time.

joshribakoff 4y ago
Maybe even more powerful would be reducing the number of examples needed to learn, eg less than one shot https://www.technologyreview.com/2020/10/16/1010566/ai-machine-learning-with-tiny-data/ https://www.technologyreview.com/2020/10/16/1010566/ai-machi... updating weights in real time is useless if each update basically does nothing because it takes an insurmountable amount of training, on the other hand if i can give my model a succinct “lesson” i’d then be very willing to wait a while for it to “process”
- dTal 4y ago
  As I understand, that's basically what fine-tuning is?
  - monkeydust 4y ago
    What sort of performance difference can you expect from in context Vs fine tuning, any relevant papers on this ?
- vintermann 4y ago
  Yeah, that's why just updating the weights on the models such as they are doesn't work. But they're right that it's desirable to have some sort of online learning, whether on top of a frozen language model, or through some not yet invented way to do it end to end.
Enginerrrd 4y ago
Yeah I've been thinking about that a lot within the context of self-driving cars. What if we could take the rare but significant examples of model confusion, and then... explain to the model what it did wrong just as we would to a human. If it could then incorporate that information into future weights, that would be game changer in a LOT of fields where it's been hard to bridge the gap.
- jurgemaister 4y ago
  Tesla does this with shadow testing[1] [1]https://www.forbes.com/sites/bradtempleton/2019/04/29/teslas-shadow-testing-offers-a-useful-advantage-on-the-biggest-problem-in-robocars/?sh=2acfc3bd3c06 https://www.forbes.com/sites/bradtempleton/2019/04/29/teslas...

Being able to learn within context, without updating weights is amazing. Imagine how much more efficient and/or powerful it could be if we found a way to update the weights in real time.

joshribakoff 4y ago

Maybe even more powerful would be reducing the number of examples needed to learn, eg less than one shot https://www.technologyreview.com/2020/10/16/1010566/ai-machine-learning-with-tiny-data/ https://www.technologyreview.com/2020/10/16/1010566/ai-machi... updating weights in real time is useless if each update basically does nothing because it takes an insurmountable amount of training, on the other hand if i can give my model a succinct “lesson” i’d then be very willing to wait a while for it to “process”

dTal 4y ago
As I understand, that's basically what fine-tuning is?
- monkeydust 4y ago
  What sort of performance difference can you expect from in context Vs fine tuning, any relevant papers on this ?
vintermann 4y ago
Yeah, that's why just updating the weights on the models such as they are doesn't work. But they're right that it's desirable to have some sort of online learning, whether on top of a frozen language model, or through some not yet invented way to do it end to end.

Enginerrrd 4y ago

Yeah I've been thinking about that a lot within the context of self-driving cars. What if we could take the rare but significant examples of model confusion, and then... explain to the model what it did wrong just as we would to a human. If it could then incorporate that information into future weights, that would be game changer in a LOT of fields where it's been hard to bridge the gap.

jurgemaister 4y ago
Tesla does this with shadow testing[1] [1]https://www.forbes.com/sites/bradtempleton/2019/04/29/teslas-shadow-testing-offers-a-useful-advantage-on-the-biggest-problem-in-robocars/?sh=2acfc3bd3c06 https://www.forbes.com/sites/bradtempleton/2019/04/29/teslas...