3 ms·
I've wondered if one could train a LLM on a closed set of curated knowledge. Then include training data that models the behaviour of not knowing. To the point t
by philipswood 3y ago
I've wondered if one could train a LLM on a closed set of curated knowledge. Then include training data that models the behaviour of not knowing. To the point that it could generalize to being able to represent its own not knowing.
Because expecting a behaviour, like knowing you don't know, that isn't represented in the training set is silly.
Kids make stuff up at first, then we correct them - so they have a way to learn not to.
- aidenn0 3y ago> I've wondered if one could train a LLM on a closed set of curated knowledge. Then include training data that models the behaviour of not knowing. To the point that it could generalize to being able to represent its own not knowing. The problem is that curating data is slow and expensive and downloading the entire web is fast and cheap. See also https://en.wikipedia.org/wiki/Cyc https://en.wikipedia.org/wiki/Cyc
- philipswood 3y agoAgreed. Using LLM to generate or curate training sets for other generations seems like a cool approach. Maybe if you trained a small base model to know it doesn't know in general and THEN trained it on the entire web with embedded not-knowing preserving training examples, it would work?
- philipswood 3y agoReminded of this approach where a tiny model was trained children's stories generated by a larger model: https://www.quantamagazine.org/tiny-language-models-thrive-with-gpt-4-as-a-teacher-20231005/ https://www.quantamagazine.org/tiny-language-models-thrive-w...
- crooked-v 3y ago> train a LLM a closed set of curated knowledge Google has one of these already, with an LLM that was trained on nothing but weather data and so can only give weather-data-prediction responses. The 'knowing it doesn't know things' part is much harder to get reliable, though.
- intended 3y agoNo it won’t work. This is not a brain. The best analogy is an English major. They are good at language, not reasoning. Humans see language and think reason. It seems we can’t separate the two.