3 ms·
Isn't the fundamental issue that it doesn't have any way to tell if what it thinks it knows is or isn't true? This article sounds like an idea I had independen
by ben_w 3y ago
Isn't the fundamental issue that it doesn't have any way to tell if what it thinks it knows is or isn't true?
This article sounds like an idea I had independent not too long ago, but with a different goal:
LLMs are great at natural language comprehension, but also have a lot of neurons dedicated to factoids. Using neurons that way is really inefficient, can we split the "language" capability from the "knowledge" capability and have the former just look things up in a database?
My question was more about reducing the size of the network rather than reducing hallucinations, but it's still a separately updatable knowledge resource.
(The answer may actually be "no"; I don't study this professionally, but technical jargon is kinda both factual domain knowledge and also linguistic comprehension, which is why Oracle isn't competing with Starbucks for Java beans).
- RyEgswuCsn 3y agoMany classic statistical modelling techniques have ways to produce some measure of confidence for their predictions; perhaps LLMs could incorporate that as well, e.g. assign probabilities(/perplexity?) to each of the tokens they generate. > LLMs are great at natural language comprehension, but also have a lot of neurons dedicated to factoids. Using neurons that way is really inefficient, can we split the "language" capability from the "knowledge" capability and have the former just look things up in a database? I think the beauty of LLMs is exactly that all we need to do is to feed them raw text --- the hope, I guess, had been that the models will be able to develop human-like insights by learning to understand and "speak" languages on its own. Introducing "feature engineering" (e.g. the distinction as you suggested) would defeat that goal.
- IanCal 3y ago> perhaps LLMs could incorporate that as well, e.g. assign probabilities(/perplexity?) to each of the tokens they generate. They do, that's how tokens are selected. Locally run models or non-chat ones from openai can return the probabilities and you can do things like modify or filter them.
- RyEgswuCsn 3y agoI was thinking more of the entropy of the distribution over the distribution sort of thing --- one may say a model is uncertain about its statements if all its potential variants are equally probable and vice versa.
- cubefox 3y agoI don't think that's possible in the current autocomplete based paradigm. But the LLM can be made to ignore most of its own knowledge. For example, Bing answers most questions by performing an Internet search, even if it could have answered without one. (Sometimes this makes it actually worse, e.g. when it is asked to solve a puzzle, and it gives answers to similar ones from the web, without trying to solve it itself.)