22 ms·
The Inherent Limitations of GPT-3
- leobg 5y agoThe biggest problem to me seems to be confabulation: 9 out of 10 answers may be correct. But then the model will give you an answer that is completely, utterly made up. This even happens when you specifically instruct the model to only chose from a set of given facts. I tried to build essentially a knowledge retrieval system on top of GPT-3: I‘d have a corpus of text. The user would ask a question. A separate semantic search engine would pick out the top 5 or so most likely answer candidates from the corpus. And GPT-3‘s job would have been to select the right answer and to rephrase it as an answer. And GPT-3, it seems to me, cannot do this. Because as soon as you give it enough flexibility to select or transform sentences, it will also start to bring up „knowledge“ from its internal parameters, i.e. to confabulate (or „hallucinate“, as OpenAI calls it). I believe the fundamental reason for this is that the model has no concept of „a fact“. It just fills in gaps, based on patterns it has seen before. That is the source of its ability to impress us with its „human like“ answers. And it is, at the same time, the reason why you can never factually trust its output. I like to joke that this is why, as far as I know, pretty much all commercial applications of GPT-3 seem to be clustering around marketing. Where making stuff up is pretty much part of the job description, and factual accuracy is not. That being said, it is still an impressive piece of technology. And if pattern-completion, rather than factual reasoning, is what you are after, then it can also be a great labor saving device. Things like transforming sentences from active to passive voice are a piece of cake, and work quite well even on the smaller models. For tasks like that, I find GPT-3 much more reliable than, say, MTurk.