5 ms·

>If it did have a real understanding, prompt construction wouldn't matter as much This is only true if we assume GPT was never trained on satire or intentional

>If it did have a real understanding, prompt construction wouldn't matter as much This is only true if we assume GPT was never trained on satire or intentionally absurd text. But there's no reason to think this. Because it continues a bad prompt in an absurd or comical way does not demonstrate it doesn't "understand" common facts. If you treat GPT as a conversation bot and expect it to call you out when you give it an absurd prompt, then it is your expectations that are wrong.

ClumsyPilot 6y ago
> if continues a bad prompt in an absurd or comical way does not demonstrate it doesn't "understand" common facts. Well then you can justify it outputting anything at all.
- dodobirdlord 6y ago
  This is why it’s fair to call some prompt designs bad. With a bad prompt you can’t really judge anything from the response. But better prompt design can uncover interesting behaviors. GPT-3 will generally not reject a prompt as nonsense or unreasonable, and will instead do a straight-faced continuation, probably because it has been trained on plenty of jokes, satires, dream sequences, etc. But if you specifically give it space in the prompt to reject nonsense, it’s actually quite good at rejecting nonsense. https://arr.am/2020/07/25/gpt-3-uncertainty-prompts https://arr.am/2020/07/25/gpt-3-uncertainty-prompts
  - ClumsyPilot 6y ago
    I have read your link, but think this is an unfair argument. We are not talking about nonsense, we are talking about unexpected input, which will happen in any real-world situation. It might be text about something a kid, or depressed person has done. The text about stirring with a cigar is firmly withing the realm of plausibility. And what happens is that GPT seemingly goes off the rails.
- im3w1l 6y ago
  It's an autocompleter. If you want an answer to question, you have to make the answer the obvious (to a human) completion. If you want it to continue in a serious tone, you have to make sure the prompt is serious.
csande17 6y ago
I can't wait for GPT-3 to be used in a production application and this to be the explanation for why it fails. "Responding to a customer inquiry with 'noone cares, go away' wasn't really a failure on the part of the model. Rather, the model was simply creating a performance-art piece commenting on the way capitalism drives an emotional wedge between 'providers' and 'consumers'. Try fine-tuning on some economics journals to get that out of its system."
mannykannot 6y ago
I think the point here is that it hs no understanding of what you are calling a 'bad' prompt. That does not mean that it lacks any understanding whatsoever (personally, I think it has made a small step in that direction), but it is a valid case of it not understanding something. Furthermore, I do not think we can assume that if it were trained on examples of satire or intentionally absurd text, it would perform better on such prompts - in fact, I would not be surprised if its performance would deteriorate on many prompts, both straightforward and tricky ones, if given such training. Now I am wondering if you need a theory of mind before you can begin to understand satire...
- IanAWP 6y ago
  GPT is both general purpose and stateless. Lacking another mechanism, it's the requirement of the prompt to request a particular genre of response if that's what you want. There can be no such thing as a bad prompt from gpt3's perspective. A bad prompt is one where the user has a specific purpose which is not expressed. It's bad because you know beforehand that gpt3 can not align with it. Someone pours grape juice into a bottle and becomes worried that is not safe to drink. GPT3 correctly grasps that there is a hidden context to this weird prompt, however when given no other information it guesses that this hidden context is something known only to the hypothetical character in the prompt. I would probably do this too. When you give it the correct context (this weird prompt is a logic test) then it gives you the answer you expected.
  - mannykannot 6y ago
    > GPT3 correctly grasps that there is a hidden context to this weird prompt. That is interesting. How does it indicate its understanding that there is a hidden context?

ClumsyPilot 6y ago

> if continues a bad prompt in an absurd or comical way does not demonstrate it doesn't "understand" common facts. Well then you can justify it outputting anything at all.

dodobirdlord 6y ago
This is why it’s fair to call some prompt designs bad. With a bad prompt you can’t really judge anything from the response. But better prompt design can uncover interesting behaviors. GPT-3 will generally not reject a prompt as nonsense or unreasonable, and will instead do a straight-faced continuation, probably because it has been trained on plenty of jokes, satires, dream sequences, etc. But if you specifically give it space in the prompt to reject nonsense, it’s actually quite good at rejecting nonsense. https://arr.am/2020/07/25/gpt-3-uncertainty-prompts https://arr.am/2020/07/25/gpt-3-uncertainty-prompts
- ClumsyPilot 6y ago
  I have read your link, but think this is an unfair argument. We are not talking about nonsense, we are talking about unexpected input, which will happen in any real-world situation. It might be text about something a kid, or depressed person has done. The text about stirring with a cigar is firmly withing the realm of plausibility. And what happens is that GPT seemingly goes off the rails.
im3w1l 6y ago
It's an autocompleter. If you want an answer to question, you have to make the answer the obvious (to a human) completion. If you want it to continue in a serious tone, you have to make sure the prompt is serious.

csande17 6y ago

I can't wait for GPT-3 to be used in a production application and this to be the explanation for why it fails. "Responding to a customer inquiry with 'noone cares, go away' wasn't really a failure on the part of the model. Rather, the model was simply creating a performance-art piece commenting on the way capitalism drives an emotional wedge between 'providers' and 'consumers'. Try fine-tuning on some economics journals to get that out of its system."

mannykannot 6y ago

I think the point here is that it hs no understanding of what you are calling a 'bad' prompt. That does not mean that it lacks any understanding whatsoever (personally, I think it has made a small step in that direction), but it is a valid case of it not understanding something. Furthermore, I do not think we can assume that if it were trained on examples of satire or intentionally absurd text, it would perform better on such prompts - in fact, I would not be surprised if its performance would deteriorate on many prompts, both straightforward and tricky ones, if given such training. Now I am wondering if you need a theory of mind before you can begin to understand satire...

IanAWP 6y ago
GPT is both general purpose and stateless. Lacking another mechanism, it's the requirement of the prompt to request a particular genre of response if that's what you want. There can be no such thing as a bad prompt from gpt3's perspective. A bad prompt is one where the user has a specific purpose which is not expressed. It's bad because you know beforehand that gpt3 can not align with it. Someone pours grape juice into a bottle and becomes worried that is not safe to drink. GPT3 correctly grasps that there is a hidden context to this weird prompt, however when given no other information it guesses that this hidden context is something known only to the hypothetical character in the prompt. I would probably do this too. When you give it the correct context (this weird prompt is a logic test) then it gives you the answer you expected.
- mannykannot 6y ago
  > GPT3 correctly grasps that there is a hidden context to this weird prompt. That is interesting. How does it indicate its understanding that there is a hidden context?