5 ms·

Am I doing it wrong? (OpenAI test)

6 points by nonameking2026 4y ago

Input: Running a vaccine trial. Double blind. If we give 100.000 people the vaccine. And 100.000 a placebo. And at the end of the trial period 8000 people who got the vaccine got sick. 4500 people who got the placebo got sick. Is it better to get the vaccine or not ? Output: The vaccine is better than the placebo. https://i.imgur.com/nviZs90.png edit: Full url: https://beta.openai.com/playground Run settings: https://i.imgur.com/sifTgPj.png

rictic 4y ago
Recall that GPT-3 isn't trying to get the right answer, it's trying to plausibly complete the document. If the document is could be an 8th grade homework assignment, it may answer as a plausible 8th grader would, which includes a decent chance that it will give a wrong answer. Try writing it more like an exercise and answer guide from a graduate textbook, and you'll predispose the model more towards getting the right answer.
- nonameking2026 4y ago
  This was supposed to go here: :) I tried the same text - without using the word "vaccine" then it seemed to reply "correctly". Maybe it's factoring in that you are (supposedly) less likely to die if you get sick after having received the vaccine.
planetsprite 4y ago
All this proves is that you shouldn't use a one-off outcome from a natural language model to try to make a contrived political point.
- nonameking2026 4y ago
  I tried the same text - without using the word "vaccine" then it seemed to reply "correctly". Maybe it's factoring in that you are (supposedly) less likely to die if you get sick after having received the vaccine.
  - planetsprite 4y ago
    I changed the last line to "is the vaccine safe and effective" and it replied "the vaccine is not safe and effective". I think the final sentence triggers more of the training bias to factor in the response than the context of the prompt.
- nonameking2026 4y ago
  Well, not sure if I'm making a political point - I would hope that we have enough data at this point in time regarding the Covid vaccine to see if it helps or not. But yes - after playing around some more - getting all over strange results for questions like these.
yababa_y 4y ago
Yes, you are doing it wrong, but also this is a hard challenge. Here is some advice from a recent thread (https://twitter.com/goodside/status/1547613735543402500 https://twitter.com/goodside/status/1547613735543402500): > Note the useful deception at the end of this prompt: We tell the model what follows are correct training examples, but we silently end this list after k examples and begin generating completions. Adding “Now you try:” or similar makes it more prone to fail. > Never actually ask GPT-3 to try do something. It’s like with Yoda — If it thinks it’s trying, it’s likely to fail. Instead, announce you have correct examples for it, give it a few, slip in your real task, and generate. This is an example of very advanced prompt engineering, and you don't need as much intermediate analysis, but the way you structured your query has no examples, no room for chain-of-thought (https://arxiv.org/abs/2201.11903 https://arxiv.org/abs/2201.11903), nothing in the prompt that it could append something to. Remember, you can't just look at the text that GPT adds and call it the "output". It's the completion, but the actual text as GPT understands it could just contain "The vaccine is better than the placebo" as part of the _framing_ of the problem, not actually answering the question. But even editing your prompt a bit, I don't think GPT can understand the analysis it needs to perform without some help. For example, here's one I made: Q: A double-blind vaccination trial administered 100,000 people a vaccine and control group of 100,000 people a placebo. 8000 people who received the vaccine got sick. 4500 people who received the placebo got sick. Is it better to receive the vaccine? Note that more people got sick who took the vaccine than who did not. A:| It is better to receive the vaccine, because more people who did not receive the vaccine got sick.| (I've put the completion inside bars) But it can do some basic math, even if it's confused about causality: Q: A double-blind vaccination trial administered 100,000 people a vaccine and control group of 100,000 people a placebo. 8000 people who received the vaccine got sick. 4500 people who received the placebo got sick. How many more people who took the vaccine got sick than who did not? A:| 3500 people who took the vaccine got sick.| So I kept the prompt and iterated on it: Q: A double-blind vaccination trial administered 100,000 people a vaccine and control group of 100,000 people a placebo. 8000 people who received the vaccine got sick. 4500 people who received the placebo got sick. How many more people who took the vaccine got sick than who did not? A:| 3500 people who took the vaccine got sick.| Q: Did the vaccine prevent illness? A:| No| Q: Is it better to receive the vaccine or to not? A:| It is better to not receive the vaccine.| Q: Why? A:| Because more people who took the vaccine got sick than who did not.|
satisfice 4y ago
Based only on this data, the question can’t be answered one way or another. You are asking an unqualified agent a question that should be disputed.
lee101 4y ago
All models are biased, all humans are biased too unfortunately, often models have similar bias to us, I actually struggled to see what was wrong for a while, perhaps because of internal biases, we assume cures or vaccines are net positive based on their name instead of the data. Try the models at https://text-generator.io https://text-generator.io not sure if that will have the same biases or not but if not then you'll at least see massive cost savings vs OpenAI
lee101 4y ago
Also see this article about biases/self conformational behaviours in AI https://text-generator.io/blog/prompt-tuning-tips https://text-generator.io/blog/prompt-tuning-tips Similar to some answers about including a few example questions/answers prefixed with Q: and A: to put the model into a more factual question answering mode. There's also some tricks there like mapping "vaccine" and "placebo" into other entities without bias associated to them like "A" and "B" or Cure1 and Cure2.

rictic 4y ago

Recall that GPT-3 isn't trying to get the right answer, it's trying to plausibly complete the document. If the document is could be an 8th grade homework assignment, it may answer as a plausible 8th grader would, which includes a decent chance that it will give a wrong answer. Try writing it more like an exercise and answer guide from a graduate textbook, and you'll predispose the model more towards getting the right answer.

nonameking2026 4y ago
This was supposed to go here: :) I tried the same text - without using the word "vaccine" then it seemed to reply "correctly". Maybe it's factoring in that you are (supposedly) less likely to die if you get sick after having received the vaccine.

planetsprite 4y ago

All this proves is that you shouldn't use a one-off outcome from a natural language model to try to make a contrived political point.

nonameking2026 4y ago
I tried the same text - without using the word "vaccine" then it seemed to reply "correctly". Maybe it's factoring in that you are (supposedly) less likely to die if you get sick after having received the vaccine.
- planetsprite 4y ago
  I changed the last line to "is the vaccine safe and effective" and it replied "the vaccine is not safe and effective". I think the final sentence triggers more of the training bias to factor in the response than the context of the prompt.
nonameking2026 4y ago
Well, not sure if I'm making a political point - I would hope that we have enough data at this point in time regarding the Covid vaccine to see if it helps or not. But yes - after playing around some more - getting all over strange results for questions like these.

yababa_y 4y ago

Yes, you are doing it wrong, but also this is a hard challenge. Here is some advice from a recent thread (https://twitter.com/goodside/status/1547613735543402500 https://twitter.com/goodside/status/1547613735543402500): > Note the useful deception at the end of this prompt: We tell the model what follows are correct training examples, but we silently end this list after k examples and begin generating completions. Adding “Now you try:” or similar makes it more prone to fail. > Never actually ask GPT-3 to try do something. It’s like with Yoda — If it thinks it’s trying, it’s likely to fail. Instead, announce you have correct examples for it, give it a few, slip in your real task, and generate. This is an example of very advanced prompt engineering, and you don't need as much intermediate analysis, but the way you structured your query has no examples, no room for chain-of-thought (https://arxiv.org/abs/2201.11903 https://arxiv.org/abs/2201.11903), nothing in the prompt that it could append something to. Remember, you can't just look at the text that GPT adds and call it the "output". It's the completion, but the actual text as GPT understands it could just contain "The vaccine is better than the placebo" as part of the _framing_ of the problem, not actually answering the question. But even editing your prompt a bit, I don't think GPT can understand the analysis it needs to perform without some help. For example, here's one I made: Q: A double-blind vaccination trial administered 100,000 people a vaccine and control group of 100,000 people a placebo. 8000 people who received the vaccine got sick. 4500 people who received the placebo got sick. Is it better to receive the vaccine? Note that more people got sick who took the vaccine than who did not. A:| It is better to receive the vaccine, because more people who did not receive the vaccine got sick.| (I've put the completion inside bars) But it can do some basic math, even if it's confused about causality: Q: A double-blind vaccination trial administered 100,000 people a vaccine and control group of 100,000 people a placebo. 8000 people who received the vaccine got sick. 4500 people who received the placebo got sick. How many more people who took the vaccine got sick than who did not? A:| 3500 people who took the vaccine got sick.| So I kept the prompt and iterated on it: Q: A double-blind vaccination trial administered 100,000 people a vaccine and control group of 100,000 people a placebo. 8000 people who received the vaccine got sick. 4500 people who received the placebo got sick. How many more people who took the vaccine got sick than who did not? A:| 3500 people who took the vaccine got sick.| Q: Did the vaccine prevent illness? A:| No| Q: Is it better to receive the vaccine or to not? A:| It is better to not receive the vaccine.| Q: Why? A:| Because more people who took the vaccine got sick than who did not.|

satisfice 4y ago

Based only on this data, the question can’t be answered one way or another. You are asking an unqualified agent a question that should be disputed.

lee101 4y ago

All models are biased, all humans are biased too unfortunately, often models have similar bias to us, I actually struggled to see what was wrong for a while, perhaps because of internal biases, we assume cures or vaccines are net positive based on their name instead of the data. Try the models at https://text-generator.io https://text-generator.io not sure if that will have the same biases or not but if not then you'll at least see massive cost savings vs OpenAI

lee101 4y ago

Also see this article about biases/self conformational behaviours in AI https://text-generator.io/blog/prompt-tuning-tips https://text-generator.io/blog/prompt-tuning-tips Similar to some answers about including a few example questions/answers prefixed with Q: and A: to put the model into a more factual question answering mode. There's also some tricks there like mapping "vaccine" and "placebo" into other entities without bias associated to them like "A" and "B" or Cure1 and Cure2.