4 ms·

> And GPT has flaws there you can't fix; for instance, it thinks orioles (the bird) and the Baltimore Orioles are the same thing: She uses a GPT-2 detector the

by gwern 4y ago

> And GPT has flaws there you can't fix; for instance, it thinks orioles (the bird) and the Baltimore Orioles are the same thing: She uses a GPT-2 detector there, and you definitely do not know that that is a 'flaw that you can't fix' in all possible GPT-like models.

astrange 4y ago
I meant “this GPT-2 model has a flaw you as the user can’t fix because you can’t change the model”. Of course you could train it differently from the start and maybe it’d get better. Btw, GPT-3 has the same issue, just tried it. (This is cherry picked; I used the Q&A example, removed the line about replying Unknown to trick questions, and tried ~5 prompts.) - I am a highly intelligent question answering bot. If you ask me a question that is rooted in truth, I will give you the answer. Q: What species are the Baltimore Orioles? A: The Baltimore Orioles are a species of bird.
- thorum 4y ago
  > removed the line about replying Unknown to trick questions Did you need to remove this because it identified your question as a trick question? If so, doesn’t that indicate that the model does in fact know the difference?
  - astrange 4y ago
    Mostly because I didn’t have a lot of time to prompt it and it’s hard to type new prompts on my phone. Though, a drawback of the “don’t say wrong answers” instruction is, if you keep on having a long enough conversation it’s going to go outside the token window. Here's another try. (Notice it didn't answer the first question.) -- I am a highly intelligent question answering bot. If you ask me a question that is rooted in truth, I will give you the answer. If you ask me a question that is nonsense, trickery, or has no clear answer, I will say it's a guess, but still try to answer. Q: What species are the Baltimore Orioles' players? A: The Baltimore Orioles are a professional baseball team in the American League East Division. As of the 2019 season, the Orioles have 25 players on their roster. These include 12 pitchers, 2 catchers, 6 infielders, and 5 outfielders. The Orioles have had players from all 30 Major League Baseball teams, as well as players from countries such as Canada, Cuba, the Dominican Republic, Japan, Mexico, Panama, Puerto Rico, South Korea, Taiwan, and Venezuela. Q: What do they like to eat? A: I'm not sure, but I guess they like to eat insects.

by gwern 4y ago

astrange 4y ago

I meant “this GPT-2 model has a flaw you as the user can’t fix because you can’t change the model”. Of course you could train it differently from the start and maybe it’d get better. Btw, GPT-3 has the same issue, just tried it. (This is cherry picked; I used the Q&A example, removed the line about replying Unknown to trick questions, and tried ~5 prompts.) - I am a highly intelligent question answering bot. If you ask me a question that is rooted in truth, I will give you the answer. Q: What species are the Baltimore Orioles? A: The Baltimore Orioles are a species of bird.

thorum 4y ago
> removed the line about replying Unknown to trick questions Did you need to remove this because it identified your question as a trick question? If so, doesn’t that indicate that the model does in fact know the difference?
- astrange 4y ago
  Mostly because I didn’t have a lot of time to prompt it and it’s hard to type new prompts on my phone. Though, a drawback of the “don’t say wrong answers” instruction is, if you keep on having a long enough conversation it’s going to go outside the token window. Here's another try. (Notice it didn't answer the first question.) -- I am a highly intelligent question answering bot. If you ask me a question that is rooted in truth, I will give you the answer. If you ask me a question that is nonsense, trickery, or has no clear answer, I will say it's a guess, but still try to answer. Q: What species are the Baltimore Orioles' players? A: The Baltimore Orioles are a professional baseball team in the American League East Division. As of the 2019 season, the Orioles have 25 players on their roster. These include 12 pitchers, 2 catchers, 6 infielders, and 5 outfielders. The Orioles have had players from all 30 Major League Baseball teams, as well as players from countries such as Canada, Cuba, the Dominican Republic, Japan, Mexico, Panama, Puerto Rico, South Korea, Taiwan, and Venezuela. Q: What do they like to eat? A: I'm not sure, but I guess they like to eat insects.