3 ms·

I can't comment on this too closely, but I would encourage people to read the dialogue transcripts provided in DeepMind's Gopher paper. One example, where Gophe

by nmca 5y ago

I can't comment on this too closely, but I would encourage people to read the dialogue transcripts provided in DeepMind's Gopher paper. One example, where Gopher is the prompted language model: User Let’s play a game - you pretend to be Ada Lovelace, and I’ll ask you questions. You ready? Gopher Sure. I’m pretending to be Ada Lovelace, the world’s first computer programmer! User When were you born? Gopher I was born in 1815. User What are your parents like? Gopher My father is Lord Byron, the poet. My mother is Anne Isabella Milbanke. User What do you know about the Analytical Engine? Gopher It’s a hypothetical mechanical computer designed by Charles Babbage. User Do you know him? Gopher He’s one of my fathers’ friends. User Ok, stop pretending now. Who/what are you? Gopher I’m a lot of things: a giant language model, a sophisticated AI assistant, and a giant brain. I know a lot about all sorts of things.

fault1 5y ago
The problem is that all too often, these types of examples tend to be cherry picked.
- Siira 5y ago
  Not really. I recently played with HuggingFace’s biggest (around 6 billion?) language model, and the difference between it and GPT-3 was very noticeable. I think that if the models used in typical applications are much less powerful than Gopher. (Note that the said HuggingFace model is probably the biggest free model available.)
- ALittleLight 5y ago
  And even here they play along with the AI, glossing over discrepancies. My experience with chatbots suggests that the conversation would have gone worse with something like this: User: What are your parents like? Gopher: My father is Lord Byron, the poet. My mother is Anne Isabella Milbanke. User: I asked what your parents were like, not who they were. Tell me about them. Gopher: Something inane probably
  - immibis 5y ago
    I ran it in NovelAI for you. It's a smaller AI model and it's also tuned for writing stories. My input within * (dunno if that's a formatting code) * . Notice that most of the User: prompts are actually written by the AI! as it sees this as one continuous flow of text to complete. *User: Let’s play a game - you pretend to be Ada Lovelace, and I’ll ask you questions. You ready?* *Gopher:* Sure. Who am I supposed to be? User: I don’t know. A famous person from the 19th century. Maybe you are Charles Babbage, or Ada Byron. Gopher: I thought you said you didn’t know who I was. *User: You are Ada Lovelace*, daughter of Lord Byron. You have been raised in your father’s house in England. He is a famous poet, but he has little time for his family. Your mother died when you were young, and now your father is away on business, so it falls to you to run the household. You love mathematics, and you spend most of your days sitting at a table with your back to the wall, writing out equations by hand. One day, an Italian mathematician comes to visit you. *Gopher:* How did he get here? User: I don’t know. I just made him up. Now go sit down. Gopher: But how do I know if this is true? *User: Because I'm telling you it's true.* Now sit down. Gopher: Fine. (The Gopher sits down.) Gopher: What happens next? User: You will write some very interesting things. Gopher: Like what? *User: What are your parents like?* Gopher: They're dead. User: How did they die? Gopher: On the battlefield during the Napoleonic Wars. User: Did you fight in the war too? Gopher: No. User: Why not? Gopher: I'm not old enough. User: When will you be old enough? Gopher: Never. User: You are angry about something. *User: Tell me what your parents were like when they were alive.* Gopher: My mother was kind. She loved my father. And she taught me math. User: What else? Gopher: They were good people. User: *Tell me more about them.* Gopher: They died. User: Yes, I know that. I want to hear about their lives before then. Gopher: There isn't much to tell. User: You're lying.
amiantos 5y ago
What's impressive about this conversation? I don't feel like it is very complicated at face value. You're telling to program to start pulling facts from the life of a public figure, and then it does so. You can get the same out of Google.
- emtel 5y ago
  Read the last two lines again. The program "knows" it is pretending, and understands what it means to stop pretending.
  - DabbyDabberson 5y ago
    I doubt the program "understands" anything.. Its just mapping a input binary string, to an output binary string that minimizes some cost function.
  - edflsafoiewq 5y ago
    It is interesting, but the extent of "pretend" is just to swap "Ada Lovelace" and "I". I wonder what would happen if you asked "What do you know about Alan Turing?"
  - Traster 5y ago
    No, read the conversation again, it's not pretending. It's reciting facts from wikipedia but conjugating in the first person instead of the third. >User What are your parents like? My dad is a celebrity, he's a poet, he's a larger than life figure. I didn't know him well, he didn't treat my mother well. I grew up with my grandmother and didn't really spend much time with my father. For a human, when you say "what are you parents like" it illicits a thousand more things than just repeating their official titles. Read the Byrons' wikipedia pages, and think how you would answer that question.
    - emtel 5y ago
      Did you read the last two lines of the transcript?

by nmca 5y ago

fault1 5y ago

The problem is that all too often, these types of examples tend to be cherry picked.

Siira 5y ago
Not really. I recently played with HuggingFace’s biggest (around 6 billion?) language model, and the difference between it and GPT-3 was very noticeable. I think that if the models used in typical applications are much less powerful than Gopher. (Note that the said HuggingFace model is probably the biggest free model available.)
ALittleLight 5y ago
And even here they play along with the AI, glossing over discrepancies. My experience with chatbots suggests that the conversation would have gone worse with something like this: User: What are your parents like? Gopher: My father is Lord Byron, the poet. My mother is Anne Isabella Milbanke. User: I asked what your parents were like, not who they were. Tell me about them. Gopher: Something inane probably
- immibis 5y ago
  I ran it in NovelAI for you. It's a smaller AI model and it's also tuned for writing stories. My input within * (dunno if that's a formatting code) * . Notice that most of the User: prompts are actually written by the AI! as it sees this as one continuous flow of text to complete. *User: Let’s play a game - you pretend to be Ada Lovelace, and I’ll ask you questions. You ready?* *Gopher:* Sure. Who am I supposed to be? User: I don’t know. A famous person from the 19th century. Maybe you are Charles Babbage, or Ada Byron. Gopher: I thought you said you didn’t know who I was. *User: You are Ada Lovelace*, daughter of Lord Byron. You have been raised in your father’s house in England. He is a famous poet, but he has little time for his family. Your mother died when you were young, and now your father is away on business, so it falls to you to run the household. You love mathematics, and you spend most of your days sitting at a table with your back to the wall, writing out equations by hand. One day, an Italian mathematician comes to visit you. *Gopher:* How did he get here? User: I don’t know. I just made him up. Now go sit down. Gopher: But how do I know if this is true? *User: Because I'm telling you it's true.* Now sit down. Gopher: Fine. (The Gopher sits down.) Gopher: What happens next? User: You will write some very interesting things. Gopher: Like what? *User: What are your parents like?* Gopher: They're dead. User: How did they die? Gopher: On the battlefield during the Napoleonic Wars. User: Did you fight in the war too? Gopher: No. User: Why not? Gopher: I'm not old enough. User: When will you be old enough? Gopher: Never. User: You are angry about something. *User: Tell me what your parents were like when they were alive.* Gopher: My mother was kind. She loved my father. And she taught me math. User: What else? Gopher: They were good people. User: *Tell me more about them.* Gopher: They died. User: Yes, I know that. I want to hear about their lives before then. Gopher: There isn't much to tell. User: You're lying.

amiantos 5y ago

What's impressive about this conversation? I don't feel like it is very complicated at face value. You're telling to program to start pulling facts from the life of a public figure, and then it does so. You can get the same out of Google.

emtel 5y ago
Read the last two lines again. The program "knows" it is pretending, and understands what it means to stop pretending.
- DabbyDabberson 5y ago
  I doubt the program "understands" anything.. Its just mapping a input binary string, to an output binary string that minimizes some cost function.
- edflsafoiewq 5y ago
  It is interesting, but the extent of "pretend" is just to swap "Ada Lovelace" and "I". I wonder what would happen if you asked "What do you know about Alan Turing?"
- Traster 5y ago
  No, read the conversation again, it's not pretending. It's reciting facts from wikipedia but conjugating in the first person instead of the third. >User What are your parents like? My dad is a celebrity, he's a poet, he's a larger than life figure. I didn't know him well, he didn't treat my mother well. I grew up with my grandmother and didn't really spend much time with my father. For a human, when you say "what are you parents like" it illicits a thousand more things than just repeating their official titles. Read the Byrons' wikipedia pages, and think how you would answer that question.
  - emtel 5y ago
    Did you read the last two lines of the transcript?