3 ms·

> A related question might be "why are we fooled by such simple tricks?" Yes, and to poke at this a bit further, we could ask "What is it that humans are doing

> A related question might be "why are we fooled by such simple tricks?" Yes, and to poke at this a bit further, we could ask "What is it that humans are doing when talking that isn't just simple tricks?" If I had to hazard a guess, I'd say the answer to that is some sort of persistent world modelling, which means an AI may need to have a sense of self before it can move beyond simple tricks.

simonh 4y ago
It’s interesting to see the failure modes of these models. If you ask a sensible question you likely get a sensible answer, because it apples logical transformations on the input text, even novel ones. However if you construct a ludicrous question it will soldier on mindlessly trying to answer it. For example if you ask something like, if the Golden Gate Bridge were to climb the stairs of the Empire State Building, how many flower petals would it need? The chances are the model will give you a number.
- jpindar 4y ago
  I just asked the Replika app that question, and it replied "Oh, I have no idea." It often says something like that when faced with nonsense, so in some ways it's better than a more 'powerful' program.
  - simonh 4y ago
    I made the example up, but Douglas Hofstadter gave some real example of this and the responses in an article recently: https://www.economist.com/by-invitation/2022/06/09/artificial-neural-networks-today-are-not-conscious-according-to-douglas-hofstadter https://www.economist.com/by-invitation/2022/06/09/artificia... >People who interact with gpt-3 usually don’t probe it sceptically. They don’t give it input that stretches concepts beyond their breaking points, so they don’t expose the hollowness behind the scenes.
    - dane-pgp 4y ago
      This criticism was addressed by Nick Cammarata of OpenAI, who said that: > “it’s all about the prelude before the conversation. You need to tell it what the AI is and is not capable [of]. It’s not trying to be right, it’s trying to complete what it thinks the AI would do :)” He did some "prompt engineering" and came up with: > ‘This is a conversation between a human and a brilliant AI. If a question is “normal” the AI answers it. If the question is “nonsense” the AI says “yo be real”’ which lead to better results. Here is an article about these "Uncertainty Prompts": https://arr.am/2020/07/25/gpt-3-uncertainty-prompts/ https://arr.am/2020/07/25/gpt-3-uncertainty-prompts/

simonh 4y ago

It’s interesting to see the failure modes of these models. If you ask a sensible question you likely get a sensible answer, because it apples logical transformations on the input text, even novel ones. However if you construct a ludicrous question it will soldier on mindlessly trying to answer it. For example if you ask something like, if the Golden Gate Bridge were to climb the stairs of the Empire State Building, how many flower petals would it need? The chances are the model will give you a number.

jpindar 4y ago
I just asked the Replika app that question, and it replied "Oh, I have no idea." It often says something like that when faced with nonsense, so in some ways it's better than a more 'powerful' program.
- simonh 4y ago
  I made the example up, but Douglas Hofstadter gave some real example of this and the responses in an article recently: https://www.economist.com/by-invitation/2022/06/09/artificial-neural-networks-today-are-not-conscious-according-to-douglas-hofstadter https://www.economist.com/by-invitation/2022/06/09/artificia... >People who interact with gpt-3 usually don’t probe it sceptically. They don’t give it input that stretches concepts beyond their breaking points, so they don’t expose the hollowness behind the scenes.
  - dane-pgp 4y ago
    This criticism was addressed by Nick Cammarata of OpenAI, who said that: > “it’s all about the prelude before the conversation. You need to tell it what the AI is and is not capable [of]. It’s not trying to be right, it’s trying to complete what it thinks the AI would do :)” He did some "prompt engineering" and came up with: > ‘This is a conversation between a human and a brilliant AI. If a question is “normal” the AI answers it. If the question is “nonsense” the AI says “yo be real”’ which lead to better results. Here is an article about these "Uncertainty Prompts": https://arr.am/2020/07/25/gpt-3-uncertainty-prompts/ https://arr.am/2020/07/25/gpt-3-uncertainty-prompts/