5 ms·

I think you're missing the point. The question could be possibly rephrased as "what are the tricks that the AI uses to fool humans?" The paper goes on to identi

I think you're missing the point. The question could be possibly rephrased as "what are the tricks that the AI uses to fool humans?" The paper goes on to identify some of the specific tricks that the AIs appear to use. A related question might be "why are we fooled by such simple tricks?"

dustingetz 4y ago
why do we assume that humans are smart? answer: ego; corollary: we are not
- MathYouF 4y ago
  We seem obviously to be the smartest living things in the universe we know of. We're also approximately the dumbest possible things that could construct the society and technology we have (if we weren't, we would've done it sooner). I agree that some humility of our own intellect as primates would do us a lot of good in the coming age of AI.
  - lalopalota 4y ago
    s/We /A few of us/
    - kadonoishi 4y ago
      Appreciate the careful space after "We ".
      - dane-pgp 4y ago
        I suppose it makes up for the missing one after "us".
  - dustingetz 4y ago
    civilization was not constructed, it evolved, like our biological bodies
    - ben_w 4y ago
      I’d say it gets more constructed and less evolved the further up the hierarchy of civilisation we get. A merchant of 1320 had relatively few laws to contend with, a corner shop in 2020 could not function without many layers of engineered rules for the people who provide them with the services they are themselves required to use due to other engineered rules for the benefit of their customers.
      - MathYouF 4y ago
        This makes me wonder what laws and tedium of administration merchants of the 1320's would complain about. I'm sure they still had a few depending on the region, and maybe more severe possible outcomes (highway robbery, unlawful arrest because of the influence of a rival merchant, arbitrary taxation and tariffs, etc.).
        ben_w 4y ago
        Yup. I think also languages were much more variable, and doing accounting in Roman numerals was so hard they did it twice and averaged the answers, and contracts and tax receipts were done by carving marks in sticks (hence, apparently, the etymology of "stocks").
        simonh 4y ago
        Even the Romans didn’t use Roman numbers much for actual maths, they used the Greek system.
        ben_w 4y ago
        Huh, TIL. Thanks :)
        simonh 4y ago
        You’re welcome, I learned that here too a few years ago. Love HN.
  - jacquesm 4y ago
    > We seem obviously to be the smartest living things in the universe we know of. Humblest too.
- lofatdairy 4y ago
  The reason why this is interesting isn't because humans are smart, but the assumption that human fallibility is both predictable and worth investigating (a supposition I'm inclined to agree with). Even if humans are easy to fool (though the fact that we're just now achieving this on a generalized scale after at least 80 years of theorizing seems to contradict this), humans being fooled can result in significant enough impacts where we should still investigate the degree to which they can be tricked, and what methods of discerning are available to us.
  - deltaonefour 4y ago
    I bet even if a researcher truly created an AGI, there'd be tons of threads like this claiming that it's just a language generator.
dalbasal 4y ago
Taking your point, the title should probably be "How we fall for AI-generated language."
FFRefresh 4y ago
If you follow that question through and think through the implications, it paints potentially a dark future for digital communication. The 'tricks' AI uses are also 'tricks' that humans use in everyday conversation, we just don't call them 'tricks' when humans are involved. If we start assuming that first-person pronoun usage, mentions of family, etc. are potential signals of AI, then I don't see how we don't end up in a state where increased dehumanization occurs.
dane-pgp 4y ago
> A related question might be "why are we fooled by such simple tricks?" Yes, and to poke at this a bit further, we could ask "What is it that humans are doing when talking that isn't just simple tricks?" If I had to hazard a guess, I'd say the answer to that is some sort of persistent world modelling, which means an AI may need to have a sense of self before it can move beyond simple tricks.
- simonh 4y ago
  It’s interesting to see the failure modes of these models. If you ask a sensible question you likely get a sensible answer, because it apples logical transformations on the input text, even novel ones. However if you construct a ludicrous question it will soldier on mindlessly trying to answer it. For example if you ask something like, if the Golden Gate Bridge were to climb the stairs of the Empire State Building, how many flower petals would it need? The chances are the model will give you a number.
  - jpindar 4y ago
    I just asked the Replika app that question, and it replied "Oh, I have no idea." It often says something like that when faced with nonsense, so in some ways it's better than a more 'powerful' program.
    - simonh 4y ago
      I made the example up, but Douglas Hofstadter gave some real example of this and the responses in an article recently: https://www.economist.com/by-invitation/2022/06/09/artificial-neural-networks-today-are-not-conscious-according-to-douglas-hofstadter https://www.economist.com/by-invitation/2022/06/09/artificia... >People who interact with gpt-3 usually don’t probe it sceptically. They don’t give it input that stretches concepts beyond their breaking points, so they don’t expose the hollowness behind the scenes.
      - dane-pgp 4y ago
        This criticism was addressed by Nick Cammarata of OpenAI, who said that: > “it’s all about the prelude before the conversation. You need to tell it what the AI is and is not capable [of]. It’s not trying to be right, it’s trying to complete what it thinks the AI would do :)” He did some "prompt engineering" and came up with: > ‘This is a conversation between a human and a brilliant AI. If a question is “normal” the AI answers it. If the question is “nonsense” the AI says “yo be real”’ which lead to better results. Here is an article about these "Uncertainty Prompts": https://arr.am/2020/07/25/gpt-3-uncertainty-prompts/ https://arr.am/2020/07/25/gpt-3-uncertainty-prompts/

dustingetz 4y ago

why do we assume that humans are smart? answer: ego; corollary: we are not

MathYouF 4y ago
We seem obviously to be the smartest living things in the universe we know of. We're also approximately the dumbest possible things that could construct the society and technology we have (if we weren't, we would've done it sooner). I agree that some humility of our own intellect as primates would do us a lot of good in the coming age of AI.
- lalopalota 4y ago
  s/We /A few of us/
  - kadonoishi 4y ago
    Appreciate the careful space after "We ".
    - dane-pgp 4y ago
      I suppose it makes up for the missing one after "us".
- dustingetz 4y ago
  civilization was not constructed, it evolved, like our biological bodies
  - ben_w 4y ago
    I’d say it gets more constructed and less evolved the further up the hierarchy of civilisation we get. A merchant of 1320 had relatively few laws to contend with, a corner shop in 2020 could not function without many layers of engineered rules for the people who provide them with the services they are themselves required to use due to other engineered rules for the benefit of their customers.
    - MathYouF 4y ago
      This makes me wonder what laws and tedium of administration merchants of the 1320's would complain about. I'm sure they still had a few depending on the region, and maybe more severe possible outcomes (highway robbery, unlawful arrest because of the influence of a rival merchant, arbitrary taxation and tariffs, etc.).
      - ben_w 4y ago
        Yup. I think also languages were much more variable, and doing accounting in Roman numerals was so hard they did it twice and averaged the answers, and contracts and tax receipts were done by carving marks in sticks (hence, apparently, the etymology of "stocks").
        simonh 4y ago
        Even the Romans didn’t use Roman numbers much for actual maths, they used the Greek system.
        ben_w 4y ago
        Huh, TIL. Thanks :)
        simonh 4y ago
        You’re welcome, I learned that here too a few years ago. Love HN.
- jacquesm 4y ago
  > We seem obviously to be the smartest living things in the universe we know of. Humblest too.
lofatdairy 4y ago
The reason why this is interesting isn't because humans are smart, but the assumption that human fallibility is both predictable and worth investigating (a supposition I'm inclined to agree with). Even if humans are easy to fool (though the fact that we're just now achieving this on a generalized scale after at least 80 years of theorizing seems to contradict this), humans being fooled can result in significant enough impacts where we should still investigate the degree to which they can be tricked, and what methods of discerning are available to us.
- deltaonefour 4y ago
  I bet even if a researcher truly created an AGI, there'd be tons of threads like this claiming that it's just a language generator.

dalbasal 4y ago

Taking your point, the title should probably be "How we fall for AI-generated language."

FFRefresh 4y ago

If you follow that question through and think through the implications, it paints potentially a dark future for digital communication. The 'tricks' AI uses are also 'tricks' that humans use in everyday conversation, we just don't call them 'tricks' when humans are involved. If we start assuming that first-person pronoun usage, mentions of family, etc. are potential signals of AI, then I don't see how we don't end up in a state where increased dehumanization occurs.

dane-pgp 4y ago

> A related question might be "why are we fooled by such simple tricks?" Yes, and to poke at this a bit further, we could ask "What is it that humans are doing when talking that isn't just simple tricks?" If I had to hazard a guess, I'd say the answer to that is some sort of persistent world modelling, which means an AI may need to have a sense of self before it can move beyond simple tricks.

simonh 4y ago
It’s interesting to see the failure modes of these models. If you ask a sensible question you likely get a sensible answer, because it apples logical transformations on the input text, even novel ones. However if you construct a ludicrous question it will soldier on mindlessly trying to answer it. For example if you ask something like, if the Golden Gate Bridge were to climb the stairs of the Empire State Building, how many flower petals would it need? The chances are the model will give you a number.
- jpindar 4y ago
  I just asked the Replika app that question, and it replied "Oh, I have no idea." It often says something like that when faced with nonsense, so in some ways it's better than a more 'powerful' program.
  - simonh 4y ago
    I made the example up, but Douglas Hofstadter gave some real example of this and the responses in an article recently: https://www.economist.com/by-invitation/2022/06/09/artificial-neural-networks-today-are-not-conscious-according-to-douglas-hofstadter https://www.economist.com/by-invitation/2022/06/09/artificia... >People who interact with gpt-3 usually don’t probe it sceptically. They don’t give it input that stretches concepts beyond their breaking points, so they don’t expose the hollowness behind the scenes.
    - dane-pgp 4y ago
      This criticism was addressed by Nick Cammarata of OpenAI, who said that: > “it’s all about the prelude before the conversation. You need to tell it what the AI is and is not capable [of]. It’s not trying to be right, it’s trying to complete what it thinks the AI would do :)” He did some "prompt engineering" and came up with: > ‘This is a conversation between a human and a brilliant AI. If a question is “normal” the AI answers it. If the question is “nonsense” the AI says “yo be real”’ which lead to better results. Here is an article about these "Uncertainty Prompts": https://arr.am/2020/07/25/gpt-3-uncertainty-prompts/ https://arr.am/2020/07/25/gpt-3-uncertainty-prompts/