4 ms·

It is such a weird question. People fall for AI-generated language because the goal of the people who made the generator was to create language like a human wou

• maurice__ 4y ago
  Interesting angle! We're showing that people fall for generated language not necessarily because generated language is all that good, but because people are looking for the wrong cues and are thus bad at identifying generated language. Back to the scissors, it would be like someone trying to tell you that their scissors cut marvelously, but you ask yourself whether the paper they are demonstrating them on is strong enough to prove that. Making something for cutting doesn't guarantee that it will cut and doesn't explain why. I even suspect that in the early days of scissors it was all that clear why they worked. Similarly, we don't understand much about GPT-3. It was trained to predict the next token in a sequence, not to create an illusion of personhood. But somehow it does so, and we're trying to understand how and why.
• readams 4y ago
  I think you're missing the point. The question could be possibly rephrased as "what are the tricks that the AI uses to fool humans?" The paper goes on to identify some of the specific tricks that the AIs appear to use. A related question might be "why are we fooled by such simple tricks?"
  • dustingetz 4y ago
    why do we assume that humans are smart? answer: ego; corollary: we are not
    • MathYouF 4y ago
      We seem obviously to be the smartest living things in the universe we know of. We're also approximately the dumbest possible things that could construct the society and technology we have (if we weren't, we would've done it sooner). I agree that some humility of our own intellect as primates would do us a lot of good in the coming age of AI.
      • lalopalota 4y ago
        s/We /A few of us/
        • kadonoishi 4y ago
          Appreciate the careful space after "We ".
          • dane-pgp 4y ago
            I suppose it makes up for the missing one after "us".
      • dustingetz 4y ago
        civilization was not constructed, it evolved, like our biological bodies
        • ben_w 4y ago
          I’d say it gets more constructed and less evolved the further up the hierarchy of civilisation we get. A merchant of 1320 had relatively few laws to contend with, a corner shop in 2020 could not function without many layers of engineered rules for the people who provide them with the services they are themselves required to use due to other engineered rules for the benefit of their customers.
          • MathYouF 4y ago
            This makes me wonder what laws and tedium of administration merchants of the 1320's would complain about. I'm sure they still had a few depending on the region, and maybe more severe possible outcomes (highway robbery, unlawful arrest because of the influence of a rival merchant, arbitrary taxation and tariffs, etc.).
  • dalbasal 4y ago
    Taking your point, the title should probably be "How we fall for AI-generated language."
  • FFRefresh 4y ago
    If you follow that question through and think through the implications, it paints potentially a dark future for digital communication. The 'tricks' AI uses are also 'tricks' that humans use in everyday conversation, we just don't call them 'tricks' when humans are involved. If we start assuming that first-person pronoun usage, mentions of family, etc. are potential signals of AI, then I don't see how we don't end up in a state where increased dehumanization occurs.
  • dane-pgp 4y ago
    > A related question might be "why are we fooled by such simple tricks?" Yes, and to poke at this a bit further, we could ask "What is it that humans are doing when talking that isn't just simple tricks?" If I had to hazard a guess, I'd say the answer to that is some sort of persistent world modelling, which means an AI may need to have a sense of self before it can move beyond simple tricks.
    • simonh 4y ago
      It’s interesting to see the failure modes of these models. If you ask a sensible question you likely get a sensible answer, because it apples logical transformations on the input text, even novel ones. However if you construct a ludicrous question it will soldier on mindlessly trying to answer it. For example if you ask something like, if the Golden Gate Bridge were to climb the stairs of the Empire State Building, how many flower petals would it need? The chances are the model will give you a number.
      • jpindar 4y ago
        I just asked the Replika app that question, and it replied "Oh, I have no idea." It often says something like that when faced with nonsense, so in some ways it's better than a more 'powerful' program.
        • simonh 4y ago
          I made the example up, but Douglas Hofstadter gave some real example of this and the responses in an article recently: https://www.economist.com/by-invitation/2022/06/09/artificial-neural-networks-today-are-not-conscious-according-to-douglas-hofstadter https://www.economist.com/by-invitation/2022/06/09/artificia... >People who interact with gpt-3 usually don’t probe it sceptically. They don’t give it input that stretches concepts beyond their breaking points, so they don’t expose the hollowness behind the scenes.
          • dane-pgp 4y ago
            This criticism was addressed by Nick Cammarata of OpenAI, who said that: > “it’s all about the prelude before the conversation. You need to tell it what the AI is and is not capable [of]. It’s not trying to be right, it’s trying to complete what it thinks the AI would do :)” He did some "prompt engineering" and came up with: > ‘This is a conversation between a human and a brilliant AI. If a question is “normal” the AI answers it. If the question is “nonsense” the AI says “yo be real”’ which lead to better results. Here is an article about these "Uncertainty Prompts": https://arr.am/2020/07/25/gpt-3-uncertainty-prompts/ https://arr.am/2020/07/25/gpt-3-uncertainty-prompts/
• synu 4y ago
  Researching semi-obvious things like what elements of AI-generated text humans mistake for being human-generated is part of the process of how people working on AIs work towards better generation. You’re just seeing how the sausage gets made.
• dvt 4y ago
  This is actually a pretty good comment. The scissor analogy, while a bit on-the-nose, is very accurate. Maybe a better example would be: why is our body fooled by artificial hearts? Simply because it was built in such a way that it simulates a real heart pretty well. Similarly, these models are built in such a way that they simulate real-life conversations pretty well. There's nothing really more to it. In my view, this phenomenon has nothing to do with intelligence or how smart we are, or whatever.
  • deltaonefour 4y ago
    Analogies aren't proof for anything. I'm pretty sure though, we obviously can't prove these chatbots are sentient. Clearly. However, for the first time, we ALSO cannot prove that these chatbots aren't sentient. The statistics are proof of that. What you and the parent poster are describing here are simply opinions. We are at a point where the null hypothesis and the hypothesis itself cannot be proven. And that is compelling. The pretentiousness of a lot of people is astounding. That statistic no matter how you look at it is a compelling statement about AGI, independent of whether or not these chatbots are AGIs.
• mattnewton 4y ago
  Maybe a bit, they aren’t just asking why scissors cut paper, but also why we landed on that design. What about it makes it ergonomic to hold and efficient, and more to the point, why does being sharp cut it? In the language model case, why can we model language this way so effectively and why does it follow these statistical patterns? It turns out that maybe a major reason has something to do with self description. This is also a more interesting question because we understand language less than we understand cutting paper, and also because the process humans used to design large language models is more indirect and alien than traditional industrial design. My takeaway was that people reading language modeling a person writing about themselves imagine it was written by a person, more than text written by people not about themselves. That’s an interesting trick! Describing human experiences in language makes people attribute the language to a human right now. Maybe this will change after a generation of people knowing about this, but that seems important to think about.
• mannykannot 4y ago
  Goals are not automatically satisfied, let alone well-satisfied. The question is asking what it is about us that allows the methods used to be particularly effective.
• yarg 4y ago
  What's the underlying structure that leads to the complex behaviour? It's not asking about the motivation for the form, it's asking about the properties of the structure. With scissors neither the mechanism or the behaviour is particularly complex, so actually understanding what's going on isn't too much of a struggle. With deep-ML you're dealing with a very large number of entanglements of increasingly abstract and opaque higher dimensional concepts or notions (depending on how much you want to anthropomorphise the machine).
• deltaonefour 4y ago
  There is a point here that even the twitter poster is missing. This machine is imitating a human better than a human itself, and we now have statistics to prove it. The question is... if statistically it's better at appearing human than a human itself, than is this machine really just a language generator? Or is it something more? I'm not saying these things are sentient. But the AI has risen to the point where this question is becoming ask-able. We are at the border here. If we can't ask this question now, then we are really damn close. A lot of pretentious people on HN claim absolutely that our best chat bot technology is clearly not sentient. It reminds me of the beginning of the COVID pandemic when the CDC said masks were ineffective and you had a bunch of know-it-alls and armchair experts just repeating that BS over and over again as if they knew what they were talking about. I think if these pretentious people were actually intelligent they would know that we actually do not HAVE enough information to make a claim in EITHER direction. We can't know if it's actually sentient or not. That fact in itself is both interesting and compelling 4 or 5 years ago chatbots COULD not imitate humans and were CLEARLY fake. What we're seeing unfold before our eyes is a first. We're not even sure what sentience is. But we do know that humans are sentient. And we don't know if whatever is going on inside of these chatbots is comparable with what's going on inside human brains. Thus when given a chatbot that imitates humans perfectly, it's actually impossible to know if it's sentient.
  • simonh 4y ago
    It depends on whether the estimation comes from pretension, or comes from actual understanding of what these bots are doing. The fact is most people on the street, including me, are very easy to fool with linguistic tricks and are quite poor at investigating unfamiliar situations. Advertising and marketing exist because of this. It’s also why we have very strict laws and regulations controlling advertising and how products are sold. Otherwise a lot of people would be very easy to rip off with very simple misdirection. What these language models are doing is automated misdirection. They are taking an input text and transforming it based on rules, but they have absolutely no understanding of any of it. This is very, very easy to demonstrate if you know how the models work. You can sit down and generate hundreds of questions one after the other that demonstrate this very easily if you understand the process. The problem is that people instinctively proceed from the assumption that the system they are talking to might be human and give it a fair chance by asking answerable questions. Since it’s trained on answerable questions it often gives a reasonable answer. But if you ask even slightly unanswerable questions the system plods on mechanically trying to answer it anyway and produces gibberish, exposing the flaws in the mindless rote process it’s following.
• joe__f 4y ago
  I got why scissors cut paper, but I'm always stuck wondering why paper wraps rock...