9 ms·

I had someone much knowledgable on this topic than myself claim ChatGPT and the like "understand" stuff. My standing argument is that they wouldn't hallucinate

by _8j50 4y ago

I had someone much knowledgable on this topic than myself claim ChatGPT and the like "understand" stuff. My standing argument is that they wouldn't hallucinate incorrect responses if they understood anything, the hallucination is when their approximation of what a real response would be falls short. These emergent abilities are not actually that, but a result of humans' poor understanding of cognition and communication. What concerns me very much is how the harms that can be caused by LLMs has been so greatly under reported. I imagine being someone with the right power and access telling ChatGPT "find all people that would vote against this candidate in real time and devise ad content and social media messaging and bit interaction to change their minds or discourage them from voting" heck, any intel org of a major country is probably already working on this. No more whistleblowing or posting anonymously on social media, companies would even share models based on private email and conversations you had so other companies could use LLMs to identify everything you posted elsewhere and to have LLMs designate a score for hoe hireable you are. Police can crack down on crime better but also crack down on dissent or any police reforms. And we aren't even talking about war time use of LLMs or what happens when you marry something like ChatGPT with Dall-E and make it all real-time. I am warning anyone who will listen. Smartphones are the most dangerous things out there. Any service or interaction that depends on them is deteimental to peace and liberty of the masses long term. People have not learned a thing from Snowden or 2016 elections. And why are all the smart journalists asleep on the job on this topic. Where are the unreasonable scaremongerers when you need them!

Mike_12345 4y ago
> I had someone much knowledgable on this topic than myself claim ChatGPT and the like "understand" stuff. My standing argument is that they wouldn't hallucinate incorrect responses if they understood anything, the hallucination is when their approximation of what a real response would be falls short. ChatGPT models semantic relationships in the data. That's what your smart buddy means by "understanding". That is a high dimensional model of the data set which infers abstract semantic relationships / concepts. But he would not claim that those semantic relationships are exactly the same as any human interpretation of the data (which you refer to as the "real" response). Language models also have limited reasoning abilities. They are capable of misunderstanding as much as understanding.
- PaulDavisThe1st 4y ago
  Not bad, but: > Language models also have limited reasoning abilities. They have no reasoning abilities at all.
  - Mike_12345 4y ago
    "Language Models Perform Reasoning via Chain of Thought" Posted by Jason Wei and Denny Zhou, Research Scientists, Google Research, Brain team https://ai.googleblog.com/2022/05/language-models-perform-reasoning-via.html https://ai.googleblog.com/2022/05/language-models-perform-re...
    - PaulDavisThe1st 4y ago
      This paper is misusing the term "reasoning" in my opinion. At no point does the LLM know that 5+6 = 11, and if asked to solve a problem in which 5+6 was an implicit component of the solution but not explicitly present in the text, it would be completely lost.
      - deleted 4y ago
        [deleted]
      - Mike_12345 4y ago
        You have a narrow definition of reasoning. Formally and technically it is solving a symbolic reasoning task through a sequence of steps. Yes we know it's not conscious and not human reasoning. > At no point does the LLM know that 5+6 = 11 Does it need to "know" that (by your narrow definition of "know") in order to reason about a word math problem? > if asked to solve a problem in which 5+6 was an implicit component of the solution but not explicitly present in the text, it would be completely lost Can you provide an example? What makes you believe it can't be trained to solve those too? That's just a higher abstraction over the language. Add more layers, more training, etc. Many humans cannot solve basic math word puzzles that this artificial neural network can already solve.
flangola7 4y ago
> My standing argument is that they wouldn't hallucinate incorrect responses if they understood anything, the hallucination is when their approximation of what a real response would be falls short. > These emergent abilities are not actually that, but a result of humans' poor understanding of cognition and communication. My counterargument is that humans hallucinate too, and often. As just one small example, eyewitness testimony is stupifyingily unreliable. Neurological research and even basic behavioral research shows our brains act as bullshit machines fabricating satisfying narratives constantly. Not to even get into the fact that the word hallucination still has a non-AI meaning, and that dreams exist. As I put see it GPT models simply hallucinate more often and, more noticably, in a different manner than humans. The hallucination frequency need not reach zero, only human equivalent or better , and GPT-4 is already much better than GPT-3. I agree with everything else you said fully. "True" reasoning machines or not, society will be catastrophically destabilized. Amongst the chaos I expect plentiful of "normal" conventional and nuclear war to go on.
- ChatGTP 4y ago
  What's the difference between "hallucinating" and just getting something wrong? Hallucinations are where you hear, see, smell, taste or feel things that appear to be real but only exist in your mind. Get medical help if you or someone else have hallucinations. Why do we say LLMs hallucinate and why do people keep parroting the same thing "oh humans hallucinate", do we really hallucinate all the time? I can think of only one time a healthy adult hallucinates and it's not sitting around drinking a cup of tea.
  - SanderNL 4y ago
    The word has a slightly different meaning here (confabulation) and more akin to the witness example: confidently telling what you think is true, but it turned out to be complete shit. It’s not about humans intentionally lying or truly hallucinating like on drugs. It’s about confidently thinking they are right about something which turns out not to be true. I mean, if I say it like this is like humanity’s core business.
    - 4y ago
p-e-w 4y ago
> My standing argument is that they wouldn't hallucinate incorrect responses if they understood anything Alexander Grothendieck, one of the greatest mathematicians of the 20th century, answered "57" when asked in a public lecture to provide an example of a prime number. 57 = 3 * 19 is not a prime number. According to your argument, this would imply Grothendieck did not understand what a prime number is. Which is laughable.
- deafpolygon 4y ago
  Humans are not infallible. It's a mistake to anthromorphize LLM.
  - SanderNL 4y ago
    LLM wrong => unredeemable and fatally flawed human wrong => no problem, mistakes happen
    - lewhoo 4y ago
      Yeah, a little bit. Our biology makes us unpredictable to a certain degree. In simple terms we're allowed to have a bad or a good day. LLM's remain unchangeable and simple mistakes they make can't be explained in the same manner I think.
      - SanderNL 4y ago
        I get the reflex, but this becomes increasingly untenable as their complexity balloons. (Our "biology" is some sort of code for "it's complex".)
      - gmerc 4y ago
        But LLMs are not unchangable, there are several mechanisms available by now to drive change and learning (especially as context windows improve) Practically for outcomes it doesn’t matter if that learning is happening in the LLM or on a layer above it.
    - 4y ago
nl 4y ago
> My standing argument is that they wouldn't hallucinate incorrect responses if they understood anything, the hallucination is when their approximation of what a real response would be falls short. This doesn't seem to make sense. If anything the opposite is true - if the things that are hallucinated make sense (even if not true) it means there is some "understanding" or a world model.
- _8j50 4y ago
  No, hallucinations are similar to but not quite the expected result which shows they are approximations. For example, you tell Dall-E to draw a picutre of a man smoking a pipe but then it draws the pipe coming out of the mans butt and instead of a head the man has a leg on his neck. This is approximation. Now if the pipe looked wrong or it was a trumpet or the guy's head looked weird but still a head maybe it knows what a head is and it just made a mistake and it knows that the pipe goes into the mouth so it will be somewhere near the face. Undesrstanding is a 3yr old child drawing terribly, approximarion is drawing really well but all wrong. All this applies to LLMs I used Dall-E because pictures are easier to talk about.
  - nl 4y ago
    I haven't used dall-e much but I've never seen stable diffusion or midjourney make an error like that, unless of course deliberately promoted. You can see this because of the big deal people made about image gen tools getting hands wrong: it was the most significant error that was systematically occuring.
    - _8j50 4y ago
      There us just something about the mistakes. Let me put it thid way, it nails down hands and faces great most of the time with stable diffusion for example, why would that ever be an issue at any point if it understood what hands and faces were? If I didn't understand what a hand was or how to draw it,I would never get it right. But if I do get it right quite a bit because I understand what the object is, then it makes no sense for me to have a significant error rate where hands and faces are deformed. An artist who knows how to draw hands and faces will never make that mistake, especially when self-correcting is so easy. The only explanation is that it is approximating based on what it learned from the large swath of training data. Kind of like if a human remembered answers to a multiplication table by memorizing every input,output and trend as opposed to knowing how to process the data within the rules of math and generate output. LLMs imo don't understand the rules of language and context they only approximate what a rule compliant system should output.

by _8j50 4y ago

Mike_12345 4y ago

> I had someone much knowledgable on this topic than myself claim ChatGPT and the like "understand" stuff. My standing argument is that they wouldn't hallucinate incorrect responses if they understood anything, the hallucination is when their approximation of what a real response would be falls short. ChatGPT models semantic relationships in the data. That's what your smart buddy means by "understanding". That is a high dimensional model of the data set which infers abstract semantic relationships / concepts. But he would not claim that those semantic relationships are exactly the same as any human interpretation of the data (which you refer to as the "real" response). Language models also have limited reasoning abilities. They are capable of misunderstanding as much as understanding.

PaulDavisThe1st 4y ago
Not bad, but: > Language models also have limited reasoning abilities. They have no reasoning abilities at all.
- Mike_12345 4y ago
  "Language Models Perform Reasoning via Chain of Thought" Posted by Jason Wei and Denny Zhou, Research Scientists, Google Research, Brain team https://ai.googleblog.com/2022/05/language-models-perform-reasoning-via.html https://ai.googleblog.com/2022/05/language-models-perform-re...
  - PaulDavisThe1st 4y ago
    This paper is misusing the term "reasoning" in my opinion. At no point does the LLM know that 5+6 = 11, and if asked to solve a problem in which 5+6 was an implicit component of the solution but not explicitly present in the text, it would be completely lost.
    - deleted 4y ago
      [deleted]
    - Mike_12345 4y ago
      You have a narrow definition of reasoning. Formally and technically it is solving a symbolic reasoning task through a sequence of steps. Yes we know it's not conscious and not human reasoning. > At no point does the LLM know that 5+6 = 11 Does it need to "know" that (by your narrow definition of "know") in order to reason about a word math problem? > if asked to solve a problem in which 5+6 was an implicit component of the solution but not explicitly present in the text, it would be completely lost Can you provide an example? What makes you believe it can't be trained to solve those too? That's just a higher abstraction over the language. Add more layers, more training, etc. Many humans cannot solve basic math word puzzles that this artificial neural network can already solve.

flangola7 4y ago

> My standing argument is that they wouldn't hallucinate incorrect responses if they understood anything, the hallucination is when their approximation of what a real response would be falls short. > These emergent abilities are not actually that, but a result of humans' poor understanding of cognition and communication. My counterargument is that humans hallucinate too, and often. As just one small example, eyewitness testimony is stupifyingily unreliable. Neurological research and even basic behavioral research shows our brains act as bullshit machines fabricating satisfying narratives constantly. Not to even get into the fact that the word hallucination still has a non-AI meaning, and that dreams exist. As I put see it GPT models simply hallucinate more often and, more noticably, in a different manner than humans. The hallucination frequency need not reach zero, only human equivalent or better , and GPT-4 is already much better than GPT-3. I agree with everything else you said fully. "True" reasoning machines or not, society will be catastrophically destabilized. Amongst the chaos I expect plentiful of "normal" conventional and nuclear war to go on.

ChatGTP 4y ago
What's the difference between "hallucinating" and just getting something wrong? Hallucinations are where you hear, see, smell, taste or feel things that appear to be real but only exist in your mind. Get medical help if you or someone else have hallucinations. Why do we say LLMs hallucinate and why do people keep parroting the same thing "oh humans hallucinate", do we really hallucinate all the time? I can think of only one time a healthy adult hallucinates and it's not sitting around drinking a cup of tea.
- SanderNL 4y ago
  The word has a slightly different meaning here (confabulation) and more akin to the witness example: confidently telling what you think is true, but it turned out to be complete shit. It’s not about humans intentionally lying or truly hallucinating like on drugs. It’s about confidently thinking they are right about something which turns out not to be true. I mean, if I say it like this is like humanity’s core business.
  - 4y ago

p-e-w 4y ago

> My standing argument is that they wouldn't hallucinate incorrect responses if they understood anything Alexander Grothendieck, one of the greatest mathematicians of the 20th century, answered "57" when asked in a public lecture to provide an example of a prime number. 57 = 3 * 19 is not a prime number. According to your argument, this would imply Grothendieck did not understand what a prime number is. Which is laughable.

deafpolygon 4y ago
Humans are not infallible. It's a mistake to anthromorphize LLM.
- SanderNL 4y ago
  LLM wrong => unredeemable and fatally flawed human wrong => no problem, mistakes happen
  - lewhoo 4y ago
    Yeah, a little bit. Our biology makes us unpredictable to a certain degree. In simple terms we're allowed to have a bad or a good day. LLM's remain unchangeable and simple mistakes they make can't be explained in the same manner I think.
    - SanderNL 4y ago
      I get the reflex, but this becomes increasingly untenable as their complexity balloons. (Our "biology" is some sort of code for "it's complex".)
    - gmerc 4y ago
      But LLMs are not unchangable, there are several mechanisms available by now to drive change and learning (especially as context windows improve) Practically for outcomes it doesn’t matter if that learning is happening in the LLM or on a layer above it.
  - 4y ago

nl 4y ago

> My standing argument is that they wouldn't hallucinate incorrect responses if they understood anything, the hallucination is when their approximation of what a real response would be falls short. This doesn't seem to make sense. If anything the opposite is true - if the things that are hallucinated make sense (even if not true) it means there is some "understanding" or a world model.

_8j50 4y ago
No, hallucinations are similar to but not quite the expected result which shows they are approximations. For example, you tell Dall-E to draw a picutre of a man smoking a pipe but then it draws the pipe coming out of the mans butt and instead of a head the man has a leg on his neck. This is approximation. Now if the pipe looked wrong or it was a trumpet or the guy's head looked weird but still a head maybe it knows what a head is and it just made a mistake and it knows that the pipe goes into the mouth so it will be somewhere near the face. Undesrstanding is a 3yr old child drawing terribly, approximarion is drawing really well but all wrong. All this applies to LLMs I used Dall-E because pictures are easier to talk about.
- nl 4y ago
  I haven't used dall-e much but I've never seen stable diffusion or midjourney make an error like that, unless of course deliberately promoted. You can see this because of the big deal people made about image gen tools getting hands wrong: it was the most significant error that was systematically occuring.
  - _8j50 4y ago
    There us just something about the mistakes. Let me put it thid way, it nails down hands and faces great most of the time with stable diffusion for example, why would that ever be an issue at any point if it understood what hands and faces were? If I didn't understand what a hand was or how to draw it,I would never get it right. But if I do get it right quite a bit because I understand what the object is, then it makes no sense for me to have a significant error rate where hands and faces are deformed. An artist who knows how to draw hands and faces will never make that mistake, especially when self-correcting is so easy. The only explanation is that it is approximating based on what it learned from the large swath of training data. Kind of like if a human remembered answers to a multiplication table by memorizing every input,output and trend as opposed to knowing how to process the data within the rules of math and generate output. LLMs imo don't understand the rules of language and context they only approximate what a rule compliant system should output.