4 ms·

> How can you not understand the difference between "humans are not absolutely perfect or reliable at this task" and "LLMs by their very nature cannot perform t

> How can you not understand the difference between "humans are not absolutely perfect or reliable at this task" and "LLMs by their very nature cannot perform this task"? This is a very good distillation of one side of it. What LLMs have taught us is a superficial grasp of language is good enough to reproduce a shocking proportion of what society has come to view as intelligent behaviors. i.e. it seems quite plausible a whole load of those people failing to grasp the point you are making are doing so because their internal models of the universe are closer to those of LLMs than you might want to think.

godelski 2y ago
I think we already knew this though. Because the Turing test was passed by Eliza in the 1960's. PARRY was even better and not even a decade later. For some reason people still talk about Chess performance as if Deep Blue didn't demonstrate this. Hell, here's even Feynman talking about many of the same things we're discussing today, but this was in the 80's https://www.youtube.com/watch?v=EKWGGDXe5MA https://www.youtube.com/watch?v=EKWGGDXe5MA
- fidotron 2y ago
  Ten years ago I was explaining to halls of appalled academic administrators that AI would be replacing them before a robot succeeds in sorting out their socks.
  - eli_gottlieb 2y ago
    The field of AI needs to be constantly retaught the lesson that being able to replace important and powerful people doesn't mean your AI is actually intelligent. It means that important and powerful people were doing bullshit jobs.
- matthewdgreen 2y ago
  ELIZA passed the Turing test in the same way a spooky halloween decoration can convince people that ghosts are real.
- famouswaffles 2y ago
  Eliza did not pass the Turing Test in any meaningful way. In fact, it did not pass it at all, and saying it did and comparing both is pretty disingenuous.
  - chipsrafferty 2y ago
    For real, even the 2023/2024 articles about chat bots passing the Turing test are pretty disingenuous. The research methods are extremely flawed.
danaris 2y ago
....But this is falling into exactly the same trap: the idea that "some people don't engage the faculties their brains do/could (with education) possess" is equivalent to the LLMs that do not and cannot possess those faculties in the first place.
AnimalMuppet 2y ago
> What LLMs have taught us is a superficial grasp of language is good enough to reproduce a shocking proportion of what society has come to view as intelligent behaviors I think that LLMs have shown that some fraction of human knowledge is encoded in the patterns of the words, and that by a "superficial grasp" of those words, you import a fairly impressive amount of knowledge without actually knowing anything. (And yes, I'm sure there are humans that do the same.) But going from that to actually knowing what the words mean is a large jump, and I don't think LLMs are at all the right direction to jump in to get there. They need at least to be paired with something fundamentally different.
- godelski 2y ago
  I think the linguists already knew this tbh and that's what Chomsky's commentary on LLMs was about. Though I wouldn't say we learned "nothing". Even confirmation is valuable in science
Yizahi 2y ago
Scary thought
dfilppi 2y ago
[dead]

godelski 2y ago

I think we already knew this though. Because the Turing test was passed by Eliza in the 1960's. PARRY was even better and not even a decade later. For some reason people still talk about Chess performance as if Deep Blue didn't demonstrate this. Hell, here's even Feynman talking about many of the same things we're discussing today, but this was in the 80's https://www.youtube.com/watch?v=EKWGGDXe5MA https://www.youtube.com/watch?v=EKWGGDXe5MA

fidotron 2y ago
Ten years ago I was explaining to halls of appalled academic administrators that AI would be replacing them before a robot succeeds in sorting out their socks.
- eli_gottlieb 2y ago
  The field of AI needs to be constantly retaught the lesson that being able to replace important and powerful people doesn't mean your AI is actually intelligent. It means that important and powerful people were doing bullshit jobs.
matthewdgreen 2y ago
ELIZA passed the Turing test in the same way a spooky halloween decoration can convince people that ghosts are real.
famouswaffles 2y ago
Eliza did not pass the Turing Test in any meaningful way. In fact, it did not pass it at all, and saying it did and comparing both is pretty disingenuous.
- chipsrafferty 2y ago
  For real, even the 2023/2024 articles about chat bots passing the Turing test are pretty disingenuous. The research methods are extremely flawed.

danaris 2y ago

....But this is falling into exactly the same trap: the idea that "some people don't engage the faculties their brains do/could (with education) possess" is equivalent to the LLMs that do not and cannot possess those faculties in the first place.

AnimalMuppet 2y ago

> What LLMs have taught us is a superficial grasp of language is good enough to reproduce a shocking proportion of what society has come to view as intelligent behaviors I think that LLMs have shown that some fraction of human knowledge is encoded in the patterns of the words, and that by a "superficial grasp" of those words, you import a fairly impressive amount of knowledge without actually knowing anything. (And yes, I'm sure there are humans that do the same.) But going from that to actually knowing what the words mean is a large jump, and I don't think LLMs are at all the right direction to jump in to get there. They need at least to be paired with something fundamentally different.

godelski 2y ago
I think the linguists already knew this tbh and that's what Chomsky's commentary on LLMs was about. Though I wouldn't say we learned "nothing". Even confirmation is valuable in science

Yizahi 2y ago

Scary thought

dfilppi 2y ago

[dead]