4 ms·

I very much doubt LLMs are the path to AGI. We just have more and more advanced "Chinese Rooms." [1] There are two gigantic risks here. One: that we assume the

I very much doubt LLMs are the path to AGI. We just have more and more advanced "Chinese Rooms." [1] There are two gigantic risks here. One: that we assume these LLMs can make reasonable decisions because they have the surface appearance of competence. Two: Their wide-spread use so spectacularly amplifies the noise (in the signal-to-noise, true fact to false fact ratio sense) that our societies cease to function correctly, because nobody "knows" anything anymore. [1] https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room

jacquesm 4y ago
The difference between AGI and a more advanced Chinese Room may not be relevant if enough people see the latter as the former. The goalposts have been moved so often now that what is and isn't intelligent behavior is no longer a bright and sharp divide. It is more like a very wide gray area and we're somewhere well into the gray by some definitions with tech people with an AI background claiming that we are still far away from it. This in contrast to similar claims by those very same people several years ago where what we take for granted today would have definitely been classified as proof of AGI. Personally I think the definition isn't all that relevant, what matters is perception of the current crop of applications by non technical people and the use that those are put to. If enough people perceive it as such and start using it as such then it may technically not be AGI but we're going to have to deal with the consequences as though it is. And those consequences may well be much worse than for an actual AGI!
- remexre 4y ago
  Well, I think a dividing line might be that if you put a Chinese Room in charge of a justice system, a corporation, or a regulatory agency, it's gonna do a pretty cruddy job of running it.
  - jacquesm 4y ago
    I don't think that is what will happen. What I do think will happen is that a lot of people in lower level functions will start to rely on these tools to help them in their every day jobs and the lack of oversight will lead to rot from within because the output of these tools will end up embedded in lots of places where it shouldn't be. And because people are not going to own up to using these tools it will be pretty hard to know which bits of 'human' output you can trust and which bits you can not. This is already happening.
    - slowmovintarget 4y ago
      Right, that's the point I was making about increased noise in the system.
- slowmovintarget 4y ago
  You'd be able to convince an AGI that is the year it is. The chat bot will be stuck on the story and statistical nearness of related responses. The chat bot can't verify, because it doesn't Know anything.

jacquesm 4y ago

The difference between AGI and a more advanced Chinese Room may not be relevant if enough people see the latter as the former. The goalposts have been moved so often now that what is and isn't intelligent behavior is no longer a bright and sharp divide. It is more like a very wide gray area and we're somewhere well into the gray by some definitions with tech people with an AI background claiming that we are still far away from it. This in contrast to similar claims by those very same people several years ago where what we take for granted today would have definitely been classified as proof of AGI. Personally I think the definition isn't all that relevant, what matters is perception of the current crop of applications by non technical people and the use that those are put to. If enough people perceive it as such and start using it as such then it may technically not be AGI but we're going to have to deal with the consequences as though it is. And those consequences may well be much worse than for an actual AGI!

remexre 4y ago
Well, I think a dividing line might be that if you put a Chinese Room in charge of a justice system, a corporation, or a regulatory agency, it's gonna do a pretty cruddy job of running it.
- jacquesm 4y ago
  I don't think that is what will happen. What I do think will happen is that a lot of people in lower level functions will start to rely on these tools to help them in their every day jobs and the lack of oversight will lead to rot from within because the output of these tools will end up embedded in lots of places where it shouldn't be. And because people are not going to own up to using these tools it will be pretty hard to know which bits of 'human' output you can trust and which bits you can not. This is already happening.
  - slowmovintarget 4y ago
    Right, that's the point I was making about increased noise in the system.
slowmovintarget 4y ago
You'd be able to convince an AGI that is the year it is. The chat bot will be stuck on the story and statistical nearness of related responses. The chat bot can't verify, because it doesn't Know anything.