5 ms·

> Everyone seems to think we've magically jumped from ML into _real_ AI But we have: - LLMs pass the Turing test more convincingly than anything before -

by sshine 2y ago

> Everyone seems to think we've magically jumped from ML into _real_ AI But we have: - LLMs pass the Turing test more convincingly than anything before - LLMs are vastly more popular than any AI/ML methods before it - "But they don't reason!" -- we're advancing chain of thought LLMs are AI (behaving artificially human) LLMs are also ML (using statistical methods)

bluefirebrand 2y ago
> LLMs pass the Turing test more convincingly than anything before I know this sounds like moving the goalposts, but maybe all this does is show that the Turing Test was not sufficient? > LLMs are vastly more popular than any AI/ML methods before it Are we back in high school? Popularity contests?
- lupire 2y ago
  The Turing test was never a deep thesis. It was an offhand illustration of the challenge facing AI efforts, citing an example of something that was clearly impossible with technology of the day.
enugu 2y ago
The Turing test is meant to see if computers can functionally replicate human thought (just focussing on textual conversations aspect for simplicity). The implications of passing the Turing Test are profound. Tricking a user with a 5 minute conversation isn't the same. The examiner is allowed to be an expert questionner. For one thing, software engineering would quickly be taken over as a client can just chat with an AI to get the code. That is far from happening, even with all the recent LLM advances. Similarly for literature, science etc. Currently, you can detect a difference between a competent human & machine in all these fields just by text chat. Another way of saying this is that the Turing Test is AI-complete and is a test for AGI.
- SAI_Peregrinus 2y ago
  And there's a more general sort of Turing test: Just like the original test, there are 3 participants in a test: an examiner, a human, and an AI. The examiner may ask any question or series of questions to the human &/or AI, and receive responses. This is done via text or other means which prevent the examiner from directly seeing the body of responder. Both AI and human are to attempt to convince the examiner that they're the human. Unlike the original test, it then needs to repeat with different examiners. If there exists an examiner which can distinguish an AI from a human (for all humans) in a Turing test with probability non-negligibly different from 50%, then the AI fails this meta-Turing Test. Just because an AI can fool some people doesn't mean it's a human-equivalent intelligence.
  - TeMPOraL 2y ago
    Your "meta-Turing Test" is rigged. There are 8 billion humans on this planet, statistically someone is bound to be able to distinguish AI from a human with near-100% accuracy across a reasonable amount of rounds through sheer chance. Existence is too strong a criterion. Even correcting for random chance, if there's 1, 10, 100, even 1 000 000 examiners who could distinguish an AI from a human with high probability, that steal means the AI passed the test in practice, because even a million examiners is still a rounding error compared to the human population.
- sshine 2y ago
  It's been a while since I read the original article, https://courses.cs.umbc.edu/471/papers/turing.pdf https://courses.cs.umbc.edu/471/papers/turing.pdf > The Imitation Game > "Can machines think?" We may need to redefine what these words mean. Whether machines can think, and whether we believe so, are distinct. If we're just a kind of machine, we can just say that LLMs have not surpassed us. While LLMs still give away obvious signatures (their flaws are LLM-y and not humanlike), they play the imitation game better than any other machines before them. I lived long enough and played enough with simpler chatbots to believe that the simple text-based imitation game that Turing suggested will be won by computers before machines achieve reasoning ability that surpass the smartest humans. > [...] the Turing Test is AI-complete and is a test for AGI. Since Turing never used the term AGI, this is an interpretation. Since we cannot determine whether a machine can think by comparing their anatomy to ours and drawing a conclusion that whatever we do, they must do the same, we are bound to compare their ability to imitate us. Which means we can't ultimately know, we can just feel ourselves convinced. So there's no objective point at which to place the goalpost. So constantly moving it is an expression of acclimatisation. I'm impressed by LLMs, in spite of the hype and the homework cheating. If anything, the ability of LLMs to imitate human labor shows how far from thinking a lot of human behavior is.
  - namaria 2y ago
    > If we're just a kind of machine That is not true. Even our cells, with features that look a lot like machinery such as a proton pump, is on the whole several orders of magnitude more complex than any machine. Even a single human cell is more like an ecosystem than a machine. Let alone entire humans or even just the human brain. Consider that both cells and humans are capable of reproduction. > If anything, the ability of LLMs to imitate human labor Within extremely narrow confines and quite often going over into just conjuring up nonsense.
    - sshine 2y ago
      > a single human cell is more like an ecosystem than a machine Good point. Not to mention the massive reliance on organisms like mitochondria and bacteria that don’t even share the host’s DNA. > [imitate human labor] within extremely narrow confines and quite often going over into just conjuring up nonsense Much like human labor. So the intelligent, reflective and thoroughly iterated work is hardly replicated at all, and the poorly imitated, easily repeatable coursework and boring paper sludgework excellently so. So we don’t just get to criticise LLMs for not actually being intelligent. We similarly get to criticise humans for not being so when we might think we are, either.
dspillett 2y ago
> LLMs pass the Turing test more convincingly than anything before Being able to converse (or convince someone the conversation is natural) is not the same as being able to perform a technical task well. The same applies to humans as well, of course, which perhaps brings us back to the outsourcing comparison — how often does that work out great or not? > LLMs are vastly more popular than any AI/ML methods before it Popular is not what I'm looking for. I've seen the decisions the general public make elsewhere, I'm not letting them chose how I do my job :) > "But they don't reason!" -- we're advancing chain of thought Call me back when we've advanced chain of thought much more than is currently apparent, and again: for being correct in technical matters, not conversation.

by sshine 2y ago

bluefirebrand 2y ago

> LLMs pass the Turing test more convincingly than anything before I know this sounds like moving the goalposts, but maybe all this does is show that the Turing Test was not sufficient? > LLMs are vastly more popular than any AI/ML methods before it Are we back in high school? Popularity contests?

lupire 2y ago
The Turing test was never a deep thesis. It was an offhand illustration of the challenge facing AI efforts, citing an example of something that was clearly impossible with technology of the day.

enugu 2y ago

The Turing test is meant to see if computers can functionally replicate human thought (just focussing on textual conversations aspect for simplicity). The implications of passing the Turing Test are profound. Tricking a user with a 5 minute conversation isn't the same. The examiner is allowed to be an expert questionner. For one thing, software engineering would quickly be taken over as a client can just chat with an AI to get the code. That is far from happening, even with all the recent LLM advances. Similarly for literature, science etc. Currently, you can detect a difference between a competent human & machine in all these fields just by text chat. Another way of saying this is that the Turing Test is AI-complete and is a test for AGI.

SAI_Peregrinus 2y ago
And there's a more general sort of Turing test: Just like the original test, there are 3 participants in a test: an examiner, a human, and an AI. The examiner may ask any question or series of questions to the human &/or AI, and receive responses. This is done via text or other means which prevent the examiner from directly seeing the body of responder. Both AI and human are to attempt to convince the examiner that they're the human. Unlike the original test, it then needs to repeat with different examiners. If there exists an examiner which can distinguish an AI from a human (for all humans) in a Turing test with probability non-negligibly different from 50%, then the AI fails this meta-Turing Test. Just because an AI can fool some people doesn't mean it's a human-equivalent intelligence.
- TeMPOraL 2y ago
  Your "meta-Turing Test" is rigged. There are 8 billion humans on this planet, statistically someone is bound to be able to distinguish AI from a human with near-100% accuracy across a reasonable amount of rounds through sheer chance. Existence is too strong a criterion. Even correcting for random chance, if there's 1, 10, 100, even 1 000 000 examiners who could distinguish an AI from a human with high probability, that steal means the AI passed the test in practice, because even a million examiners is still a rounding error compared to the human population.
sshine 2y ago
It's been a while since I read the original article, https://courses.cs.umbc.edu/471/papers/turing.pdf https://courses.cs.umbc.edu/471/papers/turing.pdf > The Imitation Game > "Can machines think?" We may need to redefine what these words mean. Whether machines can think, and whether we believe so, are distinct. If we're just a kind of machine, we can just say that LLMs have not surpassed us. While LLMs still give away obvious signatures (their flaws are LLM-y and not humanlike), they play the imitation game better than any other machines before them. I lived long enough and played enough with simpler chatbots to believe that the simple text-based imitation game that Turing suggested will be won by computers before machines achieve reasoning ability that surpass the smartest humans. > [...] the Turing Test is AI-complete and is a test for AGI. Since Turing never used the term AGI, this is an interpretation. Since we cannot determine whether a machine can think by comparing their anatomy to ours and drawing a conclusion that whatever we do, they must do the same, we are bound to compare their ability to imitate us. Which means we can't ultimately know, we can just feel ourselves convinced. So there's no objective point at which to place the goalpost. So constantly moving it is an expression of acclimatisation. I'm impressed by LLMs, in spite of the hype and the homework cheating. If anything, the ability of LLMs to imitate human labor shows how far from thinking a lot of human behavior is.
- namaria 2y ago
  > If we're just a kind of machine That is not true. Even our cells, with features that look a lot like machinery such as a proton pump, is on the whole several orders of magnitude more complex than any machine. Even a single human cell is more like an ecosystem than a machine. Let alone entire humans or even just the human brain. Consider that both cells and humans are capable of reproduction. > If anything, the ability of LLMs to imitate human labor Within extremely narrow confines and quite often going over into just conjuring up nonsense.
  - sshine 2y ago
    > a single human cell is more like an ecosystem than a machine Good point. Not to mention the massive reliance on organisms like mitochondria and bacteria that don’t even share the host’s DNA. > [imitate human labor] within extremely narrow confines and quite often going over into just conjuring up nonsense Much like human labor. So the intelligent, reflective and thoroughly iterated work is hardly replicated at all, and the poorly imitated, easily repeatable coursework and boring paper sludgework excellently so. So we don’t just get to criticise LLMs for not actually being intelligent. We similarly get to criticise humans for not being so when we might think we are, either.

dspillett 2y ago

> LLMs pass the Turing test more convincingly than anything before Being able to converse (or convince someone the conversation is natural) is not the same as being able to perform a technical task well. The same applies to humans as well, of course, which perhaps brings us back to the outsourcing comparison — how often does that work out great or not? > LLMs are vastly more popular than any AI/ML methods before it Popular is not what I'm looking for. I've seen the decisions the general public make elsewhere, I'm not letting them chose how I do my job :) > "But they don't reason!" -- we're advancing chain of thought Call me back when we've advanced chain of thought much more than is currently apparent, and again: for being correct in technical matters, not conversation.