4 ms·

I think the turing test suffers a bit from the "when a measurement becomes a target, it ceases to be a good measurement." An AI that happened to be able to pas

I think the turing test suffers a bit from the "when a measurement becomes a target, it ceases to be a good measurement." An AI that happened to be able to pass the turing test would be pretty notable because it probably implies much more capabilities behind the scenes. The problem with, for example, LLMs, they're essentially optimized turing test takers. That's about all they can do. Plus, I don't think any LLM will pass the turing test in the long term. Once something organically comes up that they aren't good at, it'll be fairly obvious they aren't human and the limits of context will also become apparent eventually.

mikepurvis 1y ago
You can also be interrogating a human and in the course of your conversation stumble across something it isn’t good at.
- taormina 1y ago
  See, humans respond very differently when that happens. The failure to do what humans do when they don’t understand something or know something is frequently what fails LLMs at the TT.
- geysersam 1y ago
  Sure, but very likely they'll be able to explain their lack to you in a satisfactory way, or, at least in a way that makes you think they're human.
  - ben_w 1y ago
    Counterpoint: people were accusing each other of being bots simply for disagreeing with each other even back when Twitter was still called that. "Mr Firstname Bunchanumbers" etc. (And we've been bemoaning "the lack of common sense these days" for at least as long as I've been an adult, and racists and sexists have been denying the intelligence of the outgroup as far back as writing can show us).
  - mikepurvis 1y ago
    IMO this is a solvable problem though. Eventually LLMs will have more awareness of their own confidence and will be able to convincingly say “huh, I’m honestly not sure about that, can you explain a bit more about what you mean?” Or even “I’ve heard of X before but not in this context; can you please clarify what you mean here?”

mikepurvis 1y ago

You can also be interrogating a human and in the course of your conversation stumble across something it isn’t good at.

taormina 1y ago
See, humans respond very differently when that happens. The failure to do what humans do when they don’t understand something or know something is frequently what fails LLMs at the TT.
geysersam 1y ago
Sure, but very likely they'll be able to explain their lack to you in a satisfactory way, or, at least in a way that makes you think they're human.
- ben_w 1y ago
  Counterpoint: people were accusing each other of being bots simply for disagreeing with each other even back when Twitter was still called that. "Mr Firstname Bunchanumbers" etc. (And we've been bemoaning "the lack of common sense these days" for at least as long as I've been an adult, and racists and sexists have been denying the intelligence of the outgroup as far back as writing can show us).
- mikepurvis 1y ago
  IMO this is a solvable problem though. Eventually LLMs will have more awareness of their own confidence and will be able to convincingly say “huh, I’m honestly not sure about that, can you explain a bit more about what you mean?” Or even “I’ve heard of X before but not in this context; can you please clarify what you mean here?”