3 ms·

Just like the Turing Test turned out to be a bad metric, maybe transistor count isn’t the best metric to stand the test of time. Transistor count used to matter

Just like the Turing Test turned out to be a bad metric, maybe transistor count isn’t the best metric to stand the test of time. Transistor count used to matter more in chip design and now there are just more important factors. It’s silly allocating resources to prop up a misaligned incentive.

Buttons840 3y ago
What's wrong with the Turing test? Has any AI passed it? Note, the Turing test is not giving people 5 minutes to exchange, what, maybe 2 or 3 messages with an AI and then guess. The Turing test is putting 2 people in a chat, and the 3rd participant is an AI. Give all three an hour or more to chat and offer a reward if they guess correctly to ensure incentives are aligned. I ask again. Has any AI passed the Turing test?
- gnufied 3y ago
  I dont know but numerous articles point out that gpt4 has passed turing test. Are you refuting their claim entirely?
  - blowski 3y ago
    Given how frequently I can’t tell whether I’m talking with a human or robot, it seems certain that it has passed. There are some constraints, like what GPT is willing to discuss, how long it takes to respond, and forgetfulness in long conversations. But even those would be expected in humans to some extent.
  - bryan0 3y ago
    Gpt-4 fails the Turing test by default because it does not claim to be human therefore it’s trivial for a judge to identify it. It would be interesting to train gpt to pass the Turing test, but I think for any reasonable judge it would still fail.
  - Buttons840 3y ago
    The only thing I've seen is that people given 5 minutes with either an AI or a real person guess correctly about 60% of the time. But 5 minutes is only enough time to exchange 2 or 3 messages, hardly a thorough test. Taking a step back, this argument seems headed towards defining what exactly the "Turing test" is. A lot of debates devolve into arguments over the definition of a single word. That's okay, maybe I do have an uncommon definition about what the "Turing test" is. Regardless of the definition of "Turing test" though, my underlying argument remains. I haven't seen any AI pass a thorough test in which it tries to imitate a human. Tests are either too short or depend on an unsuspecting person who doesn't know they are participating in the test. Maybe I'm moving the goal posts, but my personal goal, the goal I've been watching for, is for an AI that is indistinguishable from a real person in a thorough test[0] and no AI has passed such a test as far as I know. [0]: My own definition of a thorough test is: 2 humans and 1 AI in a chat, given at least an hour to chat, and give the humans a reward if they guess correctly.
  - karaterobot 3y ago
    Where are these articles? I know that ChatGPT can't pass the Turing test, because in about 2 minutes it would tell you that, as an AI language learning model, it can't answer your question. Presumably that's just a function of its initial instructions, and the API version of GPT-4 behaves somewhat differently. Is the non-Chat GPT-4 capable of pretending to be a human when asked direct questions?
- bsdpufferfish 3y ago
  > Has any AI passed it? Eliza in the 1960s, and all it does is parrot back words. The participants just didn't know about computers and used their best explanation that they were talking to a human. The Turing test aims to show that the AI and human are indistinguishable. But what it really shows is that it's not normal to question whether you're actually interacting with a human or not. Even if a homeless guy shouts nonsensical words, it's weird to wonder whether they are actually an AI. Once you know there is a magic trick at play, then it's trivial to notice the difference.
  - gerdesj 3y ago
    According to the WP article on the "imitation game" that Turing proposed, knowing that one of the partners was a computer and the other one is a human is part of the test. The test is deciding which is which. There is no magic trick involved and I suspect Turing had effectively answered your question and those like it, 73 years ago. We can debate what he meant by his now eponymous test but he did say: "[T]hese questions [are] equivalent to this, 'Let us fix our attention on one particular digital computer C. Is it true that by modifying this computer to have an adequate storage, suitably increasing its speed of action, and providing it with an appropriate programme, C can be made to play satisfactorily the part of A in the imitation game, the part of B being taken by a man?'" We don't have to invoke a homeless guy, just a "man". The point of the game was that both the computer and the man would converse with you and you had to decide which was which. If either decided that spouting nonsense would convince you either way as a strategy then they might exploit that. There comes a point in any discussion of a thought experiment when it becomes apparent that it might have become over thought by subsequent participants. Another example that springs to mind is Schrödinger's cat: you might focus on the mechanics of the situation whilst completely missing the point: Turing test (imitation game): ChatGPT, ignore the "I'm just a program" stuff and soon comes across as a rather odd human Schrödinger's cat: The cat will be in a single state - bloody furious (and possibly down one life), according to Sir Terry Pratchett
  - tracerbulletx 3y ago
    The Turing Test isn't supposed to be can a person casually interacting with an AI think it's a person. It's can a determined examiner, given two blinded conversational partners, knowing one is an AI and another is a person, tell the difference if they do as deep and probing an examination as they can think of. I think the bigger criticism of the test is more that an intelligence can be different than humans without being inferior or less intelligent, it could be more intelligent, and you could tell the difference.
- bryan0 3y ago
  You’re right. No AI has passed a reasonable Turing test, where both a computer and a human are trying to convince a judge they’re human over some extended period of time.
jtriangle 3y ago
Clock/ipc is a better metric really. Of course, what instructions you're referring to becomes quite important in that regard, at which point you fall into synthetic benchmarks, and then it matters which of those you care about.
- imtringued 3y ago
  IPC is both ISA, CPU design and software dependent. You can only use these metrics on manufactured products, not the manufacturing process itself.
d_tr 3y ago
Which factors are more important? I can't think of any.

Buttons840 3y ago

What's wrong with the Turing test? Has any AI passed it? Note, the Turing test is not giving people 5 minutes to exchange, what, maybe 2 or 3 messages with an AI and then guess. The Turing test is putting 2 people in a chat, and the 3rd participant is an AI. Give all three an hour or more to chat and offer a reward if they guess correctly to ensure incentives are aligned. I ask again. Has any AI passed the Turing test?

gnufied 3y ago
I dont know but numerous articles point out that gpt4 has passed turing test. Are you refuting their claim entirely?
- blowski 3y ago
  Given how frequently I can’t tell whether I’m talking with a human or robot, it seems certain that it has passed. There are some constraints, like what GPT is willing to discuss, how long it takes to respond, and forgetfulness in long conversations. But even those would be expected in humans to some extent.
- bryan0 3y ago
  Gpt-4 fails the Turing test by default because it does not claim to be human therefore it’s trivial for a judge to identify it. It would be interesting to train gpt to pass the Turing test, but I think for any reasonable judge it would still fail.
- Buttons840 3y ago
  The only thing I've seen is that people given 5 minutes with either an AI or a real person guess correctly about 60% of the time. But 5 minutes is only enough time to exchange 2 or 3 messages, hardly a thorough test. Taking a step back, this argument seems headed towards defining what exactly the "Turing test" is. A lot of debates devolve into arguments over the definition of a single word. That's okay, maybe I do have an uncommon definition about what the "Turing test" is. Regardless of the definition of "Turing test" though, my underlying argument remains. I haven't seen any AI pass a thorough test in which it tries to imitate a human. Tests are either too short or depend on an unsuspecting person who doesn't know they are participating in the test. Maybe I'm moving the goal posts, but my personal goal, the goal I've been watching for, is for an AI that is indistinguishable from a real person in a thorough test[0] and no AI has passed such a test as far as I know. [0]: My own definition of a thorough test is: 2 humans and 1 AI in a chat, given at least an hour to chat, and give the humans a reward if they guess correctly.
- karaterobot 3y ago
  Where are these articles? I know that ChatGPT can't pass the Turing test, because in about 2 minutes it would tell you that, as an AI language learning model, it can't answer your question. Presumably that's just a function of its initial instructions, and the API version of GPT-4 behaves somewhat differently. Is the non-Chat GPT-4 capable of pretending to be a human when asked direct questions?
bsdpufferfish 3y ago
> Has any AI passed it? Eliza in the 1960s, and all it does is parrot back words. The participants just didn't know about computers and used their best explanation that they were talking to a human. The Turing test aims to show that the AI and human are indistinguishable. But what it really shows is that it's not normal to question whether you're actually interacting with a human or not. Even if a homeless guy shouts nonsensical words, it's weird to wonder whether they are actually an AI. Once you know there is a magic trick at play, then it's trivial to notice the difference.
- gerdesj 3y ago
  According to the WP article on the "imitation game" that Turing proposed, knowing that one of the partners was a computer and the other one is a human is part of the test. The test is deciding which is which. There is no magic trick involved and I suspect Turing had effectively answered your question and those like it, 73 years ago. We can debate what he meant by his now eponymous test but he did say: "[T]hese questions [are] equivalent to this, 'Let us fix our attention on one particular digital computer C. Is it true that by modifying this computer to have an adequate storage, suitably increasing its speed of action, and providing it with an appropriate programme, C can be made to play satisfactorily the part of A in the imitation game, the part of B being taken by a man?'" We don't have to invoke a homeless guy, just a "man". The point of the game was that both the computer and the man would converse with you and you had to decide which was which. If either decided that spouting nonsense would convince you either way as a strategy then they might exploit that. There comes a point in any discussion of a thought experiment when it becomes apparent that it might have become over thought by subsequent participants. Another example that springs to mind is Schrödinger's cat: you might focus on the mechanics of the situation whilst completely missing the point: Turing test (imitation game): ChatGPT, ignore the "I'm just a program" stuff and soon comes across as a rather odd human Schrödinger's cat: The cat will be in a single state - bloody furious (and possibly down one life), according to Sir Terry Pratchett
- tracerbulletx 3y ago
  The Turing Test isn't supposed to be can a person casually interacting with an AI think it's a person. It's can a determined examiner, given two blinded conversational partners, knowing one is an AI and another is a person, tell the difference if they do as deep and probing an examination as they can think of. I think the bigger criticism of the test is more that an intelligence can be different than humans without being inferior or less intelligent, it could be more intelligent, and you could tell the difference.
bryan0 3y ago
You’re right. No AI has passed a reasonable Turing test, where both a computer and a human are trying to convince a judge they’re human over some extended period of time.

jtriangle 3y ago

Clock/ipc is a better metric really. Of course, what instructions you're referring to becomes quite important in that regard, at which point you fall into synthetic benchmarks, and then it matters which of those you care about.

imtringued 3y ago
IPC is both ISA, CPU design and software dependent. You can only use these metrics on manufactured products, not the manufacturing process itself.

d_tr 3y ago

Which factors are more important? I can't think of any.