3 ms·

The goalposts people usually talk about were just the relatively low expectations of the past - low at least by the standards set by ChatGPT. Before ChatGPT th

The goalposts people usually talk about were just the relatively low expectations of the past - low at least by the standards set by ChatGPT. Before ChatGPT there wasn't anything quite like it, so I think that the goalposts were mostly about passing the Turing test, or at least "not sucking" after 4 or 5 phrases. The leap from whatever existed before it was enormous, and I think comparable to the first iPhone versus, let's say, the Blackberry or Windows CE. Maybe even more than that. Go back to the first iPhone, it had a lot of issues but it offered so much as it was that people wouldn't look back. It's too much to expect ChatGPT 3 to be better an experienced emergency doctor or neurosurgeon. But the fact that it can be a competent assistant to a doctor instead of a toy assistant - that by itself is amazing.

saurik 4y ago
> ...I think that the goalposts were mostly about passing the Turing test... I mean, if that was ever the goalpost, ChatGPT does not clear it: the Turing test isn't about merely tricking someone who isn't paying attention, or making someone seriously doubt their belief system... it involved a careful discussion with a chat agent by someone who actively was trying to determine whether or not it/they were human. If you sit down with ChatGPT--even 4--and start discussing complex topics about the world with it, you absolutely can quickly get it starting to adamantly defend things about the world which a human would never assert... and, even more damning, if you just ask it "are you human?" it will reply "no", because we actively told it to fail the Turing test. If we instead merely re-define Turing's test to be "it is able to converse with a human and manages to occasionally trick a human into thinking it is also human" then I had a friend (Roy Keene) who managed to pull that off years ago with a chatbot that was literally just "every time someone messages it, wait a small random period of time and then reply with a random response from the following set: Yes, No, Maybe, I dunno, :), :(, ;P (and like one other thing I forget)". He had it set up to reply when he was not available and when I finally caught him (I asked about a task I gave him the day before which he claimed to have no memory of, so he admitted to me he had this bot) he gave me a log file and OMG I had had long conversations about personal topics with this thing and had never noticed... but only because I wasn't trying.
- trogdor 4y ago
  Sounds more like you failed the Turing test rather than your friends chatbot passing it :)
  - saurik 4y ago
    The point was that people mis-use the term "Turing test" as "manages to trick people sometimes into thinking its a human". To quote the description from Wikipedia: > Turing proposed that a human evaluator would judge natural language conversations between a human and a machine designed to generate human-like responses. The evaluator would be aware that one of the two partners in conversation was a machine, and all participants would be separated from one another. The conversation would be limited to a text-only channel, such as a computer keyboard and screen, so the result would not depend on the machine's ability to render words as speech. If the evaluator could not reliably tell the machine from the human, the machine would be said to have passed the test. The test results would not depend on the machine's ability to give correct answers to questions, only on how closely its answers resembled those a human would give. ChatGPT would not manage to pass an actual Turing test anymore than Roy's auto-responder; and, while we might get there frighteningly soon, that goalpost stands.
    - mannykannot 4y ago
      Turing was more far-seeing than many of his successors - he set up the first goalposts, and the first moving occurred in the late 1970s or 80s when crude hand-crafted rules engines were dubbed 'AI'. It was so bad that we had to invent a new term, AGI, to recover Turing's vision.

saurik 4y ago

> ...I think that the goalposts were mostly about passing the Turing test... I mean, if that was ever the goalpost, ChatGPT does not clear it: the Turing test isn't about merely tricking someone who isn't paying attention, or making someone seriously doubt their belief system... it involved a careful discussion with a chat agent by someone who actively was trying to determine whether or not it/they were human. If you sit down with ChatGPT--even 4--and start discussing complex topics about the world with it, you absolutely can quickly get it starting to adamantly defend things about the world which a human would never assert... and, even more damning, if you just ask it "are you human?" it will reply "no", because we actively told it to fail the Turing test. If we instead merely re-define Turing's test to be "it is able to converse with a human and manages to occasionally trick a human into thinking it is also human" then I had a friend (Roy Keene) who managed to pull that off years ago with a chatbot that was literally just "every time someone messages it, wait a small random period of time and then reply with a random response from the following set: Yes, No, Maybe, I dunno, :), :(, ;P (and like one other thing I forget)". He had it set up to reply when he was not available and when I finally caught him (I asked about a task I gave him the day before which he claimed to have no memory of, so he admitted to me he had this bot) he gave me a log file and OMG I had had long conversations about personal topics with this thing and had never noticed... but only because I wasn't trying.

trogdor 4y ago
Sounds more like you failed the Turing test rather than your friends chatbot passing it :)
- saurik 4y ago
  The point was that people mis-use the term "Turing test" as "manages to trick people sometimes into thinking its a human". To quote the description from Wikipedia: > Turing proposed that a human evaluator would judge natural language conversations between a human and a machine designed to generate human-like responses. The evaluator would be aware that one of the two partners in conversation was a machine, and all participants would be separated from one another. The conversation would be limited to a text-only channel, such as a computer keyboard and screen, so the result would not depend on the machine's ability to render words as speech. If the evaluator could not reliably tell the machine from the human, the machine would be said to have passed the test. The test results would not depend on the machine's ability to give correct answers to questions, only on how closely its answers resembled those a human would give. ChatGPT would not manage to pass an actual Turing test anymore than Roy's auto-responder; and, while we might get there frighteningly soon, that goalpost stands.
  - mannykannot 4y ago
    Turing was more far-seeing than many of his successors - he set up the first goalposts, and the first moving occurred in the late 1970s or 80s when crude hand-crafted rules engines were dubbed 'AI'. It was so bad that we had to invent a new term, AGI, to recover Turing's vision.