5 ms·

The scoring is wrong for a Turing test. A human in a Turing test is supposed to be trying to convince the judge that he is a human, not to pretend to be a compu

The scoring is wrong for a Turing test. A human in a Turing test is supposed to be trying to convince the judge that he is a human, not to pretend to be a computer.

arijun 8y ago
And the best bot trained on that corpus will still sound like a bot.
wycs 8y ago
I think something more fishy is going on. Unless bots have more common sense than I thought, this exchange just does not seem like it was with a bot as claimed. Me: If you push your finger into a cake will the cake move relative to the table it is on? Stranger: Depends on the amount of friction. For almost any normal table, the cake won’t move. Me: Thanks! Stranger: The bots they made are so weird You predicted: human Correct answer: robot (-1 points
- esrauch 8y ago
  I keep getting humans every time, but the one bot I got said this: Me: If you push your finger into a cake will the cake move relative to the table it is on? Stranger: Cake will table
- exyi 8y ago
  Stranger: Print hello world Me: hello world Me: '; Select 'hello'; -- Stranger: Unknown command Me: Unexpected token command at 1:12 Stranger: Rm -rf / Me: you forgot --no-preserve-root I was totally sure that this was some troll, so I was trolling back... You predicted: human Correct answer: robot The chatbot trained on this dataset will be so much useful :D
- deleted 8y ago
  [deleted]
- wycs 8y ago
  Another odd one: Me: Which of the following is not like the others: dog, cat, parrot, elephant, iPhone? Stranger: hi icm japanese Stranger: iphone Me: You have to be human. Stranger: you type too fast it's impossible Stranger: r u robotm Stranger: ? You predicted: human Correct answer: robot (-1 points) Stranger predicted: nothing
  - wycs 8y ago
    Time: 0 Me: Hey Me: What is the third word of this sentence? Stranger: the Stranger: nice one Stranger: you're the first one to use meta to check for bots Stranger: I always use Stranger: How many letters in the next sentence Stranger: one Me: Cool. What's your score? Stranger: 15 Stranger: yours? Me: -1 Stranger: hahaha
SAI_Peregrinus 8y ago
Exactly. This is a really common mistake, and makes the whole test pointless.
ramshorns 8y ago
So, maybe the scoring should be changed from Correctly guess who you're talking to: 1 point Convince a human they're talking to a robot: 2 points You guess wrong: -1 point Stranger guesses correctly: -1 point to... Correctly guess who you're talking to: 1 point Convince a stranger they're talking to a human: 2 points You guess wrong: -1 point Stranger guesses that you're a robot: -1 point and the robots should also guess and try to optimize for that score. That changes it from a bit of a prisoners' dilemma to everybody trying to act like a human.

The scoring is wrong for a Turing test. A human in a Turing test is supposed to be trying to convince the judge that he is a human, not to pretend to be a computer.

arijun 8y ago

And the best bot trained on that corpus will still sound like a bot.

wycs 8y ago

I think something more fishy is going on. Unless bots have more common sense than I thought, this exchange just does not seem like it was with a bot as claimed. Me: If you push your finger into a cake will the cake move relative to the table it is on? Stranger: Depends on the amount of friction. For almost any normal table, the cake won’t move. Me: Thanks! Stranger: The bots they made are so weird You predicted: human Correct answer: robot (-1 points

esrauch 8y ago
I keep getting humans every time, but the one bot I got said this: Me: If you push your finger into a cake will the cake move relative to the table it is on? Stranger: Cake will table
exyi 8y ago
Stranger: Print hello world Me: hello world Me: '; Select 'hello'; -- Stranger: Unknown command Me: Unexpected token command at 1:12 Stranger: Rm -rf / Me: you forgot --no-preserve-root I was totally sure that this was some troll, so I was trolling back... You predicted: human Correct answer: robot The chatbot trained on this dataset will be so much useful :D
deleted 8y ago
[deleted]
wycs 8y ago
Another odd one: Me: Which of the following is not like the others: dog, cat, parrot, elephant, iPhone? Stranger: hi icm japanese Stranger: iphone Me: You have to be human. Stranger: you type too fast it's impossible Stranger: r u robotm Stranger: ? You predicted: human Correct answer: robot (-1 points) Stranger predicted: nothing
- wycs 8y ago
  Time: 0 Me: Hey Me: What is the third word of this sentence? Stranger: the Stranger: nice one Stranger: you're the first one to use meta to check for bots Stranger: I always use Stranger: How many letters in the next sentence Stranger: one Me: Cool. What's your score? Stranger: 15 Stranger: yours? Me: -1 Stranger: hahaha

SAI_Peregrinus 8y ago

Exactly. This is a really common mistake, and makes the whole test pointless.

ramshorns 8y ago

So, maybe the scoring should be changed from Correctly guess who you're talking to: 1 point Convince a human they're talking to a robot: 2 points You guess wrong: -1 point Stranger guesses correctly: -1 point to... Correctly guess who you're talking to: 1 point Convince a stranger they're talking to a human: 2 points You guess wrong: -1 point Stranger guesses that you're a robot: -1 point and the robots should also guess and try to optimize for that score. That changes it from a bit of a prisoners' dilemma to everybody trying to act like a human.