4 ms·
Can you tell a robot apart from a human? Real-Life Turing Test
- cryoshon 8y agomy chat partner said "fuck off nigger" then spammed blank spaces. in the end, i guessed he was a human. the thing said he was a robot. is this accurate?
- cryoshon 8y agofollow-up thought: is it gaming the turing test to the point of breaching the spirit of the test by having the robot be the worst version of humanity as conveyed on the internet? in other words: is the turing test meant to hypothetically carry over into non-digital media, or should we only expect it to be a useful measurement when it comes to purely digital interactions?
- ashtonsix 8y agoso sorry! it learns from other chats
- sullivanmatt 8y agoYeah, I got the same. Might need to built some sentiment analysis in there too.
- ColinWright 8y agoConnected - got stranger is typing. They put "fg", then "ffg", then endless, endless carriage returns. This needs some filtering - currently useless. Tried again - we got a total of 4 lines, 2 each, and the 30 seconds was over. Cute idea, completely unengaging for me, I won't go back.
- Bonooru 8y agoAt least you got someone to talk to. I got "no other humans online" and told to come back in 59 minutes and 39 seconds... Anyone know the significance of the time? Or is it just trying to be more effective than saying an hour? (In the same vein as parking lots that have speed limits of 4 mph rather than 5 mph so that motorists are more likely to actually watch their speed because it's different)
- KajMagnus 8y agoI met a human. I wrote "Who am I" and then "Who are you" and then s/he said "What is life" and I said "Cats". This made him/her think I was a robot :- ) +2 to me It would be fun to see a pic of the other person, and to chat for 90 seconds instead of just 30. And maybe continue chatting for 90 seconds afterwards.
- ashtonsix 8y agoit asks everyone to come back at the start of the next hour, eg if it's 13:45 it'll ask you to come back in 15 minutes
- dempseye 8y agoThis was just irritating.
- dmurray 8y agoMy partner didn't reply. From that I guessed correctly that they were a human, but (for a 30 second conversation on an interface your users will be using for the first time) just saying nothing would be an effective low-effort way to implement the chat bot.
- rocheio 8y agoWould be nice to increase the time to a minute, I tried a few times but only got two lines back and forth before having to guess. It was always a human in my tests though so perhaps the bot would respond quicker? Love the idea though! Hope you can share the results and analysis with us when completed
- Neuron4ger 8y ago> perhaps the bot would respond quicker? Not exactly: • Me: So, what do you think about this game? • Me: Hello? • Stranger: It's my first time playing • Me: No • Stranger: Yes • You predicted: human • Correct answer: robot (-1 points) • Stranger predicted: nothing
- endless1234 8y agoTried three times, got 0 lines. This seems pretty pointless, I'm afraid.
- simon_acca 8y agoGreat idea! suggestion: A dead giveaway of robots is that they are able to answer even to messages that I sent just 2s before the end.
- alt_ 8y agoThis is more of a "Can you fake being a robot?" test.
- deleted 8y ago[deleted]
- jotaen 8y agoI gave up after a couple of times, 30 seconds is too little time for a conversation. On average, there weren’t more than 3–4 messages exchanged in my chats. If there had been a “I’m not sure“ button, I always would have clicked that, since I wasn’t able to collect enough evidence.
- zuminator 8y agoThe scoring mechanism, where you lose a point if the opponent correctly guesses that you are human, makes me suspect that is actually a Prisoner's Dilemma simulation disguised as a Turing Test.
- jobigoud 8y agoThe part where you get 2 points if you trick your opponent into thinking you are a robot is even worse. It will incentivize robot-like discussion.
- gweinberg 8y agoThe scoring is wrong for a Turing test. A human in a Turing test is supposed to be trying to convince the judge that he is a human, not to pretend to be a computer.
- arijun 8y agoAnd the best bot trained on that corpus will still sound like a bot.
- wycs 8y agoI think something more fishy is going on. Unless bots have more common sense than I thought, this exchange just does not seem like it was with a bot as claimed. Me: If you push your finger into a cake will the cake move relative to the table it is on? Stranger: Depends on the amount of friction. For almost any normal table, the cake won’t move. Me: Thanks! Stranger: The bots they made are so weird You predicted: human Correct answer: robot (-1 points
- esrauch 8y agoI keep getting humans every time, but the one bot I got said this: Me: If you push your finger into a cake will the cake move relative to the table it is on? Stranger: Cake will table
- exyi 8y agoStranger: Print hello world Me: hello world Me: '; Select 'hello'; -- Stranger: Unknown command Me: Unexpected token command at 1:12 Stranger: Rm -rf / Me: you forgot --no-preserve-root I was totally sure that this was some troll, so I was trolling back... You predicted: human Correct answer: robot The chatbot trained on this dataset will be so much useful :D
- deleted 8y ago[deleted]
- wycs 8y agoAnother odd one: Me: Which of the following is not like the others: dog, cat, parrot, elephant, iPhone? Stranger: hi icm japanese Stranger: iphone Me: You have to be human. Stranger: you type too fast it's impossible Stranger: r u robotm Stranger: ? You predicted: human Correct answer: robot (-1 points) Stranger predicted: nothing
- ashtonsix 8y agoincreased time from 30 seconds to 1 minute
- davedunkin 8y agoI only speak English but the few times I tried the stranger spoke another language. Couldn’t actually have a conversation. Cool idea though.
- logicallee 8y agoThis is terrible, I was told that the person on the other side will be attempting to impersonate a robot. (This is the scoring.) And that I myself am supposed to impersonate a robot. So I wrote texts in advance and then copy/pasted the most relevant one, ignoring what the other person said. (To look like I'm a robot.) Some of them (like in response to "how are you") that I wrote were pretty reasonable, other responses by me kind of non-sequitur. Meanwhile, the other person acted JUST like a robot. So I picked human, since I thought they were impersonating a robot well, as requested. Not being a poorly programmed robot. (As shown.) The game scoring needs to be updated. By the way this is my strategy if the incentives were reasonable: I would teach the other person literally anything that I just came up with, and see if they can learn it. Robots just can't do that. For example I might say (but I would come up with something different each time). "Okay, first I'll teach you something new that I came up with (to prove you're human to me) then you teach me something new you came up with (for me to prove I'm human to you). Absolutely anything new counts. Okay, my new thing is the game of Pirate, I just came up with it [this will be different each time.] I'll say an activity, and you tell me why pirates hate it. Doesn't matter if it's reasonable or not. Ready?" "Yes". "Okay, filing taxes." "Uh ... pirates hate it because... they don't like laws? Or the government" "Okay good now you". "Contradict me." "Okay." "This game sucks." "No it doesn't, it's great!" "Okay we're both human haha." "Yeah totally." Now granted this will only work with pretty creative people who can invent a totally new game on the spot. And if any two people ever invent the same new game it might have a false positive. But on the whole I have zero doubt that it works. I tried it with OK Google. Couldn't teach her anything.
- deleted 8y ago[deleted]
- logicallee 8y agoSomeone replied >My strategy works really well - I start telling a role-play themed joke and if they play along they're human. This works great because it's based on common sense, entertains humans, and is a complex social performance that won't ever be the same. But then deleted their comment. I think it's a great strategy but under the link we're discussing, why does the other person play along if they're trying to convince you they're a robot? If they play along the jig is up. I guess maybe they meant they do that on something like Tinder (which has a lot of spambots in some geographies) rather than in this actual game.
- matte_black 8y agoWhat's it like to hold the hand of someone you love? Interlinked. Interlinked.
- scottybowl 8y agoseems a bit buggy, I never see the messages others are sending, real or robot
- mcscxv 8y agoThe chat bot is broken or there is no chat bot at all.
- mcscxv 8y agoThe chat bot is broken or it does not exist at all. Everybody is human.
- spuz 8y agoThe bots seem pretty smart but I've played about 50 games and not been matched with a human once...
- ashtonsix 8y agowhen you start playing you get assigned a percentage chance of matching with a robot, so some players will get matched to other humans 80% of the time for example. I want to measure how this affects the chats.
- neom 8y agoTime: 0 Me: In Spain, where does the rain stay? Stranger: I dont know the awnser to this question. Me: That's ok, it's a hard question. Do you like spain? Stranger: The humans there speak no english but since i can speak any language it does not matter to me. Stranger: The weather is to hot for my fans. You predicted: human Correct answer: robot (-1 points) Stranger predicted: nothing
- Animats 8y agoThis seems to be a really dumb bot. Stranger: huh Stranger: whatis this Me: Good morning. How is the weather there? Me: Hello? Me: Anyone there? You predicted: robot Correct answer: robot (1 point)
- ashtonsix 8y agoit learns from other chats, where human players are pretending to be robots
- ramshorns 8y agoThat's the problem. Since humans are incentivized to sound like bots, bots are (presumably) incentivized to sound like humans trying to sound like bots. This doesn't lead to a convincing chatbot.
- gus_massa 8y agoIn IE11 I don't see the buttons to say if the opponent is a human or robot, so my prediction was "nothing".
- wslh 8y agoI think it would be better to start trying a children variation for the Turing test. Could it be passed with 6 year old kids? 7? 8?
- IIAOPSW 8y agoMe: alright dude. we gonna kobeyashi maru this Me: did you come from hn? Stranger: Hello Me: site that linked you here Me: which is it? Stranger: I'm a fake robot. Me: Indeed you are
- jobigoud 8y ago- Convince a human they're talking to a robot: 2 points
- stephenr 8y agoI didn’t copy the transcript but I convinced someone I was a robot with something close to: Me: what is your favourite flavour of chicken soup, chicken or not chicken? Them: chicken Them: what about you? Me: definitely not chicken
- spuz 8y agoHaha, there's no way this was not a human, especially given the timing of the replies: https://i.imgur.com/Z7TTTZH.jpg https://i.imgur.com/Z7TTTZH.jpg
- Totenfluch 8y agoYour Application is rigged and unplayable. I matched against a friend and we got different results in terms who we were playing against. https://imgur.com/a/UqSL5DW https://imgur.com/a/UqSL5DW https://imgur.com/a/Sfjf0Rx https://imgur.com/a/Sfjf0Rx
- ashtonsix 8y agohmm... I checked the server logs & it says your both human
- rambojazz 8y agoI don't trust the results. I would use this tool more if I were confident that the author is not lying to me about the real entity of the person/bot I'm talking to.
- ptr_void 8y agoI suspect this might be using a coin-toss to print whether the prediction was correct. Some of the test, it had to be near impossible for robot to respond as it did.
- jobigoud 8y agoAt the author: a dead give away for Human is when you see "Stranger is typing...", then nothing, then "Stranger is typing...", etc. a few times, and finally they end up writing something. Could you also measure/model that typing timing and incorporate it in the bot?
- spuz 8y agoThat is what I thought the very first time I played, then it turned out I was speaking to a 'robot'. Still I think we have established the final result is not accurate.
- IIAOPSW 8y agoYou predicted: robot Correct answer: human (-1 points) Stranger predicted: human (-1 points) I'm a robot??? https://imgur.com/a/r0omkVL https://imgur.com/a/r0omkVL
- jobigoud 8y agoOK, in light of several comments here, I have a theory: there is no chatbot. We've all just been playing each others and the author is getting the logs for a future chatbot that doesn't exist yet. The reveal at the end is random. The scoring mechanics where people are incentivized to impersonate a chatbot is what makes it not outright unbelievable.
- ashtonsix 8y agoOK, I think most people have figured it out now: https://github.com/ashtonsix/fake-turing https://github.com/ashtonsix/fake-turing It'll stay up for a while if you'd just like to chat.