11 ms·

Turing Test Success

216 points by stevejalim 12y ago

scotty79 12y ago
Any chance of seeing conversation logs?
raldi 12y ago
I appreciate HN's rule about brief, non-sensational submission titles, but perhaps this one's taken it to the point of absurdism.
- dang 12y ago
  It was the submitter's title, not a mod edit, if that matters. The submitter did a good job, because the rest of the title ("marks milestone in computing history") is certainly linkbait and arguably misleading. Having it there would likely have made this thread a lower-quality controversy than it currently is. HN threads are surprisingly sensitive to initial conditions. Edit: Oh, and considering how many fluff press articles are showing up about this [1], the submitter also showed exemplary taste in source selection. Yay, stevejalim! 1. https://hn.algolia.com/?q=turing+test#!/story/sort_by_date/0/turing%20test https://hn.algolia.com/?q=turing+test#!/story/sort_by_date/0...
  - hanru 12y ago
    It seems the submitter has passed your Turing test. ;)
Mithaldu 12y ago
Bold claim, but without any transcripts it's impossible to verify just how close to the truth the claim comes.
- GhotiFish 12y ago
  That's true. I want to talk to Eugine. Your suggestion seems to be the very minimum.
danbruc 12y ago
Surly they did not talk to this one [1] - it is light-years away from being convincing. [1] http://www.princetonai.com/bot/ http://www.princetonai.com/bot/
- keehun 12y ago
  I was impressed it could write grammatically sound sentences, but it's not even close to human-level conversation...
  - jamesrom 12y ago
    I would be impressed if it didn't. If it made common mistakes a human would make, it would seem more human.
- owenversteeg 12y ago
  I tried talking to this one, and it failed to convince me at all. Even in the one conversation I had that was slightly believable, the bot was prone to extremely rapid emotional swings.
DanBC 12y ago
> If a computer is mistaken for a human more than 30% of the time during a series of five minute keyboard conversations it passes the test. No computer has ever achieved this, until now. Eugene managed to convince 33% of the human judges that it was human. Surely it depends on who the human judges are. It seems a bit unfair that the judges normally have IQ > 100 and the other humans have IQ > 100. I strongly suspect that some simplistic AI (alicebots, for example) would beat the Turing test if the human judges had IQ between 90 and 105. (Especially if we're using the limited 30% rule above). Getting bots running on some Facebook groups might be interesting.
radio4fan 12y ago
Awesome job of putting un-followable links in the press release, Reading University. I'm going to take this with a hug pinch of salt until I've read the transcripts, due to the involvement of famous publicity-hound Kevin Warwick.
- reillyse 12y ago
  Yes, at this stage any time I read his name involved with something I cringe.
vixin 12y ago
Lots of self-congratulation all round with no sample questions to provide the merest smidgeon of 'reason to believe that this is that significant'.
eli_gottlieb 12y ago
Managing to successfully imitate an ignorant, immature child 1/3 of the time is not what I would call a success, but rather a subversion of the entire intent behind the Turing Test in the first place.
- splike 12y ago
  I think you're probably expecting to see a 50/50 chance (or higher) of guessing it was a computer. But if you think about it, anything better than 50% would make it more human than a human. So is why the target is lower than 50%
  - chippy 12y ago
    It's not a random guess though is it? It's a sample of results.
  - GhotiFish 12y ago
    Wait, could you explain how a program that is capable of being mistaken for a human more than 50% of the time is more human than human? I'm pretty sure you're human, with a high degree of confidence, much higher than 50%. So to are you more human than human?
    - qu4z-2 12y ago
      I believe that in the original formulation of the Turing Test the judges were asked to hold two conversations: one with a computer and one with a human. Afterwards they chose which of the two they believed to be the human. In that scenario, being identified as "human" more than 50% of the time would indeed make you "more human than human"; with a "perfectly human" program the test becomes essentially a coin-toss. In that context the 30% threshold makes a good deal of sense. But in this variant, "Do you believe the entity you spoke with to be a human or a program?", a 100% threshold is theoretically achievable. That means you're entirely correct: a 51% vote of confidence is decidedly less human than human. Additionally the 30% threshold they've used is laughably low in this context. Without a control group, even an 100%-confidence outcome probably says more about the beliefs of the judges than the ability of the program to simulate a human. I don't mean to take away from the no-doubt impressive achievement by the team behind Eugene. I just take issue with the hyperbole in its reporting. But, ya know, the media will be the media, and academics gotta get research grants.
TempleOS 12y ago
God damn you have a low IQ. I get a number from NIST. If it's fake, just use a human. Retard-nigger. Tin-foil-hat nigger doesn't know what a random number is. It's fucken God, nigger.
draq 12y ago
If it was hosted by the Royal Society, wouldn't it be on their website (https://royalsociety.org/events/?type=all&direction=past https://royalsociety.org/events/?type=all&direction=past)?
- rahimnathwani 12y ago
  It was hosted _by_ the University of Reading _at_ the Royal Society. https://royalsociety.org/venue-hire/ https://royalsociety.org/venue-hire/
mratzloff 12y ago
The Turing Test is a terrible measure of sapience. Generally it involves using "average people", who have been shown time and again to be overly credulous when talking to these bots. If the test is to be used at all, it should consist of computer science experts instead--people familiar with the technology and bot tricks of the trade. That the Turing Test is still used is proof that we still don't understand how to even define sapience. Without a definition and concrete, testable qualities, how can we possibly hope to ever build artificial sapience. As a result, we continue to see these toys that are little more than parlor tricks. Any true test should include looking behind the curtain. "I know you're artificial--I can see the processes working--yet I have doubts that what I'm seeing is real." In other words, real success is the tester believing he is being fooled when he is not, rather than fooling the tester into believing it is real.
- wslh 12y ago
  I have a friend who used a sex chat bot to sell things to people. It was something very basic but incredible effective. It made me think that we need to divide the turing test for different people: by age, by need, etc.
- tritium 12y ago
  Generally it involves using "average people" [...] it should consist of computer science experts instead I don't agree. The prominent reason for "dumbing down" the judges follows the same reasoning behind decisions made regarding what constitutes "adequate" encryption. How can we gauge what will honestly happen out in the real world, today? Consider DES. It was deemed inadequate, but how to prove it? The EFF came up with a budget estimate based on what was reasonably affordable for a group of attackers, and built a machine capable of cracking DES within those budget constraints. http://w2.eff.org/Privacy/Crypto/Crypto_misc/DESCracker/HTML/19980716_eff_des_faq.html http://w2.eff.org/Privacy/Crypto/Crypto_misc/DESCracker/HTML... So, let's apply similar reasoning to the concept of AI. If a group of people were to build an AI, and use it as an adversary against ordinary people, how difficult is it to manipulate and deceive ordinary people into taking action, and is it feasible to do so with AI?
  - 12y ago
muglug 12y ago
This looks awfully like a private event for Reading University that just happens to have been held at the Royal Society. I wonder whether they even had more than three non-"celebrity" judges.
skywhopper 12y ago
Where did the 30% requirement come from? Sounds like something the contest organizers added to make it possible to "pass" without fooling 2/3 of the judges. Using a young teenager as a character also seems like a cheat unless they had other 13-year olds to be the judges. The character needs to be a peer to the judges. Most 13 year olds behave oddly enough in the opinion of most adults that it's got to be easier to credit weird behavior to generational or cultural differences. So kudos to the winners on strategy, and boo to the contest organizers for having such poor rules.
- KwanEsq 12y ago
  Most likely from here: >It will simplify matters for the reader if I explain first my own beliefs in the matter. Consider first the more accurate form of the question. I believe that in about fifty years' time it will be possible, to programme computers, with a storage capacity of about 109, to make them play the imitation game so well that an average interrogator will not have more than 70 per cent chance of making the right identification after five minutes of questioning. COMPUTING MACHINERY AND INTELLIGENCE — A. M. Turing http://loebner.net/Prizef/TuringArticle.html http://loebner.net/Prizef/TuringArticle.html
  - Buge 12y ago
    Is he talking about 109 bits? Because that seems pretty impossible to me.
    - ar7hur 12y ago
      More likely 1E9
    - DanBC 12y ago
      > As we have mentioned, digital computers fall within the class of discrete-state machines. But the number of states of which such a machine is capable is usually enormously large. For instance, the number for the machine now working at Manchester is about 2 165,000, i.e., about 10 50,000. Compare this with our example of the clicking wheel described above, which had three states. It is not difficult to see why the number of states should be so immense. The computer includes a store corresponding to the paper used by a human computer. It must be possible to write into the store any one of the combinations of symbols which might have been written on the paper. For simplicity suppose that only digits from 0 to 9 are used as symbols. Variations in handwriting are ignored. Suppose the computer is allowed 100 sheets of paper each containing 50 lines each with room for 30 digits. Then the number of states is 10 100x50x30 i.e., 10 150,000 . This is about the number of states of three Manchester machines put together. The logarithm to the base two of the number of states is usually called the "storage capacity" of the machine. Thus the Manchester machine has a storage capacity of about 165,000 and the wheel machine of our example about 1.6. If two machines are put together their capacities must be added to obtain the capacity of the resultant machine. This leads to the possibility of statements such as "The Manchester machine contains 64 magnetic tracks each with a capacity of 2560, eight electronic tubes with a capacity of 1280. Miscellaneous storage amounts to about 300 making a total of 174,380."
thinkersilver 12y ago
I remember reading the first chapter on an AI book during college which laid out the objectives of AI research into two schools of thought.The first school of thought believes that AI can be achieved by mimicking real conscious beings. Remember when man first tried to fly, most devices mimicked birds and failed horribly. The other school of thought believes that intelligence has well-defined principles, when discovered, would produce real intelligence (not mimicry), having the effect that the final product may not resemble what we see on day to day basis. Compare planes, which use the principles of flight, to the early flapping machines. A bird and plane both fly but they are very different in the way they do it. Approaching flight from the mimicry angle is hard, it's only recently in the last 10 years we have made light-weight flapping machines that fly well;yet we have had planes for over a hundred years once we knew the principles. Transcripts would be handy. I doubt a conversation with a 13 year old boy is a good way to measure AI? It's not the best metric to have but it is the most universal and most widely agreed on that we have. It seems like we are happier with small gains in mimicry for now, since real intelligence is hard. Really hard.
- acchow 12y ago
  > The first school of thought believes that AI can be achieved by mimicking real conscious beings It's important to note the true meaning behind this school of thought - that mimicry and "true" intelligence are actually equivalent. Behaviorists believe this, and the validity of the Turing Test along with it.
  - swombat 12y ago
    I think it might actually be in Civ5 that I first heard the following quote: "Asking whether a computer can think is about as useful as asking whether a submarine can swim." I think this is fairly insightful and relevant.
    - marcosdumay 12y ago
      That's from Djigstra. I digress. Most of the times, mimicking Nature works very well. We tend to always remember our most spectacular failure, but that's a selection bias.
monochr 12y ago
"a computer programme that simulates a 13 year old boy [...] If a computer is mistaken for a human more than 30% of the time during a series of five minute keyboard conversations it passes the test." In short they did nothing.
grondilu 12y ago
Did they also run the experiment with an actual 13 yo kid?
- DanBC 12y ago
  ...Imagine someone releasing bots on XBOXLive. Your task is to guess which obscenity-screaming 13 year okd is real and which is a bot. Some forms of Turing test are trivially passable with dumb enough humans.
inetsee 12y ago
Although the Turing Test is interesting, it is not, in my opinion, all that useful. I would much rather see chess program level of performance in the domain of medical diagnosis, for example. There are lots of other domains where I would be entirely happy to know that I was talking to an AI, if the answers I was getting were significantly better than most human experts in that domain.
xpose2000 12y ago
Perhaps I just asked the right questions, but I just had a lovely conversation with 'Eugene'. I asked him a few questions like where he lived, his name, if he has brothers or sisters, if he wears glasses, etc. Eventually he started asking me questions like what I did for a living and where I lived. He also managed to form questions based on my answers. It almost felt like a conversation. I can honestly say, I've never thought that before while talking to an AI. So far I am pretty impressed.
- frabcus 12y ago
  Is there somewhere online you can talk to him?
  - umanwizard 12y ago
    http://www.princetonai.com/bot/bot.jsp http://www.princetonai.com/bot/bot.jsp
    - modinfo 12y ago
      mirror? it down now
    - AndrewKemendo 12y ago
      Realize that this version is from 2001 and is not the version that was in the recent test.
    - de18de18 12y ago
      what is the united states?
    - de18de18 12y ago
      how much is 2 plus 2
      - deleted 12y ago
        [deleted]
    - deleted 12y ago
fchollet 12y ago
The Turing test is to our understanding of intelligence what sleight of hand is to our understanding of physics. Tricking people, as a goal, is not conductive to science. A researcher claiming to have passed the Turing test instantly discredits himself as a prestidigitator looking for PR buzz. The present article is a textbook example of this. As a side note, if you are focusing on disembodied, language-based human-like intelligence, then the paradigm you operate in is many decades behind. The Turing test was conceived at a time when the notion of thinking machines had just started to emerge --a very different time from today, where we have 60 years of AI research behind us. The Turing test has been irrelevant for longer than most AI researchers have been alive. I have never seen it used for anything else than smoke-and-mirrors PR operations.
- acchow 12y ago
  Your opinion is not universally accepted. (Perhaps not even mainstream?). There's an element of behaviorism which stretches back to Descartes - how can you know that I exist? That I think? You can only observe through my behavior; that my behavior mimics yours. How then can we judge machines any other way?
  - VLM 12y ago
    Your spec is over generalized into all forms of observation and comparison. The Turing test is very limited into written 1 on 1 conversation as culturally conceived by humans. There's no particular reason to assume a non-human intelligence would have a human-like culture of communication, which kind of breaks it. Here's a fun idea for several intelligence tests we can call the VLM tests that have nothing to do with two way conversation like the Turing test. Given "a machine intelligence" spin up a couple million of them in a "fun" simulation environment and see how much thermodynamic dis-equilibrium they generate by whatever social interaction they see fit to apply to each other. Is it as interesting (aka thermodynamic dis-equilibrium) as a GoL or a real world anthill or a Dwarf Fortress or a Google Earth? Is their simulated culture as interesting to read as HN, or as dumb as youtube comments (which used to be the gold standard of dumbness in social media) Assuming you can crack the literary code (if any exists) another game to play is extract the meme-flow of a culture of AI vs a culture of 4chan and vote for whichever meme came from a more intelligent group. This is Turing-ish WRT human observers majority vote and such, but is completely non-interactive, merely humans, or even trained sociologists, trying to figure out given two memes which is more intelligent. Getting out the Sherlock Holmes hat, its possible to determine if an artifact came from an intelligence without talking to the intelligence for awhile. I suspect archeologists have really fun debates on this topic. Is this a stone hammer or merely a peculiar river rock, etc.
AndrewKemendo 12y ago
You can watch the goal posts shifting for "AI" as we speak. Great result nonetheless!
deleted 12y ago
[deleted]
codeulike 12y ago
When people imagine what a Turing Test conversation would look like, they frequently underestimate the conversation. I find Dennet's example of an imaginary Turing Test from Consciousness Explained to be a good counterexample: Judge: Did you hear about the Irishman who found a magic lamp? When he rubbed it a genie appeared and granted him three wishes. “I’ll have a pint of Guiness!” the Irishman replied and immediately it appeared. The Irishman eagerly set to sipping and then gulping, but the level of Guiness in the glass was always magically restored. After a while the genie became impatient. “Well, what about your second wish?” he asked. Replied the Irishman between gulps, “Oh well, I guess I’ll have another one of these.” CHINESE ROOM: Very funny. No, I hadn’t heard it– but you know I find ethnic jokes in bad taste. I laughed in spite of myself, but really, I think you should find other topics for us to discuss. J: Fair enough but I told you the joke because I want you to explain it to me. CR: Boring! You should never explain jokes. J: Nevertheless, this is my test question. Can you explain to me how and why the joke “works”? CR: If you insist. You see, it depends on the assumption that the magically refilling glass will go on refilling forever, so the Irishman has all the stout he can ever drink. So he hardly has a reason for wanting a duplicate but he is so stupid (that’s the part I object to) or so besotted by the alcohol that he doesn’t recognize this, and so, unthinkingly endorsing his delight with his first wish come true, he asks for seconds. These background assumptions aren’t true, of course, but just part of the ambient lore of joke-telling, in which we suspend our disbelief in magic and so forth. By the way we could imagine a somewhat labored continuation in which the Irishman turned out to be “right” in his second wish after all, perhaps he’s planning to throw a big party and one glass won’t refill fast enough to satisfy all his thirsty guests (and it’s no use saving it up in advance– we all know how stale stout loses its taste). We tend not to think of such complications which is part of the explanation of why jokes work. Is that enough? Dennett: "The fact is that any program that could actually hold up its end in the conversation depicted would have to be an extraordinary supple, sophisticated, and multilayered system, brimming with “world knowledge” and meta-knowledge and meta-meta-knowledge about its own responses, the likely responses of its interlocutor, and much, much more…. Maybe the billions of actions of all those highly structured parts produce genuine understanding in the system after all." I'm sure they didn't get anywhere near this with their 13-yr-old simulation. But this gives an idea of the heights AI has to scale before it can regularly pass the Turing Test.
VLM 12y ago
The point missed is the Turing test was an abstract thought experiment into how we perceive the presence of intelligence. If a decade or so of social media (whatever that means) has proven anything, its that very little intelligence occurs in virtually all conversations. The meta Turing test is being failed by many people who think it (a concrete implementation of it) means something. Much like actually building a well sealed box with a cat, a radioisotope source, and a geiger counter wouldn't actually be a "great step forward for Quantum Physics" in 2014. Any more than making a little anthropomorphic horned robot and having him divert fast "hot" molecules one direction or slow "cold" molecules another would be a great step forward for thermodynamics in 2014. The value of a thought experiment is realized when its proposed, not when someone makes a science fair demonstration of the abstract idea.
Morendil 12y ago
That headline should read: "33% of human judges flunk the Turing Test".
ilaksh 12y ago
Even though many AI researchers will agree that the Turing Test isn't a very good representation of "real" intelligence, this is still a huge milestone. Many, many researchers have tried and failed to pass the Turing Test. But people will continue to dismiss the state of the art and deny that computers have "real" intelligence, the same way they did when the computer defeated Kasparov, the same way they did when we saw Googles self driving cars, the same way they did when a computer won on Jeopardy, and now with the Turing Test. Even when we have robots that look and act exactly like humans, many people will say that they are not "really" intelligent and dismiss the accomplishment. They will still be saying that when AIs twice as smart as people arrive and they have to figure out what to do with billions of what will then be, relatively speaking, mentally challenged people.
SilasX 12y ago
>“I feel about beating the turing test in quite convenient way. Nothing original,” said Goostman, when asked how he felt after his success. So how long until the creator can pass the Turing Test? (Normally I'd ignore that, but given the subject matter...)
dreeves 12y ago
This doesn't seem like much of a milestone to me. If Ray Kurzweil wins this bet against Mitch Kapor, that will be a milestone: http://longbets.org/1/ http://longbets.org/1/
e12e 12y ago
Between this and the Ars Technica article[1], I'm still confused: Was this a regular Turing test? Who was the humans that the machines tested against? As far as I recall, the model is two participants, one human, one machine -- the judges communicate with each through writing -- and if the machine "tests" as human more than 30% of the time, it's considered a "win" at the imitation game (the machine has successfully imitated being human). Both the machine and the human are supposed to try to appear human. (And this is extended from another form of the imitation game, where the goal is to imitate being male, where participants are male and female) Have anyone been able to find any more concrete information (and perhaps some transcripts)? If not I hope someone will set up a new test, and invite "Eugene" to participate. [1] http://arstechnica.com/information-technology/2014/06/eugene-the-supercomputer-not-13-year-old-first-to-beat-the-turing-test/ http://arstechnica.com/information-technology/2014/06/eugene... [edit: We may be given some hints from the wikpedia article on the turing test: https://en.wikipedia.org/wiki/Turing_test#Imitation_Game_vs._Standard_Turing_Test https://en.wikipedia.org/wiki/Turing_test#Imitation_Game_vs.... "Huma Shah and Kevin Warwick, who organised the 2008 Loebner Prize at Reading University which staged simultaneous comparison tests (one judge-two hidden interlocutors), showed that knowing/not knowing did not make a significant difference in some judges' determination. Judges were not explicitly told about the nature of the pairs of hidden interlocutors they would interrogate. Judges were able to distinguish human from machine, including when they were faced with control pairs of two humans and two machines embedded among the machine-human set ups. Spelling errors gave away the hidden-humans; machines were identified by 'speed of response' and lengthier utterances." ]
- andreasvc 12y ago
  Adding random spelling errors and delays to responses should be one of the more trivial "improvements" to a chatbot.
- e12e 12y ago
  Some more information from Gizmodo: http://gizmodo.com/5921698/what-its-like-to-judge-the-turing-test http://gizmodo.com/5921698/what-its-like-to-judge-the-turing... (linked from http://gizmodo.com/this-is-the-first-computer-in-history-to-have-passed-th-1587780232 http://gizmodo.com/this-is-the-first-computer-in-history-to-... )
lotsofmangos 12y ago
looks like some smegging marketing smeg for smeg-heads
kimonos 12y ago
Interesting post! Great insights I see in here..
vonsydov 12y ago
Stop bitching. Its passed. Get over it.
sfbsfbsfb 12y ago
Me: Do you have more hair on your head or on your eyelash? Goostman: If I say that I haven't more hair on my head or on my eye lash - will it satisfy your inquisitive mind? By the way, what's your occupation? I mean - could you tell me about your work? This is not passing the Turing test by any stretch of the imagination.
cwhy 12y ago
Failed.. Just keep saying hello to it and imagine it is a real people.
ma2rten 12y ago
I don't know exactly how Eugene works, but I am quite sure like most chat bots it simply reacts to keywords or preprogrammed patterns. Basically it's Eliza [0], but with more scripts. I believe most people here would not give it credit for winning an intelligence test. There is actually more than one bot, which has been claimed to have passed the Turing Test before. Cleverbot is one of them [1]. There are also several competitions, but I believe the most reputable and long standing one is the Loebner Prize [2]. The bot that currently holds the Loeber Prize is Mitsuku [3]. Anyway, you can chat with Eugene at [4], I gave it a try. I believe there is one thing that the creators of Eugene got right. When chatting with other chat bots, I usually in a situation where the bot says something, I ask I followup question (like "Why?"), and it gives a generic answer like "Because I say so" or "I don't know". Eugene does the same but will ask a unrelated followup question right together with the response. That way at least there is not a weird pause in the conversation. [0] http://en.wikipedia.org/wiki/ELIZA http://en.wikipedia.org/wiki/ELIZA [1] http://www.geekosystem.com/cleverbot-passes-turing-test/ http://www.geekosystem.com/cleverbot-passes-turing-test/ [2] http://www.loebner.net/Prizef/loebner-prize.html http://www.loebner.net/Prizef/loebner-prize.html [3] http://www.mitsuku.com http://www.mitsuku.com [4] http://www.princetonai.com/bot/bot.jsp http://www.princetonai.com/bot/bot.jsp
jestinjoy1 12y ago
"If a computer is mistaken for a human more than 30% of the time during a series of five minute keyboard conversations it passes the test. No computer has ever achieved this, until now. Eugene managed to convince 33% of the human judges that it was human." SO the result can very depending on different conditions. :) Highly non deterministic
testingit 12y ago
Spoken language is just a small part of overall communication. Today Turing Test should consist of not only speaking but simulating, perhaps by a streaming video, a professor in front of a class of wild students. He should strive to capture their attention, to gain their respect and interest, to understand their inner state of mind. Speaking is only a way of getting into the stage. Once into the highlights you must prove you are a leader or, if you decide so, that you are able to gain the attention of your audience to emphasize something important that previously was not perceived as such. That is speaking is an art, is not about explaining a plot but about creating a story.
alt_f4 12y ago
Ah, Reading University - the powerhouse of computer science... Move on people, it's just a cheap PR stunt.
netcan 12y ago
How feasible is it to have hosted state of the art Turing bots available to the converse with anyone?
PythonicAlpha 12y ago
A real test of (Artificial) intelligence would be, must include that it is real capable of learning new things and developing and not trying to convince others that it has learned something (what in this case in my opinion is wrong, since it is just knowledge that is programmed into it -- meaning, not its own knowledge, but borrowed knowledge of the programmers). I would also add, that the real prove must include a topic that the artificial person was not programmed for. (not like a Bayesian filter that "develops" by "learning" new facts about a fixed topic). Learning, developing, evolving, that are the real marks of living and of intelligence (since, I would not part between intelligence and living).
Bayesianblues 12y ago
I've often wondered if the Turing Test has been decoupled from signifying its original goal due to an instance of Goodhart's Law; namely, "When a measure becomes a target, it ceases to be a good measure." Surely the ability to trick a human into believing an AI is a human is a milestone, but it was with an AI specifically optimized for this task. The deeper question is if the passing of the Turing Test in this case means we should ascribe consciousness to the bot, and I think none of us are willing to affirm it yet. I would suggest that this discrepancy is caused by the “measure becoming a target” and losing its ability to be a “good measure.” I guess this is why there is such a critical distinction between Artificial Intelligence and Artificial General Intelligence, which is where the Turing Test would have more weight.

216 points by stevejalim 12y ago

Any chance of seeing conversation logs?

raldi 12y ago

I appreciate HN's rule about brief, non-sensational submission titles, but perhaps this one's taken it to the point of absurdism.

dang 12y ago
It was the submitter's title, not a mod edit, if that matters. The submitter did a good job, because the rest of the title ("marks milestone in computing history") is certainly linkbait and arguably misleading. Having it there would likely have made this thread a lower-quality controversy than it currently is. HN threads are surprisingly sensitive to initial conditions. Edit: Oh, and considering how many fluff press articles are showing up about this [1], the submitter also showed exemplary taste in source selection. Yay, stevejalim! 1. https://hn.algolia.com/?q=turing+test#!/story/sort_by_date/0/turing%20test https://hn.algolia.com/?q=turing+test#!/story/sort_by_date/0...
- hanru 12y ago
  It seems the submitter has passed your Turing test. ;)

Mithaldu 12y ago

Bold claim, but without any transcripts it's impossible to verify just how close to the truth the claim comes.

GhotiFish 12y ago
That's true. I want to talk to Eugine. Your suggestion seems to be the very minimum.

danbruc 12y ago

Surly they did not talk to this one [1] - it is light-years away from being convincing. [1] http://www.princetonai.com/bot/ http://www.princetonai.com/bot/

keehun 12y ago
I was impressed it could write grammatically sound sentences, but it's not even close to human-level conversation...
- jamesrom 12y ago
  I would be impressed if it didn't. If it made common mistakes a human would make, it would seem more human.
owenversteeg 12y ago
I tried talking to this one, and it failed to convince me at all. Even in the one conversation I had that was slightly believable, the bot was prone to extremely rapid emotional swings.

DanBC 12y ago

> If a computer is mistaken for a human more than 30% of the time during a series of five minute keyboard conversations it passes the test. No computer has ever achieved this, until now. Eugene managed to convince 33% of the human judges that it was human. Surely it depends on who the human judges are. It seems a bit unfair that the judges normally have IQ > 100 and the other humans have IQ > 100. I strongly suspect that some simplistic AI (alicebots, for example) would beat the Turing test if the human judges had IQ between 90 and 105. (Especially if we're using the limited 30% rule above). Getting bots running on some Facebook groups might be interesting.

radio4fan 12y ago

Awesome job of putting un-followable links in the press release, Reading University. I'm going to take this with a hug pinch of salt until I've read the transcripts, due to the involvement of famous publicity-hound Kevin Warwick.

reillyse 12y ago
Yes, at this stage any time I read his name involved with something I cringe.

vixin 12y ago

Lots of self-congratulation all round with no sample questions to provide the merest smidgeon of 'reason to believe that this is that significant'.

eli_gottlieb 12y ago

Managing to successfully imitate an ignorant, immature child 1/3 of the time is not what I would call a success, but rather a subversion of the entire intent behind the Turing Test in the first place.

splike 12y ago
I think you're probably expecting to see a 50/50 chance (or higher) of guessing it was a computer. But if you think about it, anything better than 50% would make it more human than a human. So is why the target is lower than 50%
- chippy 12y ago
  It's not a random guess though is it? It's a sample of results.
- GhotiFish 12y ago
  Wait, could you explain how a program that is capable of being mistaken for a human more than 50% of the time is more human than human? I'm pretty sure you're human, with a high degree of confidence, much higher than 50%. So to are you more human than human?
  - qu4z-2 12y ago
    I believe that in the original formulation of the Turing Test the judges were asked to hold two conversations: one with a computer and one with a human. Afterwards they chose which of the two they believed to be the human. In that scenario, being identified as "human" more than 50% of the time would indeed make you "more human than human"; with a "perfectly human" program the test becomes essentially a coin-toss. In that context the 30% threshold makes a good deal of sense. But in this variant, "Do you believe the entity you spoke with to be a human or a program?", a 100% threshold is theoretically achievable. That means you're entirely correct: a 51% vote of confidence is decidedly less human than human. Additionally the 30% threshold they've used is laughably low in this context. Without a control group, even an 100%-confidence outcome probably says more about the beliefs of the judges than the ability of the program to simulate a human. I don't mean to take away from the no-doubt impressive achievement by the team behind Eugene. I just take issue with the hyperbole in its reporting. But, ya know, the media will be the media, and academics gotta get research grants.

TempleOS 12y ago

God damn you have a low IQ. I get a number from NIST. If it's fake, just use a human. Retard-nigger. Tin-foil-hat nigger doesn't know what a random number is. It's fucken God, nigger.

draq 12y ago

If it was hosted by the Royal Society, wouldn't it be on their website (https://royalsociety.org/events/?type=all&direction=past https://royalsociety.org/events/?type=all&direction=past)?

rahimnathwani 12y ago
It was hosted _by_ the University of Reading _at_ the Royal Society. https://royalsociety.org/venue-hire/ https://royalsociety.org/venue-hire/

mratzloff 12y ago

The Turing Test is a terrible measure of sapience. Generally it involves using "average people", who have been shown time and again to be overly credulous when talking to these bots. If the test is to be used at all, it should consist of computer science experts instead--people familiar with the technology and bot tricks of the trade. That the Turing Test is still used is proof that we still don't understand how to even define sapience. Without a definition and concrete, testable qualities, how can we possibly hope to ever build artificial sapience. As a result, we continue to see these toys that are little more than parlor tricks. Any true test should include looking behind the curtain. "I know you're artificial--I can see the processes working--yet I have doubts that what I'm seeing is real." In other words, real success is the tester believing he is being fooled when he is not, rather than fooling the tester into believing it is real.

wslh 12y ago
I have a friend who used a sex chat bot to sell things to people. It was something very basic but incredible effective. It made me think that we need to divide the turing test for different people: by age, by need, etc.
tritium 12y ago
Generally it involves using "average people" [...] it should consist of computer science experts instead I don't agree. The prominent reason for "dumbing down" the judges follows the same reasoning behind decisions made regarding what constitutes "adequate" encryption. How can we gauge what will honestly happen out in the real world, today? Consider DES. It was deemed inadequate, but how to prove it? The EFF came up with a budget estimate based on what was reasonably affordable for a group of attackers, and built a machine capable of cracking DES within those budget constraints. http://w2.eff.org/Privacy/Crypto/Crypto_misc/DESCracker/HTML/19980716_eff_des_faq.html http://w2.eff.org/Privacy/Crypto/Crypto_misc/DESCracker/HTML... So, let's apply similar reasoning to the concept of AI. If a group of people were to build an AI, and use it as an adversary against ordinary people, how difficult is it to manipulate and deceive ordinary people into taking action, and is it feasible to do so with AI?
- 12y ago

muglug 12y ago

This looks awfully like a private event for Reading University that just happens to have been held at the Royal Society. I wonder whether they even had more than three non-"celebrity" judges.

skywhopper 12y ago

Where did the 30% requirement come from? Sounds like something the contest organizers added to make it possible to "pass" without fooling 2/3 of the judges. Using a young teenager as a character also seems like a cheat unless they had other 13-year olds to be the judges. The character needs to be a peer to the judges. Most 13 year olds behave oddly enough in the opinion of most adults that it's got to be easier to credit weird behavior to generational or cultural differences. So kudos to the winners on strategy, and boo to the contest organizers for having such poor rules.

KwanEsq 12y ago
Most likely from here: >It will simplify matters for the reader if I explain first my own beliefs in the matter. Consider first the more accurate form of the question. I believe that in about fifty years' time it will be possible, to programme computers, with a storage capacity of about 109, to make them play the imitation game so well that an average interrogator will not have more than 70 per cent chance of making the right identification after five minutes of questioning. COMPUTING MACHINERY AND INTELLIGENCE — A. M. Turing http://loebner.net/Prizef/TuringArticle.html http://loebner.net/Prizef/TuringArticle.html
- Buge 12y ago
  Is he talking about 109 bits? Because that seems pretty impossible to me.
  - ar7hur 12y ago
    More likely 1E9
  - DanBC 12y ago
    > As we have mentioned, digital computers fall within the class of discrete-state machines. But the number of states of which such a machine is capable is usually enormously large. For instance, the number for the machine now working at Manchester is about 2 165,000, i.e., about 10 50,000. Compare this with our example of the clicking wheel described above, which had three states. It is not difficult to see why the number of states should be so immense. The computer includes a store corresponding to the paper used by a human computer. It must be possible to write into the store any one of the combinations of symbols which might have been written on the paper. For simplicity suppose that only digits from 0 to 9 are used as symbols. Variations in handwriting are ignored. Suppose the computer is allowed 100 sheets of paper each containing 50 lines each with room for 30 digits. Then the number of states is 10 100x50x30 i.e., 10 150,000 . This is about the number of states of three Manchester machines put together. The logarithm to the base two of the number of states is usually called the "storage capacity" of the machine. Thus the Manchester machine has a storage capacity of about 165,000 and the wheel machine of our example about 1.6. If two machines are put together their capacities must be added to obtain the capacity of the resultant machine. This leads to the possibility of statements such as "The Manchester machine contains 64 magnetic tracks each with a capacity of 2560, eight electronic tubes with a capacity of 1280. Miscellaneous storage amounts to about 300 making a total of 174,380."

thinkersilver 12y ago

I remember reading the first chapter on an AI book during college which laid out the objectives of AI research into two schools of thought.The first school of thought believes that AI can be achieved by mimicking real conscious beings. Remember when man first tried to fly, most devices mimicked birds and failed horribly. The other school of thought believes that intelligence has well-defined principles, when discovered, would produce real intelligence (not mimicry), having the effect that the final product may not resemble what we see on day to day basis. Compare planes, which use the principles of flight, to the early flapping machines. A bird and plane both fly but they are very different in the way they do it. Approaching flight from the mimicry angle is hard, it's only recently in the last 10 years we have made light-weight flapping machines that fly well;yet we have had planes for over a hundred years once we knew the principles. Transcripts would be handy. I doubt a conversation with a 13 year old boy is a good way to measure AI? It's not the best metric to have but it is the most universal and most widely agreed on that we have. It seems like we are happier with small gains in mimicry for now, since real intelligence is hard. Really hard.

acchow 12y ago
> The first school of thought believes that AI can be achieved by mimicking real conscious beings It's important to note the true meaning behind this school of thought - that mimicry and "true" intelligence are actually equivalent. Behaviorists believe this, and the validity of the Turing Test along with it.
- swombat 12y ago
  I think it might actually be in Civ5 that I first heard the following quote: "Asking whether a computer can think is about as useful as asking whether a submarine can swim." I think this is fairly insightful and relevant.
  - marcosdumay 12y ago
    That's from Djigstra. I digress. Most of the times, mimicking Nature works very well. We tend to always remember our most spectacular failure, but that's a selection bias.

monochr 12y ago

"a computer programme that simulates a 13 year old boy [...] If a computer is mistaken for a human more than 30% of the time during a series of five minute keyboard conversations it passes the test." In short they did nothing.

grondilu 12y ago

Did they also run the experiment with an actual 13 yo kid?

DanBC 12y ago
...Imagine someone releasing bots on XBOXLive. Your task is to guess which obscenity-screaming 13 year okd is real and which is a bot. Some forms of Turing test are trivially passable with dumb enough humans.

inetsee 12y ago

Although the Turing Test is interesting, it is not, in my opinion, all that useful. I would much rather see chess program level of performance in the domain of medical diagnosis, for example. There are lots of other domains where I would be entirely happy to know that I was talking to an AI, if the answers I was getting were significantly better than most human experts in that domain.

xpose2000 12y ago

Perhaps I just asked the right questions, but I just had a lovely conversation with 'Eugene'. I asked him a few questions like where he lived, his name, if he has brothers or sisters, if he wears glasses, etc. Eventually he started asking me questions like what I did for a living and where I lived. He also managed to form questions based on my answers. It almost felt like a conversation. I can honestly say, I've never thought that before while talking to an AI. So far I am pretty impressed.

frabcus 12y ago
Is there somewhere online you can talk to him?
- umanwizard 12y ago
  http://www.princetonai.com/bot/bot.jsp http://www.princetonai.com/bot/bot.jsp
  - modinfo 12y ago
    mirror? it down now
  - AndrewKemendo 12y ago
    Realize that this version is from 2001 and is not the version that was in the recent test.
  - de18de18 12y ago
    what is the united states?
  - de18de18 12y ago
    how much is 2 plus 2
    - deleted 12y ago
      [deleted]
  - deleted 12y ago

fchollet 12y ago

The Turing test is to our understanding of intelligence what sleight of hand is to our understanding of physics. Tricking people, as a goal, is not conductive to science. A researcher claiming to have passed the Turing test instantly discredits himself as a prestidigitator looking for PR buzz. The present article is a textbook example of this. As a side note, if you are focusing on disembodied, language-based human-like intelligence, then the paradigm you operate in is many decades behind. The Turing test was conceived at a time when the notion of thinking machines had just started to emerge --a very different time from today, where we have 60 years of AI research behind us. The Turing test has been irrelevant for longer than most AI researchers have been alive. I have never seen it used for anything else than smoke-and-mirrors PR operations.

acchow 12y ago
Your opinion is not universally accepted. (Perhaps not even mainstream?). There's an element of behaviorism which stretches back to Descartes - how can you know that I exist? That I think? You can only observe through my behavior; that my behavior mimics yours. How then can we judge machines any other way?
- VLM 12y ago
  Your spec is over generalized into all forms of observation and comparison. The Turing test is very limited into written 1 on 1 conversation as culturally conceived by humans. There's no particular reason to assume a non-human intelligence would have a human-like culture of communication, which kind of breaks it. Here's a fun idea for several intelligence tests we can call the VLM tests that have nothing to do with two way conversation like the Turing test. Given "a machine intelligence" spin up a couple million of them in a "fun" simulation environment and see how much thermodynamic dis-equilibrium they generate by whatever social interaction they see fit to apply to each other. Is it as interesting (aka thermodynamic dis-equilibrium) as a GoL or a real world anthill or a Dwarf Fortress or a Google Earth? Is their simulated culture as interesting to read as HN, or as dumb as youtube comments (which used to be the gold standard of dumbness in social media) Assuming you can crack the literary code (if any exists) another game to play is extract the meme-flow of a culture of AI vs a culture of 4chan and vote for whichever meme came from a more intelligent group. This is Turing-ish WRT human observers majority vote and such, but is completely non-interactive, merely humans, or even trained sociologists, trying to figure out given two memes which is more intelligent. Getting out the Sherlock Holmes hat, its possible to determine if an artifact came from an intelligence without talking to the intelligence for awhile. I suspect archeologists have really fun debates on this topic. Is this a stone hammer or merely a peculiar river rock, etc.

AndrewKemendo 12y ago

You can watch the goal posts shifting for "AI" as we speak. Great result nonetheless!

deleted 12y ago

[deleted]

codeulike 12y ago

When people imagine what a Turing Test conversation would look like, they frequently underestimate the conversation. I find Dennet's example of an imaginary Turing Test from Consciousness Explained to be a good counterexample: Judge: Did you hear about the Irishman who found a magic lamp? When he rubbed it a genie appeared and granted him three wishes. “I’ll have a pint of Guiness!” the Irishman replied and immediately it appeared. The Irishman eagerly set to sipping and then gulping, but the level of Guiness in the glass was always magically restored. After a while the genie became impatient. “Well, what about your second wish?” he asked. Replied the Irishman between gulps, “Oh well, I guess I’ll have another one of these.” CHINESE ROOM: Very funny. No, I hadn’t heard it– but you know I find ethnic jokes in bad taste. I laughed in spite of myself, but really, I think you should find other topics for us to discuss. J: Fair enough but I told you the joke because I want you to explain it to me. CR: Boring! You should never explain jokes. J: Nevertheless, this is my test question. Can you explain to me how and why the joke “works”? CR: If you insist. You see, it depends on the assumption that the magically refilling glass will go on refilling forever, so the Irishman has all the stout he can ever drink. So he hardly has a reason for wanting a duplicate but he is so stupid (that’s the part I object to) or so besotted by the alcohol that he doesn’t recognize this, and so, unthinkingly endorsing his delight with his first wish come true, he asks for seconds. These background assumptions aren’t true, of course, but just part of the ambient lore of joke-telling, in which we suspend our disbelief in magic and so forth. By the way we could imagine a somewhat labored continuation in which the Irishman turned out to be “right” in his second wish after all, perhaps he’s planning to throw a big party and one glass won’t refill fast enough to satisfy all his thirsty guests (and it’s no use saving it up in advance– we all know how stale stout loses its taste). We tend not to think of such complications which is part of the explanation of why jokes work. Is that enough? Dennett: "The fact is that any program that could actually hold up its end in the conversation depicted would have to be an extraordinary supple, sophisticated, and multilayered system, brimming with “world knowledge” and meta-knowledge and meta-meta-knowledge about its own responses, the likely responses of its interlocutor, and much, much more…. Maybe the billions of actions of all those highly structured parts produce genuine understanding in the system after all." I'm sure they didn't get anywhere near this with their 13-yr-old simulation. But this gives an idea of the heights AI has to scale before it can regularly pass the Turing Test.

VLM 12y ago

The point missed is the Turing test was an abstract thought experiment into how we perceive the presence of intelligence. If a decade or so of social media (whatever that means) has proven anything, its that very little intelligence occurs in virtually all conversations. The meta Turing test is being failed by many people who think it (a concrete implementation of it) means something. Much like actually building a well sealed box with a cat, a radioisotope source, and a geiger counter wouldn't actually be a "great step forward for Quantum Physics" in 2014. Any more than making a little anthropomorphic horned robot and having him divert fast "hot" molecules one direction or slow "cold" molecules another would be a great step forward for thermodynamics in 2014. The value of a thought experiment is realized when its proposed, not when someone makes a science fair demonstration of the abstract idea.

Morendil 12y ago

That headline should read: "33% of human judges flunk the Turing Test".

ilaksh 12y ago

Even though many AI researchers will agree that the Turing Test isn't a very good representation of "real" intelligence, this is still a huge milestone. Many, many researchers have tried and failed to pass the Turing Test. But people will continue to dismiss the state of the art and deny that computers have "real" intelligence, the same way they did when the computer defeated Kasparov, the same way they did when we saw Googles self driving cars, the same way they did when a computer won on Jeopardy, and now with the Turing Test. Even when we have robots that look and act exactly like humans, many people will say that they are not "really" intelligent and dismiss the accomplishment. They will still be saying that when AIs twice as smart as people arrive and they have to figure out what to do with billions of what will then be, relatively speaking, mentally challenged people.

SilasX 12y ago

>“I feel about beating the turing test in quite convenient way. Nothing original,” said Goostman, when asked how he felt after his success. So how long until the creator can pass the Turing Test? (Normally I'd ignore that, but given the subject matter...)

dreeves 12y ago

This doesn't seem like much of a milestone to me. If Ray Kurzweil wins this bet against Mitch Kapor, that will be a milestone: http://longbets.org/1/ http://longbets.org/1/

e12e 12y ago

Between this and the Ars Technica article[1], I'm still confused: Was this a regular Turing test? Who was the humans that the machines tested against? As far as I recall, the model is two participants, one human, one machine -- the judges communicate with each through writing -- and if the machine "tests" as human more than 30% of the time, it's considered a "win" at the imitation game (the machine has successfully imitated being human). Both the machine and the human are supposed to try to appear human. (And this is extended from another form of the imitation game, where the goal is to imitate being male, where participants are male and female) Have anyone been able to find any more concrete information (and perhaps some transcripts)? If not I hope someone will set up a new test, and invite "Eugene" to participate. [1] http://arstechnica.com/information-technology/2014/06/eugene-the-supercomputer-not-13-year-old-first-to-beat-the-turing-test/ http://arstechnica.com/information-technology/2014/06/eugene... [edit: We may be given some hints from the wikpedia article on the turing test: https://en.wikipedia.org/wiki/Turing_test#Imitation_Game_vs._Standard_Turing_Test https://en.wikipedia.org/wiki/Turing_test#Imitation_Game_vs.... "Huma Shah and Kevin Warwick, who organised the 2008 Loebner Prize at Reading University which staged simultaneous comparison tests (one judge-two hidden interlocutors), showed that knowing/not knowing did not make a significant difference in some judges' determination. Judges were not explicitly told about the nature of the pairs of hidden interlocutors they would interrogate. Judges were able to distinguish human from machine, including when they were faced with control pairs of two humans and two machines embedded among the machine-human set ups. Spelling errors gave away the hidden-humans; machines were identified by 'speed of response' and lengthier utterances." ]

andreasvc 12y ago
Adding random spelling errors and delays to responses should be one of the more trivial "improvements" to a chatbot.
e12e 12y ago
Some more information from Gizmodo: http://gizmodo.com/5921698/what-its-like-to-judge-the-turing-test http://gizmodo.com/5921698/what-its-like-to-judge-the-turing... (linked from http://gizmodo.com/this-is-the-first-computer-in-history-to-have-passed-th-1587780232 http://gizmodo.com/this-is-the-first-computer-in-history-to-... )

lotsofmangos 12y ago

looks like some smegging marketing smeg for smeg-heads

kimonos 12y ago

Interesting post! Great insights I see in here..

vonsydov 12y ago

Stop bitching. Its passed. Get over it.

sfbsfbsfb 12y ago

Me: Do you have more hair on your head or on your eyelash? Goostman: If I say that I haven't more hair on my head or on my eye lash - will it satisfy your inquisitive mind? By the way, what's your occupation? I mean - could you tell me about your work? This is not passing the Turing test by any stretch of the imagination.

cwhy 12y ago

Failed.. Just keep saying hello to it and imagine it is a real people.

ma2rten 12y ago

I don't know exactly how Eugene works, but I am quite sure like most chat bots it simply reacts to keywords or preprogrammed patterns. Basically it's Eliza [0], but with more scripts. I believe most people here would not give it credit for winning an intelligence test. There is actually more than one bot, which has been claimed to have passed the Turing Test before. Cleverbot is one of them [1]. There are also several competitions, but I believe the most reputable and long standing one is the Loebner Prize [2]. The bot that currently holds the Loeber Prize is Mitsuku [3]. Anyway, you can chat with Eugene at [4], I gave it a try. I believe there is one thing that the creators of Eugene got right. When chatting with other chat bots, I usually in a situation where the bot says something, I ask I followup question (like "Why?"), and it gives a generic answer like "Because I say so" or "I don't know". Eugene does the same but will ask a unrelated followup question right together with the response. That way at least there is not a weird pause in the conversation. [0] http://en.wikipedia.org/wiki/ELIZA http://en.wikipedia.org/wiki/ELIZA [1] http://www.geekosystem.com/cleverbot-passes-turing-test/ http://www.geekosystem.com/cleverbot-passes-turing-test/ [2] http://www.loebner.net/Prizef/loebner-prize.html http://www.loebner.net/Prizef/loebner-prize.html [3] http://www.mitsuku.com http://www.mitsuku.com [4] http://www.princetonai.com/bot/bot.jsp http://www.princetonai.com/bot/bot.jsp

jestinjoy1 12y ago

"If a computer is mistaken for a human more than 30% of the time during a series of five minute keyboard conversations it passes the test. No computer has ever achieved this, until now. Eugene managed to convince 33% of the human judges that it was human." SO the result can very depending on different conditions. :) Highly non deterministic

testingit 12y ago

Spoken language is just a small part of overall communication. Today Turing Test should consist of not only speaking but simulating, perhaps by a streaming video, a professor in front of a class of wild students. He should strive to capture their attention, to gain their respect and interest, to understand their inner state of mind. Speaking is only a way of getting into the stage. Once into the highlights you must prove you are a leader or, if you decide so, that you are able to gain the attention of your audience to emphasize something important that previously was not perceived as such. That is speaking is an art, is not about explaining a plot but about creating a story.

alt_f4 12y ago

Ah, Reading University - the powerhouse of computer science... Move on people, it's just a cheap PR stunt.

netcan 12y ago

How feasible is it to have hosted state of the art Turing bots available to the converse with anyone?

PythonicAlpha 12y ago

A real test of (Artificial) intelligence would be, must include that it is real capable of learning new things and developing and not trying to convince others that it has learned something (what in this case in my opinion is wrong, since it is just knowledge that is programmed into it -- meaning, not its own knowledge, but borrowed knowledge of the programmers). I would also add, that the real prove must include a topic that the artificial person was not programmed for. (not like a Bayesian filter that "develops" by "learning" new facts about a fixed topic). Learning, developing, evolving, that are the real marks of living and of intelligence (since, I would not part between intelligence and living).

Bayesianblues 12y ago

I've often wondered if the Turing Test has been decoupled from signifying its original goal due to an instance of Goodhart's Law; namely, "When a measure becomes a target, it ceases to be a good measure." Surely the ability to trick a human into believing an AI is a human is a milestone, but it was with an AI specifically optimized for this task. The deeper question is if the passing of the Turing Test in this case means we should ascribe consciousness to the bot, and I think none of us are willing to affirm it yet. I would suggest that this discrepancy is caused by the “measure becoming a target” and losing its ability to be a “good measure.” I guess this is why there is such a critical distinction between Artificial Intelligence and Artificial General Intelligence, which is where the Turing Test would have more weight.