6 ms·

I was struck by the prescience and clarity of Mitch Kapor's writing. He correctly predicted that a computer would be able to win on Jeopardy [0], 9 years before

I was struck by the prescience and clarity of Mitch Kapor's writing. He correctly predicted that a computer would be able to win on Jeopardy [0], 9 years before the fact. And he understood that the Turing Test was more of a thought experiment than an actual test [1], which is why I assume he took the bet in the first place. Conversation is actually a rather poor measure of intelligence. I would say, show me a computer that can learn anything that it wasn't specifically programmed to learn. This doesn't mean unsupervised categorization or learning to play a video game. I'm talking about a scenario where programmers present their code or machine with no knowledge of what the task will be. Not something chosen from a known list of possibilities, but any task that a human could conceivably be taught to perform in less than an hour. Anything from writing a sonnet in iambic pentameter to assembling ikea furniture based on instructions. A true test of _general_ intelligence. I would take that long bet out to 2129, and beyond. I don't see software with that level of intellectual flexibility being written in our lifetimes, or the lifetimes of our children or their children. [0] "While it is possible to imagine a machine obtaining a perfect score on the SAT or winning Jeopardy..." [1] "... a skeptic about machine intelligence could fairly ask how and why the Turing Test was transformed from its origins as a provocative thought experiment by Alan Turing to a challenge seriously sought."

icebraining 12y ago
Well, indirectly, the Turing test does allow you to test for learning, as long as it can be taught during the period of time available for the interview and using a chat. Assembling furniture might be hard to test for, but you could certainly ask it to write a sonnet.
- tim333 12y ago
  I wonder how the humans in the turing test would get on with sonnet writing. I imagine if they were, say computer scientists, rather poorly. I don't think I'd do terribly well at that. Looking just now at Turing's paper he actually deals with Sonnets and suggests the conversation might go: >Q: Please write me a sonnet on the subject of the Forth Bridge. >A : Count me out on this one. I never could write poetry. Now that I could probably do.
lloeki 12y ago
> Not something chosen from a known list of possibilities, but any task that a human could conceivably be taught to perform in less than an hour. Anything from writing a sonnet in iambic pentameter to assembling ikea furniture based on instructions. A true test of _general_ intelligence. The sad thing is I happen to know humans who would utterly fail at this (which, I know, is not invalidating the fact that a computer passing this would be worthwhile, but care has to be taken as to what that means), routinely challenging my faith in mankind.
cheatsheet 12y ago
I don't think specific problems are the issue. Even assembling furniture given instructions is not too difficult to ask a machine to solve, given the right programming language. It's the jump from one specific problem to another, without a heuristic used to leap between problem types. That would be a machine learning the ability to generalize and infer like a human. The Turing test is simple because it is easy to add complexity to something that is simple. As a human, I can convince myself that an idiot is a genius and a genius is an idiot because of introspection. Humans have imagination, and we very often don't see how that warps our perception of intelligence. We still think we can be the distant, disconnected, scientific, observer - carefully constructing a pristine and universally objective technology. But computers, out of all kinds of technology, show us how much of our humanity goes into our creations.
sumitviii 12y ago
>I would take that long bet out to 2129, and beyond. I don't see software with that level of intellectual flexibility being written in our lifetimes, or the lifetimes of our children or their children https://xkcd.com/1425/ https://xkcd.com/1425/
- p1esk 12y ago
  This comic is a great illustration of how a "virtually impossible" task just a few years ago (identifying a bird in a picture) is now "virtually trivial".
pharke 12y ago
I would think that your conversation with the computer would play a vital role while you were instructing it in the task it is to perform, even if it were a purely one-sided, instructional conversation the computer would still need to acknowledge truthfully whether it had correctly understood the instructions. More likely, the goal of creating such a system would be for it to be capable of engaging in active learning through inquiry but I suppose that would not be absolutely necessary given a specific task. There are also a number of assumptions when approaching problems such as composing a sonnet or assembling furniture. With a human, you assume that they have a broad understanding of the language you are communicating in, that they have some knowledge of poetry even if just in passing, have experienced cadence, rhyme, or other constructs and have prior experience of the world from which to draw 'inspiration'. You also assume that they have seen and used a variety of furniture, tools, and even just generally explored a physical environment to a degree where they are capable of manipulating objects by moving, rotating, stacking, joining, etc. or even as basic as having the concept of object permanence. They must also know all of the relevant nouns and verbs referring to the pieces and tools they will be using. If you continue to think along these lines you very quickly realize that the only reason you can teach a human being something in an hour is because they possess years or decades of previous instruction, programming, and prior learning attempts as well as a data set of visual, aural, tactile, olfactory, and spatial experiences encompassing a similar time frame.
ObviousScience 12y ago
I think you presented a false comparison. You said anything we can teach a human within an hour: do you mean a fresh human that's only just been supplied with a brain? Because you can't teach a baby anything besides basic stimulus response on that time frame, and I'm pretty sure we can get computers to do that if we rig up the stimulus response hardware in a way comparable to a newborn. What I think you actually meant, and what I think is a completely rigged test, is comparing a human with years of training and adaptive hardware modification to a computer with absolutely no training in its ability to learn a new skill or building out its knowledge base. That, of course, is a completely terrible comparison. I'll take your bet, and call it by 2050, if you're willing to compare a human with 5 years of hardware and knowledge base training and adaptation to a computer with 1 year of hardware adaption (eg, FPGA circuit reprogramming) and 1 year of knowledge base training.
- geon 12y ago
  I assume the AI would be pre-trained to (or beyond) the level which could be expected of an adult. The "within an hour" part refers to the test itself.
  - ObviousScience 12y ago
    I'm willing to modify the bet for that: I think an AI with 20 years to be trained and have things like FPGA components adjusted can learn in 1 hour anything I could teach a person in the same amount of time with the same amount of pre-training.
emiliobumachar 12y ago
Write a sonnet in iambic pentameter an hour? Okay, but keep in mind you're holding your AI to a higher standard than the average human. Most people can't write a sonnet in an hour, though it's very easy to find those who can.
yk 12y ago
A conversation is probably the best measure of general intelligence we have, at least if you allow the judges to formulate arbitrary challenges: please solve exercise 3a from Caroll. Simply because you can encode almost all human experience as text. Having said that, I do not believe that passing the Turing test necessarily a desirable property of an AI. At the very least, a AI would need to be able to lie about its childhood. Furthermore, a AI may be truly alien, so that even if we assume a truly sentient AI it would necessarily need the ability to hide its own internal states, for example a AI may view the entire sex/gender/procreation topic as mere curiosity, and instead emulate a Human well enough that it can fool other humans.
- waps 12y ago
  Sure but the AI wouldn't necessarily have to respond. I think the main flaw in the Turing test is it's cultural dependence. I would argue that Eliza has passed plenty of Turing tests, mostly with people who had a chance encounter with it. But why would an AGI individual have to be able to lie about it's childhood ? It could simply have a childhood. But I understand your argument : "Human" AI is constantly trained for the function "be a human". Machine intelligence is being trained with "is human X cheating", "is this spam", "what is the french for apple", "which way will stock X go" (and other "what will <large number of humans> do in situation X" questions) ... functions. Keep in mind that your own brain is also simply running an algorithm, with "hidden state". Here's the first thing it hides pretty well : there's no single "you", there's about 300 regions in your brain, that are physically different, in different locations, and not directly linked. And if the rest of your brain gets disabled, each of those 300 regions can use your body to convince your own mother it's really you she's talking to. And the function your brain is executing is not to "have a soul" or something like that, but is mostly a behaviour-copy algorithm. It searches out other humans, observes their actions, learns to predict them and then uses what it's learned to "be you". Most people will not believe this is how it works, not even when they know quite a bit about neurons, and know pretty well that this is what our neurons do. But when it comes right down to it, you, me, everyone is an "AI algorithm" just like Google search. Aside from the shear scale, number of connections mostly, there is nothing all that remarkable about the hardware it's running on. I love how an AGI algorithm is presented in the follow up to the Battlestar Galactica series, "Caprica". Zoe Graystone explains it at one point in the series. It's fiction of course, but still. Her algorithm searches the internet, your computer, and everything and anything it can find for pictures and videos of it's "target" and then builds the function "given situation X, what would <human> do ?" and then puts the algorithm in a virtual body that looks like the human, and uploads it to V-world. She runs it on herself, a friend and accidentally on a third person she doesn't know and then proceeds to get herself blown up by a friend/terrorist (they can be both at the same time, you know) in the first episode, and their avatars wake up in V-world, sort of a facebook virtual reality edition, unaware of what has happened (and as V-world is mostly used for sex, dancing and debauchery it's not that easy to find out unless you're looking for it). Currently we don't have nearly enough data on any person to make this work ... but I am pretty convinced it is possible to make this work. This would work, since it's doing the same thing your brain does when it hears the question "do you remember feeling happy when your mother held you when you were a baby ?". First, your brain doesn't actually remember that either way. It simply works it's way through to a good answer to that question. The thing it'll do is predict how you should answer that question, given (mostly) how other humans answer that question. This is a very different problem from "lying about its childhood", specifically it's much more solvable. Can I predict how humans would answer questions about their childhood ? Certainly. Easy. Make the algorithm take enough contextual information into account (so that it both realizes what realistic answers are and either answers realistically or responds in a way that humans respond when they don't have a "good prediction" for an answer, for instance by saying they don't remember, or by changing the subject, or ...) If you have young kids you will see this process in action. You have to "teach" kids memory. When they talk only a little, if you ask them something about earlier that day, they will respond. And the response makes sense, but it's not actually what happened. You will at some point realise this and correct them. Correcting them teaches them to match up what happened with their response, because they learn to predict that that's what's expected of people. Over time this correcting them will work, and they will have learned memory (and then they learn to lie, especially on questions like "did you hit your sister ?", and when that ends badly they learn to "not lie", like adults do. "Did you hit your sister ?" "She stole my lego"). Note that the underlying memory is also a prediction. If I ask you what happened this morning, and you want to try to answer correctly, you will start with an obvious fact (which isn't necessarily the truth), like "I woke up". Your brain will take that thought, and predict (not remember) what happened next. So I woke up, ok next, brushing teeth, ok next, read cell phone messages, ok next, ... ah ! the answer to the question.
flycaliguy 12y ago
If I computer ever writes a sonnet we will now have to make sure it didn't read your post.

icebraining 12y ago

Well, indirectly, the Turing test does allow you to test for learning, as long as it can be taught during the period of time available for the interview and using a chat. Assembling furniture might be hard to test for, but you could certainly ask it to write a sonnet.

tim333 12y ago
I wonder how the humans in the turing test would get on with sonnet writing. I imagine if they were, say computer scientists, rather poorly. I don't think I'd do terribly well at that. Looking just now at Turing's paper he actually deals with Sonnets and suggests the conversation might go: >Q: Please write me a sonnet on the subject of the Forth Bridge. >A : Count me out on this one. I never could write poetry. Now that I could probably do.

lloeki 12y ago

> Not something chosen from a known list of possibilities, but any task that a human could conceivably be taught to perform in less than an hour. Anything from writing a sonnet in iambic pentameter to assembling ikea furniture based on instructions. A true test of _general_ intelligence. The sad thing is I happen to know humans who would utterly fail at this (which, I know, is not invalidating the fact that a computer passing this would be worthwhile, but care has to be taken as to what that means), routinely challenging my faith in mankind.

cheatsheet 12y ago

I don't think specific problems are the issue. Even assembling furniture given instructions is not too difficult to ask a machine to solve, given the right programming language. It's the jump from one specific problem to another, without a heuristic used to leap between problem types. That would be a machine learning the ability to generalize and infer like a human. The Turing test is simple because it is easy to add complexity to something that is simple. As a human, I can convince myself that an idiot is a genius and a genius is an idiot because of introspection. Humans have imagination, and we very often don't see how that warps our perception of intelligence. We still think we can be the distant, disconnected, scientific, observer - carefully constructing a pristine and universally objective technology. But computers, out of all kinds of technology, show us how much of our humanity goes into our creations.

sumitviii 12y ago

>I would take that long bet out to 2129, and beyond. I don't see software with that level of intellectual flexibility being written in our lifetimes, or the lifetimes of our children or their children https://xkcd.com/1425/ https://xkcd.com/1425/

p1esk 12y ago
This comic is a great illustration of how a "virtually impossible" task just a few years ago (identifying a bird in a picture) is now "virtually trivial".

pharke 12y ago

I would think that your conversation with the computer would play a vital role while you were instructing it in the task it is to perform, even if it were a purely one-sided, instructional conversation the computer would still need to acknowledge truthfully whether it had correctly understood the instructions. More likely, the goal of creating such a system would be for it to be capable of engaging in active learning through inquiry but I suppose that would not be absolutely necessary given a specific task. There are also a number of assumptions when approaching problems such as composing a sonnet or assembling furniture. With a human, you assume that they have a broad understanding of the language you are communicating in, that they have some knowledge of poetry even if just in passing, have experienced cadence, rhyme, or other constructs and have prior experience of the world from which to draw 'inspiration'. You also assume that they have seen and used a variety of furniture, tools, and even just generally explored a physical environment to a degree where they are capable of manipulating objects by moving, rotating, stacking, joining, etc. or even as basic as having the concept of object permanence. They must also know all of the relevant nouns and verbs referring to the pieces and tools they will be using. If you continue to think along these lines you very quickly realize that the only reason you can teach a human being something in an hour is because they possess years or decades of previous instruction, programming, and prior learning attempts as well as a data set of visual, aural, tactile, olfactory, and spatial experiences encompassing a similar time frame.

ObviousScience 12y ago

I think you presented a false comparison. You said anything we can teach a human within an hour: do you mean a fresh human that's only just been supplied with a brain? Because you can't teach a baby anything besides basic stimulus response on that time frame, and I'm pretty sure we can get computers to do that if we rig up the stimulus response hardware in a way comparable to a newborn. What I think you actually meant, and what I think is a completely rigged test, is comparing a human with years of training and adaptive hardware modification to a computer with absolutely no training in its ability to learn a new skill or building out its knowledge base. That, of course, is a completely terrible comparison. I'll take your bet, and call it by 2050, if you're willing to compare a human with 5 years of hardware and knowledge base training and adaptation to a computer with 1 year of hardware adaption (eg, FPGA circuit reprogramming) and 1 year of knowledge base training.

geon 12y ago
I assume the AI would be pre-trained to (or beyond) the level which could be expected of an adult. The "within an hour" part refers to the test itself.
- ObviousScience 12y ago
  I'm willing to modify the bet for that: I think an AI with 20 years to be trained and have things like FPGA components adjusted can learn in 1 hour anything I could teach a person in the same amount of time with the same amount of pre-training.

emiliobumachar 12y ago

Write a sonnet in iambic pentameter an hour? Okay, but keep in mind you're holding your AI to a higher standard than the average human. Most people can't write a sonnet in an hour, though it's very easy to find those who can.

yk 12y ago

A conversation is probably the best measure of general intelligence we have, at least if you allow the judges to formulate arbitrary challenges: please solve exercise 3a from Caroll. Simply because you can encode almost all human experience as text. Having said that, I do not believe that passing the Turing test necessarily a desirable property of an AI. At the very least, a AI would need to be able to lie about its childhood. Furthermore, a AI may be truly alien, so that even if we assume a truly sentient AI it would necessarily need the ability to hide its own internal states, for example a AI may view the entire sex/gender/procreation topic as mere curiosity, and instead emulate a Human well enough that it can fool other humans.

waps 12y ago
Sure but the AI wouldn't necessarily have to respond. I think the main flaw in the Turing test is it's cultural dependence. I would argue that Eliza has passed plenty of Turing tests, mostly with people who had a chance encounter with it. But why would an AGI individual have to be able to lie about it's childhood ? It could simply have a childhood. But I understand your argument : "Human" AI is constantly trained for the function "be a human". Machine intelligence is being trained with "is human X cheating", "is this spam", "what is the french for apple", "which way will stock X go" (and other "what will <large number of humans> do in situation X" questions) ... functions. Keep in mind that your own brain is also simply running an algorithm, with "hidden state". Here's the first thing it hides pretty well : there's no single "you", there's about 300 regions in your brain, that are physically different, in different locations, and not directly linked. And if the rest of your brain gets disabled, each of those 300 regions can use your body to convince your own mother it's really you she's talking to. And the function your brain is executing is not to "have a soul" or something like that, but is mostly a behaviour-copy algorithm. It searches out other humans, observes their actions, learns to predict them and then uses what it's learned to "be you". Most people will not believe this is how it works, not even when they know quite a bit about neurons, and know pretty well that this is what our neurons do. But when it comes right down to it, you, me, everyone is an "AI algorithm" just like Google search. Aside from the shear scale, number of connections mostly, there is nothing all that remarkable about the hardware it's running on. I love how an AGI algorithm is presented in the follow up to the Battlestar Galactica series, "Caprica". Zoe Graystone explains it at one point in the series. It's fiction of course, but still. Her algorithm searches the internet, your computer, and everything and anything it can find for pictures and videos of it's "target" and then builds the function "given situation X, what would <human> do ?" and then puts the algorithm in a virtual body that looks like the human, and uploads it to V-world. She runs it on herself, a friend and accidentally on a third person she doesn't know and then proceeds to get herself blown up by a friend/terrorist (they can be both at the same time, you know) in the first episode, and their avatars wake up in V-world, sort of a facebook virtual reality edition, unaware of what has happened (and as V-world is mostly used for sex, dancing and debauchery it's not that easy to find out unless you're looking for it). Currently we don't have nearly enough data on any person to make this work ... but I am pretty convinced it is possible to make this work. This would work, since it's doing the same thing your brain does when it hears the question "do you remember feeling happy when your mother held you when you were a baby ?". First, your brain doesn't actually remember that either way. It simply works it's way through to a good answer to that question. The thing it'll do is predict how you should answer that question, given (mostly) how other humans answer that question. This is a very different problem from "lying about its childhood", specifically it's much more solvable. Can I predict how humans would answer questions about their childhood ? Certainly. Easy. Make the algorithm take enough contextual information into account (so that it both realizes what realistic answers are and either answers realistically or responds in a way that humans respond when they don't have a "good prediction" for an answer, for instance by saying they don't remember, or by changing the subject, or ...) If you have young kids you will see this process in action. You have to "teach" kids memory. When they talk only a little, if you ask them something about earlier that day, they will respond. And the response makes sense, but it's not actually what happened. You will at some point realise this and correct them. Correcting them teaches them to match up what happened with their response, because they learn to predict that that's what's expected of people. Over time this correcting them will work, and they will have learned memory (and then they learn to lie, especially on questions like "did you hit your sister ?", and when that ends badly they learn to "not lie", like adults do. "Did you hit your sister ?" "She stole my lego"). Note that the underlying memory is also a prediction. If I ask you what happened this morning, and you want to try to answer correctly, you will start with an obvious fact (which isn't necessarily the truth), like "I woke up". Your brain will take that thought, and predict (not remember) what happened next. So I woke up, ok next, brushing teeth, ok next, read cell phone messages, ok next, ... ah ! the answer to the question.

flycaliguy 12y ago

If I computer ever writes a sonnet we will now have to make sure it didn't read your post.