5 ms·

This post seems to be a response to some criticism of Sam's opinions in other blog posts. The relevant line is this one > Superhuman machine intelligence is pr

This post seems to be a response to some criticism of Sam's opinions in other blog posts. The relevant line is this one > Superhuman machine intelligence is prima facie ridiculous. > - Many otherwise smart people, 2015 I was among those criticizing Sam's point of view, but not because I think the idea of superhuman machine intelligence is ridiculous. I don't know one way or the other whether we'll achieve it or not, but I don't consider it impossible. What I, and I believe many others as well, was criticizing was Sam's insistence that the current techniques have the potential to cause such great harm that they need to be regulated and that extreme measures need to be taken to address the possible harm of modern AI and ML techniques. This type of fear-driven reasoning is wildly out of proportion with the facts, as anyone who does research in the area of is familiar with the research in the area can attest to. Modern machine learning techniques are heuristic methods to find optima in functions over high dimensional spaces. Neural networks and support vector machines produce impressive results on image classification and other well defined problems, but are far from general purpose techniques that can be easily applied to any given problem. So, the bullet points: 1. Current ML techniques aren't anywhere near creating general intelligence, nor are there areas of research that appear likely to yield superhuman levels of intelligence in the near to moderate-term future. 2. This doesn't mean superhuman levels of machine intelligence are impossible, just that we don't have any current methods that are likely to lead to them. 3. Taking action against an ill-defined, likely phantasmic threat is neither cost effective nor helpful for the progress of AI and ML. 4. When the people who have real, expert knowledge of something all tell you one thing and the people who have something to gain from getting your attention by promoting sensational opinions and cherry-picked facts tell you something else you should rationally assess their motives and weight that when deciding who's views more closely approximate the truth.

rndn 12y ago
Regarding 1, 2 and 4: It certainly seems like a long way to create something like the human brain, but perhaps there are far simpler systems that yield general intelligence. We don't seem to be very good at predicting the invention of algorithms, so with all due respect I doubt experts can provide reliable predictions. Hardware as powerful as the simplest models of the human brain already exists and growing computational resources enable studies of neural networks at an increasing rate. I think, in that light, the claim that the breakthrough is far in the future seems to be about as strong as saying it could happen within the next ten years. I've asked this question in the discussion yesterday as well, but I didn't receive any response: What kind of breakthrough do you expect after which it would be justified to be concerned about general intelligence? And how do you justify that it's unlikely to expect that surpassing human intelligence is not just a matter of scaling it up at that point, i.e. that the invention can be regulated quickly enough once it exists?
- jeffreyrogers 12y ago
  The difficulty with general intelligence isn't so much that any particular problem is hard to solve, but that creating a way to generalize from specific problems to solving any arbitrary problem is very difficult. Given enough time researchers do pretty well at creating "intelligent" machines, but these are all focused on very specific, narrow problems. For example, we can classify tweet sentiment pretty well, but we can't summarize documents yet. The reason that the general cases become more difficult is because success is less well defined and the number of possibilities is usually bigger. For example with tweet sentiment there aren't that many possibly results (in the simplest case it's just binary: positive or negative). But say you want to summarize a document, well that's much harder, since how do you even know when a document is well summarized? And assuming you can know when a document is well summarized, how do you get enough data to train your model on? But even the more general case of summarizing documents is still pretty narrow. It's just dealing with text. Now what if you want your document summarizing robot to learn when to drive your car to pick you up from you doctors appointment. Well now your robot needs to know about your schedule, it needs to know about cars, it needs to know about traffic laws, and so on. So the more you generalize the more data your machine learning algorithm needs to know to draw the proper conclusions. And that data either needs to be hard coded into it or otherwise fed into it so that it can learn. If it's hard coded then you have a labor problem, since it takes lots of time for someone to write all the things we take for granted into a way that a machine can understand and if you try to get it to learn it then you have the problem of needing to specify what a good and bad outcome is like we did above with document classification. So the reason I don't think scaling up is possible is because we don't have good ways to measure what a good outcome looks like. I don't expect a breakthrough to come, rather, I see steady progress as the norm, but I think what can improve the capabilities of current AI and ML techniques is to find some way of baking in basic facts and logic into our programs. For example, you were born with the ability to recognize when someone is angry with you or glad to see you and how to reason spatially, etc. If we can do something analogous with machines then I think that would be a big improvement. But again, it's really hard to say how to do this, since if we hard code it we have problems and if we try to learn it we also have problems, as I noted earlier.
  - 12y ago
jbhatab 12y ago
I don't think he was ever indicating that we should start making regulations right now. I got the sense that he was planning for when that technology starts to come out that we can have some plans ready. Imagine if we just planned for a lot of other technologies before they came out that we didn't think would have. We could have avoided some serious problems. It is obviously a balance of fostering innovation vs safety regulations, but some safety considerations are a must in my opinion.
stupidcar 12y ago
What about Numenta? They claim to have a general, neocortex-inspired learning algorithm. http://numenta.com/ http://numenta.com/
- fchollet 12y ago
  I refer you to this post by the founder of Numenta, where he tries to inject some sanity into the "debate": http://recode.net/2015/03/02/the-terminator-is-not-coming-the-future-will-thank-us/ http://recode.net/2015/03/02/the-terminator-is-not-coming-th...
  - stupidcar 12y ago
    The post I was replying to said that "current ML techniques aren't anywhere near creating general intelligence". I wasn't suggesting the doomsday scenarios are likely, just that some people, Jeff Hawkins at least, believe they are close to developing general intelligence. Indeed, to quote Hawkins in the very article you linked to: "What is new is that intelligent machines will soon be a reality, and this has people thinking seriously about the consequences." I don't have enough expertise in ML to judge whether he's correct, but I'd be curious to hear from the OP because it seems to contradict his claim that current ML techniques are limited to finding optima in functions over high dimensional spaces.
    - jeffreyrogers 12y ago
      So essentially the way ML works is you have some error function that you test your output against and then you have some model (neural networks and their variants seem to be performing the best currently, but different models do better at different tasks) that can be thought of as a function between inputs and outputs. Typically you have lots of inputs (for example a picture could be represented as an array of pixels, so you'd have one input for every pixel). The model then guesses how to transform the input into an output and measures the result against the error function. The goal is then to improve the model iteratively (and hopefully not overfit!) to eventually minimize the error. I'm not very familiar with Numenta or Hawkins, so take what I'm saying with a grain of salt, however, I think there are two important things to consider. First, you can have intelligence without having superhuman any sort of general intelligence. For example, the best neural networks for handwritten digit recognition can perform better than I or most other humans can at the task. However, if you asked this same neural network to predict whether an image is a dog or a cat, it wouldn't terribly. At least until you retrained it and tweaked it a bit. But it certainly wouldn't have human level performance. Similarly, the best computers can beat any human at chess. So these are all demonstrations of "intelligence", and I'm sure you can think of a lot more. Second, the problem isn't so much that any given task is very difficult (although many are), but that generalizing solving one task requiring intelligence from the specific cases we're good at (namely classification) is a really hard problem and we don't seem to be getting better at it. We're pretty good at learning a model on narrowly defined tasks, but then if you take that model and use it on something it wasn't designed for it will just give you garbage. So I'd say that intelligent machines are already something we're seeing, but machines that learn how to navigate the world and retrain themselves on arbitrary problems are a long ways off and this seems like it will be the case for the foreseeable future.
karmacondon 12y ago
Did anyone actually say that superhuman machine intelligence is prima facie ridiculous? I followed the discussion pretty closely, both on hackernews and the quotes and discussion by prominent researchers, and I don't remember reading that. It's possible that one or two people somewhere in the discussion said the idea was completely ridiculous, but it didn't seem to be the common sentiment. The main dissenting ideas I saw said that it was too far away to be considered as a threat, and that most high profile researchers weren't giving it serious consideration. I don't doubt that "otherwise smart people" have said that superhuman machine intelligence is completely improbable, but I don't know about the "many" part. There's always someone who will say anything. The closest I've seen to a consensus is disputing the timeframe for AGI/SMI, not the possibility. For that matter, I don't remember seeing a lot of people say "Computers will never play chess" or "Computers could never drive cars" or "Computers could never compete on Jeopardy". I can't recall one prominent researcher saying any of those things definitively, much less "many" people saying any of them. It seems like there might be some inserting of quotations into the mouths of others going on here.
- jeffreyrogers 12y ago
  Yes, I think your assessment of the situation is correct.
- andreyf 12y ago
  > I don't know about the "many" part I'll go there, as has everyone I've talked to about the subject (probably half a dozen or so people). I have not heard a convincing proposal of a path towards a rogue autonomous AI, and I've looked reasonably hard. I'm not sure why otherwise smart people are for some reason just now getting worried about an idea first popularized by The Terminator in 1984. > Superhuman machine intelligence is prima facie ridiculous It's not just prima facie ridiculous, it's ridiculous after thinking about it for some time. We will have problems with enormously powerful machine-augmented individuals and machine-augmented totalitarian organizations (be they corporations or governments) long before we have problems with a completely autonomous out-of-control AI.
leot 12y ago
> When the people who have real, expert knowledge of something all tell you one thing and the people who have something to gain from getting your attention by promoting sensational opinions and cherry-picked facts tell you something else you should rationally assess their motives and weight that when deciding who's views more closely approximate the truth. Those with deep expertise in machine intelligence (and related foundational concepts, such as philosophy of mind, linguistics, psychology, neuroscience, etc.) definitely do not "all tell you one thing" (hence the "ethics board" for DeepMind). I won't claim to be such an expert (though I have multiple degrees on related topics), but if you, e.g., review Nick Bostrom's C.V., and read his work, you'll find few people more qualified to comment. He's brought a very sober, clear-headed, and decidedly non-sensationalistic assessment to these issues, and devoted an entire book to the risk posed by "Superintelligence". When very smart, knowledgeable, thoughtful, and seemingly well-adjusted people are willing to put themselves "out there", it's worth paying attention. Exponential change looks tiny until it's really not. If recursively self-improving A.I. is possible, it might only require one relatively short bit of code to get off the ground, and then it's basically game-over (depending on the A.I.'s objective function). Many people possess imaginations rich enough to see how this could come to be. Further, claiming that something which has a clear and rational path to becoming dangerous doesn't "have the potential to cause such great harm", especially when lots of relevant trend lines are nearing vertical, is extremely foolish. It is incredibly easy to unsympathetically criticize another viewpoint, especially when that viewpoint is outside of the mainstream. Those espousing such "sensational opinions" rarely win more friends than they lose, as your (and many others') comments would attest to.
- AndrewKemendo 12y ago
  something which has a clear and rational path to becoming dangerous But this is exactly the point, it DOESN'T have a clear and rational path. Go read Superintelligence again, or go read Global Catastrophic Risks or any of the other books like "Our last invention." All of it, across the board is wild speculation about paperclip maximizers and out of control drones. There is no path, no one has a path - not even AGI researchers, the people trying to build the thing for god's sake!!
  - 12y ago

rndn 12y ago

Regarding 1, 2 and 4: It certainly seems like a long way to create something like the human brain, but perhaps there are far simpler systems that yield general intelligence. We don't seem to be very good at predicting the invention of algorithms, so with all due respect I doubt experts can provide reliable predictions. Hardware as powerful as the simplest models of the human brain already exists and growing computational resources enable studies of neural networks at an increasing rate. I think, in that light, the claim that the breakthrough is far in the future seems to be about as strong as saying it could happen within the next ten years. I've asked this question in the discussion yesterday as well, but I didn't receive any response: What kind of breakthrough do you expect after which it would be justified to be concerned about general intelligence? And how do you justify that it's unlikely to expect that surpassing human intelligence is not just a matter of scaling it up at that point, i.e. that the invention can be regulated quickly enough once it exists?

jeffreyrogers 12y ago
The difficulty with general intelligence isn't so much that any particular problem is hard to solve, but that creating a way to generalize from specific problems to solving any arbitrary problem is very difficult. Given enough time researchers do pretty well at creating "intelligent" machines, but these are all focused on very specific, narrow problems. For example, we can classify tweet sentiment pretty well, but we can't summarize documents yet. The reason that the general cases become more difficult is because success is less well defined and the number of possibilities is usually bigger. For example with tweet sentiment there aren't that many possibly results (in the simplest case it's just binary: positive or negative). But say you want to summarize a document, well that's much harder, since how do you even know when a document is well summarized? And assuming you can know when a document is well summarized, how do you get enough data to train your model on? But even the more general case of summarizing documents is still pretty narrow. It's just dealing with text. Now what if you want your document summarizing robot to learn when to drive your car to pick you up from you doctors appointment. Well now your robot needs to know about your schedule, it needs to know about cars, it needs to know about traffic laws, and so on. So the more you generalize the more data your machine learning algorithm needs to know to draw the proper conclusions. And that data either needs to be hard coded into it or otherwise fed into it so that it can learn. If it's hard coded then you have a labor problem, since it takes lots of time for someone to write all the things we take for granted into a way that a machine can understand and if you try to get it to learn it then you have the problem of needing to specify what a good and bad outcome is like we did above with document classification. So the reason I don't think scaling up is possible is because we don't have good ways to measure what a good outcome looks like. I don't expect a breakthrough to come, rather, I see steady progress as the norm, but I think what can improve the capabilities of current AI and ML techniques is to find some way of baking in basic facts and logic into our programs. For example, you were born with the ability to recognize when someone is angry with you or glad to see you and how to reason spatially, etc. If we can do something analogous with machines then I think that would be a big improvement. But again, it's really hard to say how to do this, since if we hard code it we have problems and if we try to learn it we also have problems, as I noted earlier.
- 12y ago

jbhatab 12y ago

I don't think he was ever indicating that we should start making regulations right now. I got the sense that he was planning for when that technology starts to come out that we can have some plans ready. Imagine if we just planned for a lot of other technologies before they came out that we didn't think would have. We could have avoided some serious problems. It is obviously a balance of fostering innovation vs safety regulations, but some safety considerations are a must in my opinion.

stupidcar 12y ago

What about Numenta? They claim to have a general, neocortex-inspired learning algorithm. http://numenta.com/ http://numenta.com/

fchollet 12y ago
I refer you to this post by the founder of Numenta, where he tries to inject some sanity into the "debate": http://recode.net/2015/03/02/the-terminator-is-not-coming-the-future-will-thank-us/ http://recode.net/2015/03/02/the-terminator-is-not-coming-th...
- stupidcar 12y ago
  The post I was replying to said that "current ML techniques aren't anywhere near creating general intelligence". I wasn't suggesting the doomsday scenarios are likely, just that some people, Jeff Hawkins at least, believe they are close to developing general intelligence. Indeed, to quote Hawkins in the very article you linked to: "What is new is that intelligent machines will soon be a reality, and this has people thinking seriously about the consequences." I don't have enough expertise in ML to judge whether he's correct, but I'd be curious to hear from the OP because it seems to contradict his claim that current ML techniques are limited to finding optima in functions over high dimensional spaces.
  - jeffreyrogers 12y ago
    So essentially the way ML works is you have some error function that you test your output against and then you have some model (neural networks and their variants seem to be performing the best currently, but different models do better at different tasks) that can be thought of as a function between inputs and outputs. Typically you have lots of inputs (for example a picture could be represented as an array of pixels, so you'd have one input for every pixel). The model then guesses how to transform the input into an output and measures the result against the error function. The goal is then to improve the model iteratively (and hopefully not overfit!) to eventually minimize the error. I'm not very familiar with Numenta or Hawkins, so take what I'm saying with a grain of salt, however, I think there are two important things to consider. First, you can have intelligence without having superhuman any sort of general intelligence. For example, the best neural networks for handwritten digit recognition can perform better than I or most other humans can at the task. However, if you asked this same neural network to predict whether an image is a dog or a cat, it wouldn't terribly. At least until you retrained it and tweaked it a bit. But it certainly wouldn't have human level performance. Similarly, the best computers can beat any human at chess. So these are all demonstrations of "intelligence", and I'm sure you can think of a lot more. Second, the problem isn't so much that any given task is very difficult (although many are), but that generalizing solving one task requiring intelligence from the specific cases we're good at (namely classification) is a really hard problem and we don't seem to be getting better at it. We're pretty good at learning a model on narrowly defined tasks, but then if you take that model and use it on something it wasn't designed for it will just give you garbage. So I'd say that intelligent machines are already something we're seeing, but machines that learn how to navigate the world and retrain themselves on arbitrary problems are a long ways off and this seems like it will be the case for the foreseeable future.

karmacondon 12y ago

Did anyone actually say that superhuman machine intelligence is prima facie ridiculous? I followed the discussion pretty closely, both on hackernews and the quotes and discussion by prominent researchers, and I don't remember reading that. It's possible that one or two people somewhere in the discussion said the idea was completely ridiculous, but it didn't seem to be the common sentiment. The main dissenting ideas I saw said that it was too far away to be considered as a threat, and that most high profile researchers weren't giving it serious consideration. I don't doubt that "otherwise smart people" have said that superhuman machine intelligence is completely improbable, but I don't know about the "many" part. There's always someone who will say anything. The closest I've seen to a consensus is disputing the timeframe for AGI/SMI, not the possibility. For that matter, I don't remember seeing a lot of people say "Computers will never play chess" or "Computers could never drive cars" or "Computers could never compete on Jeopardy". I can't recall one prominent researcher saying any of those things definitively, much less "many" people saying any of them. It seems like there might be some inserting of quotations into the mouths of others going on here.

jeffreyrogers 12y ago
Yes, I think your assessment of the situation is correct.
andreyf 12y ago
> I don't know about the "many" part I'll go there, as has everyone I've talked to about the subject (probably half a dozen or so people). I have not heard a convincing proposal of a path towards a rogue autonomous AI, and I've looked reasonably hard. I'm not sure why otherwise smart people are for some reason just now getting worried about an idea first popularized by The Terminator in 1984. > Superhuman machine intelligence is prima facie ridiculous It's not just prima facie ridiculous, it's ridiculous after thinking about it for some time. We will have problems with enormously powerful machine-augmented individuals and machine-augmented totalitarian organizations (be they corporations or governments) long before we have problems with a completely autonomous out-of-control AI.

leot 12y ago

> When the people who have real, expert knowledge of something all tell you one thing and the people who have something to gain from getting your attention by promoting sensational opinions and cherry-picked facts tell you something else you should rationally assess their motives and weight that when deciding who's views more closely approximate the truth. Those with deep expertise in machine intelligence (and related foundational concepts, such as philosophy of mind, linguistics, psychology, neuroscience, etc.) definitely do not "all tell you one thing" (hence the "ethics board" for DeepMind). I won't claim to be such an expert (though I have multiple degrees on related topics), but if you, e.g., review Nick Bostrom's C.V., and read his work, you'll find few people more qualified to comment. He's brought a very sober, clear-headed, and decidedly non-sensationalistic assessment to these issues, and devoted an entire book to the risk posed by "Superintelligence". When very smart, knowledgeable, thoughtful, and seemingly well-adjusted people are willing to put themselves "out there", it's worth paying attention. Exponential change looks tiny until it's really not. If recursively self-improving A.I. is possible, it might only require one relatively short bit of code to get off the ground, and then it's basically game-over (depending on the A.I.'s objective function). Many people possess imaginations rich enough to see how this could come to be. Further, claiming that something which has a clear and rational path to becoming dangerous doesn't "have the potential to cause such great harm", especially when lots of relevant trend lines are nearing vertical, is extremely foolish. It is incredibly easy to unsympathetically criticize another viewpoint, especially when that viewpoint is outside of the mainstream. Those espousing such "sensational opinions" rarely win more friends than they lose, as your (and many others') comments would attest to.

AndrewKemendo 12y ago
something which has a clear and rational path to becoming dangerous But this is exactly the point, it DOESN'T have a clear and rational path. Go read Superintelligence again, or go read Global Catastrophic Risks or any of the other books like "Our last invention." All of it, across the board is wild speculation about paperclip maximizers and out of control drones. There is no path, no one has a path - not even AGI researchers, the people trying to build the thing for god's sake!!
- 12y ago