6 ms·

I'm not sure I understand what you mean by "straw man" here. The usual meaning is that to attack a straw man means to argue against a position that no one actua

by davmre 12y ago

I'm not sure I understand what you mean by "straw man" here. The usual meaning is that to attack a straw man means to argue against a position that no one actually holds, but which is easier to attack than your opponent's actual position. The concerns about the long-term future of AI are real, actual beliefs held by serious people. At this point there's a respectable literature on the potential dangers of unconstrained, powerful optimizing agents; I linked a couple of examples. These arguments are well thought through and worth engaging. By contrast, this article and many others are implicitly arguing against an actual straw man position, the position of "let's shut down AI research before it kills us all in six months", which no serious person on either side of this debate really holds (though it's understandable how someone who only read the discussion in the mainstream press could come to think this way).

AndrewKemendo 12y ago
at this point there's a respectable literature on the potential dangers of unconstrained, powerful optimizing agents, of which I linked a couple of examples. There isn't really though. Superintelligence, while an interesting book, doesn't do much more than pontificate on sci-fi futures based largely on writing from Yudkowski. Bostrom gives no clear pathway to AGI. Neither does Yudkowski in his own writings or Barrat (Our final invention). By the way Super intelligence is kind of an offshoot book from Global Catastrophic Risks. All of them take these interesting approaches, WBE, BCI, AGI etc... and just assume they will achieve the goal without looking realistically about where the field is or how it would get there. So what they do is they say: We are here right now. In some possible fantasy world we could be there. The problem is they can't draw the dots between them. For example, find me someone who can tell me what kind of hardware we need for an AGI. Can't do it, because no one has any idea. What about interface, what is the interface for an AGI? Even better (this was my thesis project): What group would have a strong enough requirement that an AGI would be required to build? Industry, Academia, Military? Ok great, can they get funding or resources for it? etc... Note, I am not saying that there is no potential that AGI could be a problem. The point here is that nobody has firm footing to say that it will definitely be a huge risk and that we need to regulate it's development. There are actually people calling for legislation to prevent/slow AGI development - see Altman's blog post and many of the writings on MIRI. So that is what I am rallying against. We need more development into AGI not less.
- maaku 12y ago
  I'm more in your camp than MIRI's but... Could you argue against the position directly? You kinda took a swerve in the middle there. Bostrom, Yudkowsky, and Barrat's positions are basically "superintelligence is possible, it will eventually happen in some way, and when it does ..." It's not within the scope of the quoted works to elaborate a technical roadmap or provide firm dates for the arrival of superhuman machine intelligence. So you would like to see unencumbered research in this area. They argue that the long term outcome of this research is existentially risky and therefore should be tightly regulated, and the sooner the better. Do you have any points regarding this particular difference of opinion?
  - csallen 12y ago
    I'm not worried about the immediate future at all. We definitely haven't reached the point where hastily-implemented regulations will do more good than harm. That said, you put it exactly right. The arguments about the potential long-term risk are persuasive regardless of where we are today. It bothers me to see this argument ignored time and time again.
    - darkmighty 12y ago
      Maybe it's precisely because the it can't be argued against that it's not much use? There's no way we could tell it will ever happen in the moment, no way to tell how intelligent we can make computers, and so on. You can't reliably argue for or against. So what's the point? I mean, you may want to form a group and discuss those scenarios, but it's not excusable to spread fear mongering in the media with no substance like that. It's unscientific in a way. We don't really need to start writing traffic rules for flying cars.
      - AndrewKemendo 12y ago
        We don't really need to start writing traffic rules for flying cars. This is a great metaphor and one I wish I would have thought up sooner.
themgt 12y ago
Really I would describe these people as bikeshedding. They're pontificating about the only abstractions they understand, some hand-wavey idea of AI and then the sci-fi short stories of the imagined dangers. Because if you can't do, fanfic.
- davmre 12y ago
  I don't pretend that an argument from authority resolves this debate, but the fact that people like Stuart Russell take these arguments seriously implies that there's a bit more substance there than you're acknowledging. To actually argue the point a little bit, the theory of expected-utility-maximizing agents is pretty much the framework in which all of mainstream AI research is situated. Yes, most current work is focused on tiny special cases, in limited domains, with a whole lot of tricks, hacks, and one-off implementations required to get decent results. You really do need a lot of specialized knowledge to be a successful researcher in deep learning, computer vision, probabilistic inference, robotics, etc. But almost all of that knowledge is ultimately in the service of trying to implement better and better approximations to optimal decision-theoretic agents. It's not an unreasonable question to ask, "what if this project succeeds?" (not at true optimality -- that's obviously excluded by computational hardness results -- but just at approximations that are as good or better than what the human brain does). Do Nick Bostrom and Eliezer Yudkowsky understand when you would use a tanh vs rectified linear nonlinearity in a deep network? Do they know the relative merits of extended vs unscented Kalman filters, MCMC vs variational inference, gradient descent vs BFGS? I don't know, but I'd guess largely not. Is it relevant to their arguments? Not really. You can do a lot of interesting and clarifying reasoning about the behavior of agents at the decision-theoretic level of abstraction, without cluttering the argument with details of current techniques that may or may not be relevant to the limitations of whatever we eventually build.
  - JoeAltmaier 12y ago
    All that talk about maximizing utility, isn't about intelligence at all. Its about a sensory feedback loop maybe. Intelligence is what lets you say "This isn't working. Maybe I should try a different approach. Maybe I should change the problem. Maybe I should get a different job". Until you're operating at that meta level, you're not talking 'intelligence' at all, just control systems.

by davmre 12y ago

AndrewKemendo 12y ago

at this point there's a respectable literature on the potential dangers of unconstrained, powerful optimizing agents, of which I linked a couple of examples. There isn't really though. Superintelligence, while an interesting book, doesn't do much more than pontificate on sci-fi futures based largely on writing from Yudkowski. Bostrom gives no clear pathway to AGI. Neither does Yudkowski in his own writings or Barrat (Our final invention). By the way Super intelligence is kind of an offshoot book from Global Catastrophic Risks. All of them take these interesting approaches, WBE, BCI, AGI etc... and just assume they will achieve the goal without looking realistically about where the field is or how it would get there. So what they do is they say: We are here right now. In some possible fantasy world we could be there. The problem is they can't draw the dots between them. For example, find me someone who can tell me what kind of hardware we need for an AGI. Can't do it, because no one has any idea. What about interface, what is the interface for an AGI? Even better (this was my thesis project): What group would have a strong enough requirement that an AGI would be required to build? Industry, Academia, Military? Ok great, can they get funding or resources for it? etc... Note, I am not saying that there is no potential that AGI could be a problem. The point here is that nobody has firm footing to say that it will definitely be a huge risk and that we need to regulate it's development. There are actually people calling for legislation to prevent/slow AGI development - see Altman's blog post and many of the writings on MIRI. So that is what I am rallying against. We need more development into AGI not less.

maaku 12y ago
I'm more in your camp than MIRI's but... Could you argue against the position directly? You kinda took a swerve in the middle there. Bostrom, Yudkowsky, and Barrat's positions are basically "superintelligence is possible, it will eventually happen in some way, and when it does ..." It's not within the scope of the quoted works to elaborate a technical roadmap or provide firm dates for the arrival of superhuman machine intelligence. So you would like to see unencumbered research in this area. They argue that the long term outcome of this research is existentially risky and therefore should be tightly regulated, and the sooner the better. Do you have any points regarding this particular difference of opinion?
- csallen 12y ago
  I'm not worried about the immediate future at all. We definitely haven't reached the point where hastily-implemented regulations will do more good than harm. That said, you put it exactly right. The arguments about the potential long-term risk are persuasive regardless of where we are today. It bothers me to see this argument ignored time and time again.
  - darkmighty 12y ago
    Maybe it's precisely because the it can't be argued against that it's not much use? There's no way we could tell it will ever happen in the moment, no way to tell how intelligent we can make computers, and so on. You can't reliably argue for or against. So what's the point? I mean, you may want to form a group and discuss those scenarios, but it's not excusable to spread fear mongering in the media with no substance like that. It's unscientific in a way. We don't really need to start writing traffic rules for flying cars.
    - AndrewKemendo 12y ago
      We don't really need to start writing traffic rules for flying cars. This is a great metaphor and one I wish I would have thought up sooner.

themgt 12y ago

Really I would describe these people as bikeshedding. They're pontificating about the only abstractions they understand, some hand-wavey idea of AI and then the sci-fi short stories of the imagined dangers. Because if you can't do, fanfic.

davmre 12y ago
I don't pretend that an argument from authority resolves this debate, but the fact that people like Stuart Russell take these arguments seriously implies that there's a bit more substance there than you're acknowledging. To actually argue the point a little bit, the theory of expected-utility-maximizing agents is pretty much the framework in which all of mainstream AI research is situated. Yes, most current work is focused on tiny special cases, in limited domains, with a whole lot of tricks, hacks, and one-off implementations required to get decent results. You really do need a lot of specialized knowledge to be a successful researcher in deep learning, computer vision, probabilistic inference, robotics, etc. But almost all of that knowledge is ultimately in the service of trying to implement better and better approximations to optimal decision-theoretic agents. It's not an unreasonable question to ask, "what if this project succeeds?" (not at true optimality -- that's obviously excluded by computational hardness results -- but just at approximations that are as good or better than what the human brain does). Do Nick Bostrom and Eliezer Yudkowsky understand when you would use a tanh vs rectified linear nonlinearity in a deep network? Do they know the relative merits of extended vs unscented Kalman filters, MCMC vs variational inference, gradient descent vs BFGS? I don't know, but I'd guess largely not. Is it relevant to their arguments? Not really. You can do a lot of interesting and clarifying reasoning about the behavior of agents at the decision-theoretic level of abstraction, without cluttering the argument with details of current techniques that may or may not be relevant to the limitations of whatever we eventually build.
- JoeAltmaier 12y ago
  All that talk about maximizing utility, isn't about intelligence at all. Its about a sensory feedback loop maybe. Intelligence is what lets you say "This isn't working. Maybe I should try a different approach. Maybe I should change the problem. Maybe I should get a different job". Until you're operating at that meta level, you're not talking 'intelligence' at all, just control systems.