4 ms·

Your opinion about how far off AGI is is a totally reasonable one, and some AI experts agree with it. Other AI experts disagree, and think that AGI is basically

Your opinion about how far off AGI is is a totally reasonable one, and some AI experts agree with it. Other AI experts disagree, and think that AGI is basically tractable with current research methods plus a breakthrough or two. Yes, there's some additional uncertainty about whether you're even asking the right people. But you can take account of that uncertainty, both by widening your error bars and by asking people from other fields as well (including philosophers, e.g. Bostrom). What you can't do is just throw up your hands and say "it's unknowable". This is all beside the original point, which is that these arguments are much more rigorously grounded than just a wave in the direction of AGI. Could they be better? Sure, and there are a lot of people who'd be interested in seeing some better estimates. But for now, they're the best we have, and they're a completely reasonable thing to base decisions on. > We can formally prove what a system is capable of. That fixes the problem. That's exactly what some of the people working on this problem are trying to do, but it's a hell of a lot harder than you make it sound. Formal methods have come a really long way, but they're not even remotely close to being able to prove that an AGI system is safe (yet).

joshuamorton 10y ago
You don't need to prove anything about an AGI system if you can prove things about its I/O capabilities. I recognize that proving things about machine learning models is very hard, I tried to do some research in that area and got practically no where. And I think you and I have very different definitions of Rigorous. Like I said, unless you'd take Wilbur Wright's thoughts on a mars mission seriously, I don't think you should give much thought to Geoff Hinton's thoughts about when we'll get AGI, and I say that having enormous respect for him and his achievements. (and using him as a simple example) Its pseudoscience. We're notoriously bad about predicting the future. I don't see any reason to trust people going on about the dangers of AGI any more than the futurologists of my parents generation, who predicted flying cars and interstellar travel, but missed smartphones.
- schoen 10y ago
  > You don't need to prove anything about an AGI system if you can prove things about its I/O capabilities. But then you can't benefit from it, or you can only benefit from it except in narrowly predefined ways.
  - joshuamorton 10y ago
    Perhaps, but if the alternative is either incredible fear of the technology, or the technology potentially killing humanity, then "making great strides in a select few fields" seems quite good.
- nbouscal 10y ago
  I really, really don't think modern AI capabilities and AGI are as dissimilar as airplanes and rockets. Neither do many AI researchers. You're obviously welcome to disagree, but you don't just get to declare their opinions invalid because you think they're out of their depth.
  - rspeer 10y ago
    I don't think AI researchers are "out of their depth" (I am one myself). And yes, there is room for different opinions. However, I believe that the ones who say sensational things about AI doomsday are the ones who are disproportionately quoted by the media.
    - joshuamorton 10y ago
      This is also true. You see all of these things like Hawking and Musk talking about the AI Apocalypse (when they aren't even experts in the field), and it gets people scared when, at this point, there's very little reason to be. On the other hand, I don't actually know Hinton's opinion on these things, and he might agree with me (in which case, you absolutely should listen to him!). But instead the loudest voices are perhaps the most ridiculous. That said, I do think that if I asked you to make a bet with me on when we would approach with even 50% confidence, your error bars would be on the order of a century.

joshuamorton 10y ago

You don't need to prove anything about an AGI system if you can prove things about its I/O capabilities. I recognize that proving things about machine learning models is very hard, I tried to do some research in that area and got practically no where. And I think you and I have very different definitions of Rigorous. Like I said, unless you'd take Wilbur Wright's thoughts on a mars mission seriously, I don't think you should give much thought to Geoff Hinton's thoughts about when we'll get AGI, and I say that having enormous respect for him and his achievements. (and using him as a simple example) Its pseudoscience. We're notoriously bad about predicting the future. I don't see any reason to trust people going on about the dangers of AGI any more than the futurologists of my parents generation, who predicted flying cars and interstellar travel, but missed smartphones.

schoen 10y ago
> You don't need to prove anything about an AGI system if you can prove things about its I/O capabilities. But then you can't benefit from it, or you can only benefit from it except in narrowly predefined ways.
- joshuamorton 10y ago
  Perhaps, but if the alternative is either incredible fear of the technology, or the technology potentially killing humanity, then "making great strides in a select few fields" seems quite good.
nbouscal 10y ago
I really, really don't think modern AI capabilities and AGI are as dissimilar as airplanes and rockets. Neither do many AI researchers. You're obviously welcome to disagree, but you don't just get to declare their opinions invalid because you think they're out of their depth.
- rspeer 10y ago
  I don't think AI researchers are "out of their depth" (I am one myself). And yes, there is room for different opinions. However, I believe that the ones who say sensational things about AI doomsday are the ones who are disproportionately quoted by the media.
  - joshuamorton 10y ago
    This is also true. You see all of these things like Hawking and Musk talking about the AI Apocalypse (when they aren't even experts in the field), and it gets people scared when, at this point, there's very little reason to be. On the other hand, I don't actually know Hinton's opinion on these things, and he might agree with me (in which case, you absolutely should listen to him!). But instead the loudest voices are perhaps the most ridiculous. That said, I do think that if I asked you to make a bet with me on when we would approach with even 50% confidence, your error bars would be on the order of a century.