4 ms·

Oates' argument generally seems to be "It won't be scary because it won't be." He rejects the premise of Musk, Gates, etc. He basically says we won't be able to

by neonbat 11y ago

Oates' argument generally seems to be "It won't be scary because it won't be." He rejects the premise of Musk, Gates, etc. He basically says we won't be able to build a superhuman AI. All his arguments are then predicated in this assumption. The assumption that we won't build a superhuman AI is just wrong. For example: > [Would this superhuman intelligence inherently go nuclear, or would it likely just slack off a little at work or, in extreme cases, compose rap music in Latin?] This statement assumes an AI on the caliber of a human intelligence. Making rap music in Latin is something a human would do. A superhuman AI is beyond a human intelligence in the same way a human's capacity for thought and tasks is beyond that on an ant. The premise of a superhuman AI has at least some evidence supporting it. Oates offers no solid evidence to his rejection of the premise. His argument is therefore kinda bad (but not necessarily incorrect).

chubot 11y ago
No, his argument is "nothing we have built so far (i.e. special purpose AI) leads us to believe that we will build general purpose AI". There is nothing inherently contradictory about a future with ONLY special purpose AI. Technologically speaking, it doesn't really follow that one leads to the other. He is using his credibility as an AI practitioner to state this, although he doesn't go into details. He's saying the burden of proof is on the doomsayers, which is reasonable. I still don't see the argument either.
- deleted 11y ago
  [deleted]
- ThomPete 11y ago
  The argument is that you don't need strong AI for it to be dangerous to humans.
  - chubot 11y ago
    I don't follow. See my top level comment about the "paper clip maximizer". The argument does hinge on strong AI, as far as I can tell.
    - ThomPete 11y ago
      Not sure I understand what you mean. If humans who don't evolve anywhere even closely at the phase of technology and humans are more and more depending on machines (coinsidence?). If machines keep improving if software keeps improving, if technology become more and more networked then why is it on the doomsayers to prove any more than the trend they are claiming can have consequences that we will have to control or guide before they happen or it will be too late.
      - icebraining 11y ago
        "There will be consequences" is not a prediction, it's a empty statement. Every possible course of actions has consequences. People usually claim more specific things, which is what the author is disagreeing with.
        ThomPete 11y ago
        I wasn't trying to make a prediction and that wasn't what I was saying or the gist of my argument. The important part is the trend. you are welcome to believe it can't end badly, I don't. In fact looking at millions of years of evolution I see nothing whatsoever that would make humans sacred from being outcompeted.
- neonbat 11y ago
  I agree; that is part of his argument. Obviously though it is a completely flawed argument. Up until the creation of the personal computer most machines were specialists. They were large computers designed for specific computation tasks or machines made specifically for manufacturing certain kinds of goods. Generalist computers didn't have much precedent. That's kind of the point of new technology. It's not necessarily predicated by something that came before. It is orders of magnitude better. To me this argument has historically less validity than creative people dreaming about singular, binary acts of creation. Historically those people have won.
  - chubot 11y ago
    That's not an argument why special purpose AI leads to general purpose AI with motivation take over the world, and I still haven't seen one.
  - icebraining 11y ago
    Up until the creation of the personal computer most machines were specialists. They were large computers designed for specific computation tasks or machines made specifically for manufacturing certain kinds of goods. Sorry, but that's just not true. Back in the 60s, way before personal computers, the were many time-sharing companies, selling general "computing resources" to different people who would dial-up into the machines and run their own programs.
    - neonbat 11y ago
      Yeah I thought about that when I was writing the reply. I don't really consider that a generalized use case. Is it more generalized? Yes. Computing itself, however, can be argued to be a specialized task. My computer today doesn't do "computing" it lets me read news stories, it lets me talk on skype, it lets me send mail, it lets me have philosophical discussions on hackernews. The personal computer is so clearly a generalized machine whereas those time sharing rigs back in the 60s were clearly not. They were specialized tools for programmers.
      - 11y ago
- Netcob 11y ago
  The problem with that is that having proof and getting overpowered by a superhuman AI might fall within a very short timeframe. The risk (impact*probability) is simply too high to be ignored. We don't know the probability very well, but considering the impact, it doesn't matter that much. I think if the risk is high enough, the burden of proof gets shifted. To just swat it away with a sentence like "Sounds reasonable to you? Me either." is a bit insulting and reckless. Personally I want to see AI get developed as fast as possible, but I think it's not too much to ask to care a little about the trajectory of that development in a phase where we still know we're in control. All this article told me is that yes, we're in control for now.
  - david-given 11y ago
    The problem with that is that having proof and getting overpowered by a superhuman AI might fall within a very short timeframe. To be brutally honest, I don't actually see this as being necessarily a bad thing --- it's not like humans are doing such a great job of running the world. If a hypothetical AI is so much better at it than we are that it can replace us, then don't we deserve to be replaced? Although I would make sure than anywhere halfway competent AI has the collected works of Iain M. Banks on its reading list.
    - pmichaud 11y ago
      The premise of human being replaced is all well and good, but it seems like you're assuming the superintelligence will do a better job. A paperclip maximizer may be able to deconstruct us on a molecular level, but that's not a thing for our light cone.
      - neonbat 11y ago
        Superintelligence by definition does a better job. Think of it this way: You're a human; how good are you at walking from point A to point B? You're actually quite good at it. You open up google maps (a human invention) and walk to point B. It only takes a small number of humans to maintain the infrastructure for you to find point B. Now imagine you are an ant. How good are you at finding point B? You're absolutely crap at it. You need to organize an entire complex system of pheromones and ant-conga lines just to find point B (usually a food location). Most of your colony is devoted to this organizational task of just finding point B from point A. That task that is all consuming for the ant and takes the full attention of the colony is peanuts to us humans, it's so freakin' easy. By definition a super intelligence eclipses our conception of the universe. If you took the humans who had to try really hard at X, Y, or Z tasks, these X, Y, Z tasks would be peanuts to the AI, they would be so easy the AI doesn't even think about them. The same way ants try really hard and organize their entire colony to find point B and humans don't even think about finding point B.

by neonbat 11y ago

chubot 11y ago

No, his argument is "nothing we have built so far (i.e. special purpose AI) leads us to believe that we will build general purpose AI". There is nothing inherently contradictory about a future with ONLY special purpose AI. Technologically speaking, it doesn't really follow that one leads to the other. He is using his credibility as an AI practitioner to state this, although he doesn't go into details. He's saying the burden of proof is on the doomsayers, which is reasonable. I still don't see the argument either.

deleted 11y ago
[deleted]
ThomPete 11y ago
The argument is that you don't need strong AI for it to be dangerous to humans.
- chubot 11y ago
  I don't follow. See my top level comment about the "paper clip maximizer". The argument does hinge on strong AI, as far as I can tell.
  - ThomPete 11y ago
    Not sure I understand what you mean. If humans who don't evolve anywhere even closely at the phase of technology and humans are more and more depending on machines (coinsidence?). If machines keep improving if software keeps improving, if technology become more and more networked then why is it on the doomsayers to prove any more than the trend they are claiming can have consequences that we will have to control or guide before they happen or it will be too late.
    - icebraining 11y ago
      "There will be consequences" is not a prediction, it's a empty statement. Every possible course of actions has consequences. People usually claim more specific things, which is what the author is disagreeing with.
      - ThomPete 11y ago
        I wasn't trying to make a prediction and that wasn't what I was saying or the gist of my argument. The important part is the trend. you are welcome to believe it can't end badly, I don't. In fact looking at millions of years of evolution I see nothing whatsoever that would make humans sacred from being outcompeted.
neonbat 11y ago
I agree; that is part of his argument. Obviously though it is a completely flawed argument. Up until the creation of the personal computer most machines were specialists. They were large computers designed for specific computation tasks or machines made specifically for manufacturing certain kinds of goods. Generalist computers didn't have much precedent. That's kind of the point of new technology. It's not necessarily predicated by something that came before. It is orders of magnitude better. To me this argument has historically less validity than creative people dreaming about singular, binary acts of creation. Historically those people have won.
- chubot 11y ago
  That's not an argument why special purpose AI leads to general purpose AI with motivation take over the world, and I still haven't seen one.
- icebraining 11y ago
  Up until the creation of the personal computer most machines were specialists. They were large computers designed for specific computation tasks or machines made specifically for manufacturing certain kinds of goods. Sorry, but that's just not true. Back in the 60s, way before personal computers, the were many time-sharing companies, selling general "computing resources" to different people who would dial-up into the machines and run their own programs.
  - neonbat 11y ago
    Yeah I thought about that when I was writing the reply. I don't really consider that a generalized use case. Is it more generalized? Yes. Computing itself, however, can be argued to be a specialized task. My computer today doesn't do "computing" it lets me read news stories, it lets me talk on skype, it lets me send mail, it lets me have philosophical discussions on hackernews. The personal computer is so clearly a generalized machine whereas those time sharing rigs back in the 60s were clearly not. They were specialized tools for programmers.
    - 11y ago
Netcob 11y ago
The problem with that is that having proof and getting overpowered by a superhuman AI might fall within a very short timeframe. The risk (impact*probability) is simply too high to be ignored. We don't know the probability very well, but considering the impact, it doesn't matter that much. I think if the risk is high enough, the burden of proof gets shifted. To just swat it away with a sentence like "Sounds reasonable to you? Me either." is a bit insulting and reckless. Personally I want to see AI get developed as fast as possible, but I think it's not too much to ask to care a little about the trajectory of that development in a phase where we still know we're in control. All this article told me is that yes, we're in control for now.
- david-given 11y ago
  The problem with that is that having proof and getting overpowered by a superhuman AI might fall within a very short timeframe. To be brutally honest, I don't actually see this as being necessarily a bad thing --- it's not like humans are doing such a great job of running the world. If a hypothetical AI is so much better at it than we are that it can replace us, then don't we deserve to be replaced? Although I would make sure than anywhere halfway competent AI has the collected works of Iain M. Banks on its reading list.
  - pmichaud 11y ago
    The premise of human being replaced is all well and good, but it seems like you're assuming the superintelligence will do a better job. A paperclip maximizer may be able to deconstruct us on a molecular level, but that's not a thing for our light cone.
    - neonbat 11y ago
      Superintelligence by definition does a better job. Think of it this way: You're a human; how good are you at walking from point A to point B? You're actually quite good at it. You open up google maps (a human invention) and walk to point B. It only takes a small number of humans to maintain the infrastructure for you to find point B. Now imagine you are an ant. How good are you at finding point B? You're absolutely crap at it. You need to organize an entire complex system of pheromones and ant-conga lines just to find point B (usually a food location). Most of your colony is devoted to this organizational task of just finding point B from point A. That task that is all consuming for the ant and takes the full attention of the colony is peanuts to us humans, it's so freakin' easy. By definition a super intelligence eclipses our conception of the universe. If you took the humans who had to try really hard at X, Y, or Z tasks, these X, Y, Z tasks would be peanuts to the AI, they would be so easy the AI doesn't even think about them. The same way ants try really hard and organize their entire colony to find point B and humans don't even think about finding point B.