4 ms·

Lets start by figuring out how to control humans before we start on the AIs.

_yosefk 11y ago
Actually, unlike controlling humans, whatever that means, it's very easy to limit the impact of AI - all you have to do is treat advanced chip fabrication processes the way nuclear technology is treated, and put a legal limit on processor power so it's somewhere where it was in the late 80s. (So you have office software and email but neither YouTube nor Doom 3; a reasonable price for our survival, I find.) Fewer organizations today can manufacture chips using the latest process nodes than there are governments that can make nuclear weapons, and it's probably harder to build a chip fab in secrecy and make a large enough number of chips to power an AI than it is to build nuclear weapons in secrecy. Berglas argues for this in http://berglas.org/Articles/AIKillGrandchildren/AIKillGrandchildren.html http://berglas.org/Articles/AIKillGrandchildren/AIKillGrandc... Of course the reason governments don't do this is because almost nobody sees the risks of AI "acting on its own" (and they're probably right), nor the risks of things like rogue drones made from cheap commodity hardware (which might be sensible as well - these are more like RPGs in the wrong hands than they are like nukes in the wrong hands.)
- sawwit 11y ago
  > The risks of things like rogue drones made from cheap commodity hardware, which might be sensible as well … In Prof. Tegmark’s recent presentation at the UN he mentioned the possibility of extremely cheap drones that approach the victim's face very quickly and pierce a bolt into their brain through one of their eyes. Such a drone wouldn't require high precision projectiles which would make it cheap to build. > Of course the reason governments don't do this is because almost nobody sees the risks of AI "acting on its own"… It is near impossible to enforce something like this globally and forever. At best, it would be a near-term solution, especially so because there is a huge military and economic interest in technology and AI. Quite possibly, the only long-term solution is solving the control problem.
  - _yosefk 11y ago
    > extremely cheap drones that approach the victim's face very quickly and pierce a bolt into their brain It's not much more of a threat to society than a handgun. A WMD it is not, unless you make a lot of these and launch them at the same time, which is probably less effective than an H-bomb. (The one major difference between such a drone and a handgun is you might be able to target politicians and other people who're hard to shoot; a somewhat higher mortality rate among politicians is hardly a huge threat to society though.) > there is a huge military and economic interest in technology and AI There's also a huge interest in nuclear energy, and it doesn't follow that a consumer should be or is able to own a nuclear reactor. If anyone took the dangers seriously, it wouldn't be that hard to at least limit what consumer devices can do. Right now a single botnet made from infected consumer PCs has more raw computing power than the biggest tech company server farm, which is ridiculous if you think of rogue AI as an existential threat. Actually it's ridiculous even if you think of "properly controlled" AI in the wrong hands as an existential threat; the botnet could run AI controlled to serve the interest of an evil programmer. Nobody cares because nobody has ever seen an AI that doesn't fail the Turing test in its first couple of minutes, much less come up with anything that would be an existential threat to any society.
    - sawwit 11y ago
      > It's not much more of a threat to society than a handgun. These drones could be programmed to target specific groups of people, for example of a certain ethnicity, and attack them almost autonomously. Short range slingshot mechanisms are several orders of magnitude cheaper to build than firearms. Moreover, the inhibition threshold is much lower if you are not involved in first-hand violence. There is also a much lower risk of getting busted and no need for intricate escape planning.
      - baq 11y ago
        >Short range slingshot mechanisms you got me thinking, googling, then frowning. imagine this https://www.youtube.com/watch?v=crzXD6NjBAE https://www.youtube.com/watch?v=crzXD6NjBAE milspecced.
- wyager 11y ago
  >all you have to do is treat advanced chip fabrication processes the way nuclear technology is treated, and put a legal limit on processor power so it's somewhere where it was in the late 80s. Are you a villain in a Vernor Vinge novel? I may not like the numbers you're crunching, but I will defend to the death your right to crunch them.
amelius 11y ago
Let's figure out the human brain first. Let's say you have these two options: (1) A technology that will make you feel the best you possibly can, without any negative consequences, or (2) a robot that will do stuff for you, like cleaning your house, driving you to work. Now which one would you choose?

Actually, unlike controlling humans, whatever that means, it's very easy to limit the impact of AI - all you have to do is treat advanced chip fabrication processes the way nuclear technology is treated, and put a legal limit on processor power so it's somewhere where it was in the late 80s. (So you have office software and email but neither YouTube nor Doom 3; a reasonable price for our survival, I find.) Fewer organizations today can manufacture chips using the latest process nodes than there are governments that can make nuclear weapons, and it's probably harder to build a chip fab in secrecy and make a large enough number of chips to power an AI than it is to build nuclear weapons in secrecy. Berglas argues for this in http://berglas.org/Articles/AIKillGrandchildren/AIKillGrandchildren.html http://berglas.org/Articles/AIKillGrandchildren/AIKillGrandc... Of course the reason governments don't do this is because almost nobody sees the risks of AI "acting on its own" (and they're probably right), nor the risks of things like rogue drones made from cheap commodity hardware (which might be sensible as well - these are more like RPGs in the wrong hands than they are like nukes in the wrong hands.)

sawwit 11y ago
> The risks of things like rogue drones made from cheap commodity hardware, which might be sensible as well … In Prof. Tegmark’s recent presentation at the UN he mentioned the possibility of extremely cheap drones that approach the victim's face very quickly and pierce a bolt into their brain through one of their eyes. Such a drone wouldn't require high precision projectiles which would make it cheap to build. > Of course the reason governments don't do this is because almost nobody sees the risks of AI "acting on its own"… It is near impossible to enforce something like this globally and forever. At best, it would be a near-term solution, especially so because there is a huge military and economic interest in technology and AI. Quite possibly, the only long-term solution is solving the control problem.
- _yosefk 11y ago
  > extremely cheap drones that approach the victim's face very quickly and pierce a bolt into their brain It's not much more of a threat to society than a handgun. A WMD it is not, unless you make a lot of these and launch them at the same time, which is probably less effective than an H-bomb. (The one major difference between such a drone and a handgun is you might be able to target politicians and other people who're hard to shoot; a somewhat higher mortality rate among politicians is hardly a huge threat to society though.) > there is a huge military and economic interest in technology and AI There's also a huge interest in nuclear energy, and it doesn't follow that a consumer should be or is able to own a nuclear reactor. If anyone took the dangers seriously, it wouldn't be that hard to at least limit what consumer devices can do. Right now a single botnet made from infected consumer PCs has more raw computing power than the biggest tech company server farm, which is ridiculous if you think of rogue AI as an existential threat. Actually it's ridiculous even if you think of "properly controlled" AI in the wrong hands as an existential threat; the botnet could run AI controlled to serve the interest of an evil programmer. Nobody cares because nobody has ever seen an AI that doesn't fail the Turing test in its first couple of minutes, much less come up with anything that would be an existential threat to any society.
  - sawwit 11y ago
    > It's not much more of a threat to society than a handgun. These drones could be programmed to target specific groups of people, for example of a certain ethnicity, and attack them almost autonomously. Short range slingshot mechanisms are several orders of magnitude cheaper to build than firearms. Moreover, the inhibition threshold is much lower if you are not involved in first-hand violence. There is also a much lower risk of getting busted and no need for intricate escape planning.
    - baq 11y ago
      >Short range slingshot mechanisms you got me thinking, googling, then frowning. imagine this https://www.youtube.com/watch?v=crzXD6NjBAE https://www.youtube.com/watch?v=crzXD6NjBAE milspecced.
wyager 11y ago
>all you have to do is treat advanced chip fabrication processes the way nuclear technology is treated, and put a legal limit on processor power so it's somewhere where it was in the late 80s. Are you a villain in a Vernor Vinge novel? I may not like the numbers you're crunching, but I will defend to the death your right to crunch them.

amelius 11y ago

Let's figure out the human brain first. Let's say you have these two options: (1) A technology that will make you feel the best you possibly can, without any negative consequences, or (2) a robot that will do stuff for you, like cleaning your house, driving you to work. Now which one would you choose?