10 ms·

> Musk: I think the best defense against the misuse of AI is to empower as many people as possible to have AI. If everyone has AI powers, then there’s not any o

by vonnik 11y ago

> Musk: I think the best defense against the misuse of AI is to empower as many people as possible to have AI. If everyone has AI powers, then there’s not any one person or a small set of individuals who can have AI superpower. In a sense, we have no other defense. AI is just math and code, and I know of no way to distinguish good linear algebra from evil linear algebra. The barriers to putting that math and code together for AI, at least physically, are only slightly higher than writing "Hello World." Certainly much lower than other possible existential threats, like nuclear weapons. Two people in a basement might make significant advances in AI research. So from the start, AI appears to be impossible to regulate. If an AGI is possible, then it is inevitable. I happen to support the widespread use of AI, and see many potential benefits. (Disclosure: I'm part of an AI startup: http://www.skymind.io http://www.skymind.io) Thinking about AI is the cocaine of technologists; i.e. it makes them needlessly paranoid. But if I adopt Elon's caution toward the technology, then I'm not sure if I agree with his reasoning. If he believes in the potential harm of AI, then supporting its widespread use doesn't seem logical. If you take the quote above, and substitute the word "guns" for "AI", you basically have the NRA, and the NRA is not making the world a safer place.

mindcrime 11y ago
If you take the quote above, and substitute the word "guns" for "AI", you basically have the NRA, and the NRA is not making the world a safer place. They may actually be. Although mostly not in the way you're talking about, but there is something to be said for the dispersing of power. If one or two players have a power no one else has, there's more temptation to use it. If it's widely distributed, it seems reasonable that any one actor would be less likely to wield that power. (I admit I'm being a bit hand-wavy here on what I mean by "power" but bear with me. It's kind of an abstract point).
- vonnik 11y ago
  If we're focused on weapons of mass destruction, I prefer a world of nuclear nonproliferation to the opposite. There are relatively few nations that possess nuclear weapons, and we have very few instances of them using those weapons against anyone else. To argue against myself, I'd say that the difference between weapons and AI is that AI is more general. It's not just a killing machine. In fact, I hope that killing represents the minority of its use cases.
  - mindcrime 11y ago
    If we're focused on weapons of mass destruction, I prefer a world of nuclear nonproliferation to the opposite So do I generally speaking, but with a caveat... I think that having multiple (eg, more than 1 or 2) nuclear powers is likely a Good Thing (given that the tech exists at all). The whole MAD principle seems very likely to be one reason the world has yet to descend into nuclear war. The main reason I prefer nuclear non-proliferation though, isn't because I genuinely expect something like a US/Russia nuclear war, it's more the possibility of terrorists or non-state actors getting their hands on a nuke. It's interesting though, because these various analogies between guns, nukes and AI's don't necessarily hold up. I was about to say a lot more, but on second thought, I want to think about this more.
    - jfoutz 11y ago
      I actually prefer japan's approach. They know exactly how to build nukes. They don't have any on hand. If they're backed into a corner, they'll produce as many as they feel they need. Unfortunately during the cold war, neither side could really rely on the other to just chill out for a couple of days. So now we can deliver hundreds (thousands?) anywhere in the world in about 45 minutes.
  - deleted 11y ago
    [deleted]
athenot 11y ago
> If he believes in the potential harm of AI, then supporting its widespread use doesn't seem logical. If you take the quote above, and substitute the word "guns" for "AI", you basically have the NRA, and the NRA is not making the world a safer place. Guns are not exactly good at healing, making or creating. A better comparison would be knives. Knives can be used for stabbing and killing but also for sustenance (cooking), for healing (surgery), for arts (sculpture). So perhaps this is akin to National Cutlery Association (not sure if such an entity exists but you get the idea).
- sombremesa 11y ago
  Well, with that logic you can bring guns right back into the equation. Soldering guns, glue guns, nail guns, vaccination guns...
- nicolashahn 11y ago
  You're right, guns are pure evil. Clearly, we should take them out of the hands of cops, bodyguards, hunters, and civilians defending themselves.
  - tempestn 11y ago
    That's a strawman argument. Parent is pointing out that unlike guns, AI (and knives) have purposes other than use as a weapon, and therefore it is possible that their widespread proliferation would be good, even if that of weapons is bad. Certainly weapons (including guns) have non-evil purposes as well, but that's beside the point.
    - Lawtonfogle 11y ago
      Guns also have uses other than a weapon. They shoot, but that doesn't mean they are inherently shooting living things. Just like knives stab and cut, but do not always target living things.
  - gmac 11y ago
    Ironically, you're absolutely right. Cops, bodyguards and civilians defending themselves generally only need guns because their adversaries have guns. Just take them out of everyone's hands. I know this works, if you can make it happen, because I've seen the gun death statistics for countries with effective gun control. Hunters are a different case, but their weapons are rather different too. To be honest I wouldn't that much care about depriving them of a pastime if it meant turning US gun death figures into European ones. But that's probably unnecessary.
djokkataja 11y ago
> But if I adopt Elon's caution toward the technology, then I'm not sure if I agree with his reasoning. If he believes in the potential harm of AI, then supporting its widespread use doesn't seem logical. If you take the quote above, and substitute the word "guns" for "AI", you basically have the NRA, and the NRA is not making the world a safer place. They aren't interchangeable concepts, though: guns can only be used to harm or threaten harm. Artificial general intelligence could invent ways to harm but could also invent ways to anticipate, defend against, prevent, mitigate, and repair harm. > AI appears to be impossible to regulate. It could be regulated if there were extremely authoritarian restrictions on all computing. But such a state would be 1. impractical on a global scale, 2. probably undesirable by most people and 3. fuel for extremist responses and secretive AI development. > If an AGI is possible, then it is inevitable. The only thing that could preclude the possibility of creating AGI would be if there was something magical required for human-level reasoning and consciousness. If there's no magic, then everything "human" emerges from physical phenomena. Ie short of a sudden catastrophe that wipes humanity out or makes further technological development impossible, we are going to create AGI. Personally, I think that Musk and the OpenAI group may already have a vision for how to make it happen. Figuring out how to make neural networks work at human-comparable levels for tasks like machine vision was the hardest part IMO. Once you have that, if you break down how the brain would have to work (or could work) to perform various functions and limit yourself to using neural networks as building blocks, it's not that difficult to come up with a synthetic architecture that performs all of the same functions, provided you steer clear of magical thinking about things like free will.
- eli_gottlieb 11y ago
  >Figuring out how to make neural networks work at human-comparable levels for tasks like machine vision was the hardest part IMO. Once you have that, if you break down how the brain would have to work (or could work) to perform various functions and limit yourself to using neural networks as building blocks, it's not that difficult to come up with a synthetic architecture that performs all of the same functions, provided you steer clear of magical thinking Actually, you need a number of things other than neural networks, but... nevermind, everyone here is clearly fixated on pro-Musk-Bostrom-bloc vs anti rather than on the science.
nazgulnarsil 11y ago
As a declaration it also seems a little premature. We don't yet know whether narrow AIs (extrapolations of what we have now, autonomous drones etc.) will favor defense or offense. If it favors defense we would like to spread it as widely as possible, but if it favors offense we would like to contain it if possible.
Moshe_Silnorin 11y ago
Ecosystems are hellish malthusian processes rife with extinction. I'm unsure why they think an ecosystem of superhuman agents will be any different. What sort of selective pressures would exist in such a world that would ensure the survival of humanity? It's pretty naive to think a competitive ecology would select for anything more than very intelligent entities that value replication.
- cwp 11y ago
  Physics. You're right that ecosystems are brutal. That's exactly why I'm not worried about AI as an existential threat to humanity. A few years back Bill Joy was sounding the alarm on nanotechnology. He sounded a lot like Elon Musk does today. Nanobots could be a run-away technology that would reduce the world to "grey goo". But nothing like that will ever happen. The world is already awash in nanobots. We call them bacteria. Given the right conditions, they grow at an exponential rate. But they don't consume the entire earth in a couple of days, because "the right conditions" can't be sustained. They run out of energy. They drown in their own waste. AI will be the same. Yes, machines are better than us at some things, and that list is growing all the time. But biology is ferociously good at converting sunlight into pockets of low entropy. AI such as it exists today is terrible at dealing with the physical world, and only through a tremendous amount of effort are we able to keep it running. If the machines turn on us, we can just stop repairing them.
  - Houshalter 11y ago
    The danger of nanotechnology is that it can be built better than biological life. It could outcompete it in it's own environment, or at least different environments. Solar panels can collect more energy than photosynthesis. Planes can fly faster than any bird. Guns are far more effective than any animal's weapons. Steam engines can run more efficiently than biological digestion. And we can get power from fuel sources biology doesn't touch, like fossil fuels or nuclear. We conquered the macro world before we even invented electricity. Now we are just starting to conquer the micro world. But AI is far more dangerous. It would take many many decades - perhaps centuries - of work to advance to that level. It's probably possible to build grey goo, it's just not easy or near. However AI could be much closer given the rapid rate of progress. If you make an unfriendly AI, you can't just shut it off. It could spread it's source code through the internet. And it won't tell you that it's dangerous. It will pretend to be benevolent until it no longer needs you.
astrofinch 11y ago
It might be useful to differentiate between technological equilibria that favor either attackers or defenders. * Two guys with pistols in a crowded bar: Attacker is favored. * Trench warfare during WWI: Defender is favored. * Nukes: Attacker is favored, although with the invention of nuclear submarines that could lurk under the ocean and offer credible retaliation even in the event of a devastating strike, the attacker became favored less. In general equilibria where the defender is favored seem better. Venice became one of the wealthier cities in the medieval world because it was situated in a lagoon that gave it a strong defensive advantage. The Chinese philosopher Mozi was one of the first consequentialist philosophers; during the Warring States period his followers went around advancing the state of the art in defensive siege warfare tactics: http://www.tor.com/2015/02/09/let-me-tell-you-a-little-bit-about-the-mohists/ http://www.tor.com/2015/02/09/let-me-tell-you-a-little-bit-a... Notably, I'm told that computer security current favors attackers in most areas: http://lesswrong.com/lw/dq9/work_on_security_instead_of_friendliness/ http://lesswrong.com/lw/dq9/work_on_security_instead_of_frie... (BTW the author of this post is a potential Satoshi Nakamoto candidate and knows his stuff.) In equilibria where the attacker is favored, the best solution is to have some kind of trusted central power with a monopoly on force that keeps the peace between everyone. That's what a modern state looks like. Even prisoners form their own semiformal governing structures, with designated ruling authorities, to deal with the fact that prison violence favors the attacker: http://www.econtalk.org/archives/2015/03/david_skarbek_o.html http://www.econtalk.org/archives/2015/03/david_skarbek_o.htm... Thought experiment: Let's say someone invents a personal force field that grants immunity to fists and bullets. In this world you can imagine that the need for a police force, and therefore the central state authority that manages the use of this police force, lessens considerably. The enforcement powers available to a central government also lessen considerably. This is somewhat similar to the situation with online communities. We don't have a central structure governing discussions on the web because online communities favor the defender. It's relatively easy to ban users based on their IP address or simply lock new users out of your forum entirely and therebye keep the troll mobs out. Hence the internet gives us something like Scott Alexander's idea of "Archipelago" where people get to be a part of the community they want (and deserve): http://slatestarcodex.com/2014/06/07/archipelago-and-atomic-communitarianism/ http://slatestarcodex.com/2014/06/07/archipelago-and-atomic-... Note the work done by the word "archipelago" which implies islands that are easy to defend and hard to attack (like Venice). Let's assume that superintelligent AI, when weaponized, shoves us into an entirely new and unexplored conflict equilibrium. If the new equilibrium favors defense we'd like to give everyone AIs so they can create their own atomic communities. If only a few people have AIs, they might be able to monopolize AI tech and prevent anyone else from getting one, though the info could leak eventually. If the new equilibrium favors offense we'd like to keep AIs in the hands of a small set of trusted, well-designed institutions--the same way we treat nuclear weapons. It could be that at the highest level of technological development, physics overwhelmingly favors offense. If everyone has AIs there's the possibility of destructive anarchy. In this world, research on the design of trustworthy, robust, inclusive institutions (to manage a monopoly on AI-created power) could be seen as AI safety research. The great filter http://waitbutwhy.com/2014/05/fermi-paradox.html http://waitbutwhy.com/2014/05/fermi-paradox.html weakly suggests that the new equilibrium favors offense. If the new equilibrium favors defense, even the "worst case scenario" autocratic regimes would have had plenty of time to colonize the galaxy by now. If the new equilibrium favors offense, it's entirely possible that civs reaching the superintelligent AI stage always destroy themselves in destructive anarchy and go no further. But the great filter is a very complicated topic and this line of reasoning has caveats, e.g. see http://lesswrong.com/lw/m2x/resolving_the_fermi_paradox_new_directions/ http://lesswrong.com/lw/m2x/resolving_the_fermi_paradox_new_... Anyway this entire comment is basically spitballing... point is that if $1B+ is going to be spent on this project, I would like to see at least a fraction of this capital go towards hammering issues like these out. (It'd be cool to set up an institute devoted to studying the great filter for instance.) As Enrico Fermi said: "History of science and technology has consistently taught us that scientific advances in basic understanding have sooner or later led to technical and industrial applications that have revolutionized our way of life... What is less certain, and what we all fervently hope, is that man will soon grow sufficiently adult to make good use of the powers that he acquires over nature." And I'm slightly worried that by virtue of choosing the name OpenAI, the team has committed themselves to a particular path without fully thinking it through.
chongli 11y ago
Two people in a basement might make significant advances in AI research. So from the start, AI appears to be impossible to regulate. If an AGI is possible, then it is inevitable. Not necessarily. AGI might be possible but it's not necessarily possible for two people in a basement. AGI might require some exotic computer architecture which hasn't been invented yet, for example. This would put it a lot closer to nuclear weapons in terms of barriers to existence.
- Houshalter 11y ago
  Computers are far more general purpose. Any computer can theoretically run any program, it would just be slower, at worst. Developing specialized hardware isn't out of reach, because of FPGAs.
  - chongli 11y ago
    We're talking about AGI, an area where speed can represent a difference in kind, not merely degree. The same sort of distinction goes for cryptography too. One sort of exotic computer architecture I had in mind was a massively parallel (billions of "cores"), NUMA type machine. You can't really do that with an FPGA, can you?
    - p1esk 11y ago
      If all we needed was "billions of cores", we could have done it by now by simply putting together a million of GPU cards in a large cluster. No exotic architectures needed.
      - chongli 11y ago
        That's not the same thing though. All those GPU cards have to talk to their memory through a memory bus. I'm talking about a system where memory is divided up among the cores and they all communicate with one another by message passing. This is analogous to the architecture of the human brain.
rebootthesystem 11y ago
> If you take the quote above, and substitute the word "guns" for "AI", you basically have the NRA, and the NRA is not making the world a safer place. Sorry but this "guns > NRA > bad > evil" thing is getting pretty old. I don't even own a gun and I have trouble making the "gun > bad" connection when there are hundreds of millions of guns in the US. We should have rivers of blood running down every street. We don't. Just stop it. They are not the problem. Crazy people are the problem. Crazy people with drones are a problem. That does not make drones bad. Crazy people, well, intelligent crazy people, with AI are a problem. That does not make AI bad. If we are going to have intelligent arguments let's first be intelligent. The minute someone makes the "guns > NRA > bad > evil" argument in some form I know they are making it from a total ideological corner or from political indoctrination. I challenge anyone to put the numbers to Excel and make an argument to support "guns > NRA > bad > evil" or "drones > no rules > bad > evil" or "AI > watched Terminator too many times > bad evil" without having a crazy person as part of the equation.
- ZenoArrow 11y ago
  It's a commonly held belief that gun violence is linked to mental illness, but it doesn't appear to be true: http://news.vanderbilt.edu/2014/12/mental-illness-wrong-scapegoat-shootings/ http://news.vanderbilt.edu/2014/12/mental-illness-wrong-scap...
  - rebootthesystem 11y ago
    Well, I don't know what term to use. I am using "crazy" to refer to the fact that some abnormal thought process is behind someone picking up a gun, knife, sword, axe, brick or driving a car to kill even one person or to engage in mass killings, to drive a car through a crowd, blow-up a building or use a gun or guns to mow down people in a theater. Normal people don't take a gun out of their safe, load it, throw a bunch of rounds in the back of their car, put on a bullet-proof vest and go shoot-up the neighborhood community center, church, school or mall. Those other people, the one's who would, those are the "crazies", not in a clinical sense but in that there's something seriously wrong with them that they would actually do the above. The down-votes on my original statement show I didn't do a good job of presenting my case. I do firmly believe we need to do something about access to guns. That does NOT mean taking guns away from law-abiding people. That means criminals or people who are living under circumstances that might compel them to commit crimes. The overwhelming majority of guns and gun owners do absolutely nothing to harm anyone. In fact, I'd be willing to bet most guns sit unused except for an occasional trip to the range or hunting. Some of us who would like to engage in a truly sensible conversation about guns or drones or green lasers pointed at planes and, yes, AI and robots. Yet if we come off the line making statements like "We have too many guns! The NRA is a terrorist organization!" we, in fact, have become the crazies. Because these statements are undeniably insane in the face of equally undeniable evidence. These statements only serve to instantly stop the conversation. The come back goes from "Guns don't kill people, people kill people" to "More guns in Paris would have saved lives". Both of which are undeniably factual statements. And, with that, the conversation stops. We can replace "AI", "knifes", "drones", "lasers" and more into these and similar statements. The end result is the same. Those advocating for some control become the crazies and the conversation goes nowhere. Because you are telling a perfectly harmless, responsible gun owner who might have a few guns in a safe that he is the problem. You are calling him a criminal. You are calling him "the problem". And, in his context, well, you are certifiably insane for saying so. The guy who believes he needs a gun to protect his home isn't going to take that gun and go shoot-up a theater, school, mall or community center. If we claim he is we are the crazies, not him. The fact that a number of us disagree with the need for such protection (I personally can't see the need) is irrelevant. Calling him a criminal is simply insane. I know people like that. I know people with over 20 guns in a safe. And I know they have not been out of that safe but for an occasional cleaning in ten or twenty years. And when those people hear the anti-gun, anti-NRA language spewing out they conclude "they" are insane. They are absolutely 100% correct in reaching that conclusion. Because he is not dangerous and his guns require a dangerous person in order to be loaded, carried to a destination and used to inflict harm. He is right and everyone else is crazy and the conversation stops. The right approach is to recognize that he isn't the problem. He is part of the solution. Because these types of gun owners --responsible and law abiding-- also happen to be the kind of people who abhor the use of guns to commit crimes. This is a powerful intersection of ideology gun control advocates have not woken up to. You acknowledge them as what they are, harmless law-abiding people, and ask them for help in figuring out how to reduce the incidence of guns being used to kill innocent people. Then you'll engage him, the community he represents and, yes, the NRA, in finding a solution. Becoming the crazy person who calls all of them dangerous criminals despite the overwhelming evidence to the contrary gets you nowhere. The conversation stops instantly, and rightly so. Let's not do the same with AI and technology in general. Let's not come off the line with statements that make us the crazies. Military use of AI and drones is very different subject, just like military use of guns is a different subject.
Houshalter 11y ago
The risk of AI comes from the control problem. We have a vague idea on how to build AIs. The work is just to optimize the algorithms extensively so they can run in real time. But we have no idea how to control those AIs. We can push a "reward button" every time it does something we like. But we can't prevent it from killing the programmer and stealing the button for itself. It doesn't matter at all who makes AI first. Without the control problem solved, no one will be able to use it for evil ends, even if they want to. But it's still incredibly dangerous if it gets loose in some uncontrolled setting. So this project is the exact opposite of what you would want to do if you wanted to minimize AI risk. This is rushing towards the risk as fast as possible.
- chei0aiV 11y ago
  Lets start by figuring out how to control humans before we start on the AIs.
  - _yosefk 11y ago
    Actually, unlike controlling humans, whatever that means, it's very easy to limit the impact of AI - all you have to do is treat advanced chip fabrication processes the way nuclear technology is treated, and put a legal limit on processor power so it's somewhere where it was in the late 80s. (So you have office software and email but neither YouTube nor Doom 3; a reasonable price for our survival, I find.) Fewer organizations today can manufacture chips using the latest process nodes than there are governments that can make nuclear weapons, and it's probably harder to build a chip fab in secrecy and make a large enough number of chips to power an AI than it is to build nuclear weapons in secrecy. Berglas argues for this in http://berglas.org/Articles/AIKillGrandchildren/AIKillGrandchildren.html http://berglas.org/Articles/AIKillGrandchildren/AIKillGrandc... Of course the reason governments don't do this is because almost nobody sees the risks of AI "acting on its own" (and they're probably right), nor the risks of things like rogue drones made from cheap commodity hardware (which might be sensible as well - these are more like RPGs in the wrong hands than they are like nukes in the wrong hands.)
    - 11y ago
Lawtonfogle 11y ago
>If you take the quote above, and substitute the word "guns" for "AI", you basically have the NRA, and the NRA is not making the world a safer place. I agree that replacing AI with guns makes an interesting point to consider, but is safety a good metric to use? For example, banning alcohol makes the world much safer. And if you prioritize the safety of those who follow the ban over those who don't, things get really disturbing (poisoning alcohol, like the US government did in the past). A dictator who has a monopoly on weapons/AI can be pretty safe for everyone who falls in line. But it isn't very free. So perhaps the better question is what increases overall freedom, and I think that having equal AI for everyone is the best approach.

by vonnik 11y ago

mindcrime 11y ago

If you take the quote above, and substitute the word "guns" for "AI", you basically have the NRA, and the NRA is not making the world a safer place. They may actually be. Although mostly not in the way you're talking about, but there is something to be said for the dispersing of power. If one or two players have a power no one else has, there's more temptation to use it. If it's widely distributed, it seems reasonable that any one actor would be less likely to wield that power. (I admit I'm being a bit hand-wavy here on what I mean by "power" but bear with me. It's kind of an abstract point).

vonnik 11y ago
If we're focused on weapons of mass destruction, I prefer a world of nuclear nonproliferation to the opposite. There are relatively few nations that possess nuclear weapons, and we have very few instances of them using those weapons against anyone else. To argue against myself, I'd say that the difference between weapons and AI is that AI is more general. It's not just a killing machine. In fact, I hope that killing represents the minority of its use cases.
- mindcrime 11y ago
  If we're focused on weapons of mass destruction, I prefer a world of nuclear nonproliferation to the opposite So do I generally speaking, but with a caveat... I think that having multiple (eg, more than 1 or 2) nuclear powers is likely a Good Thing (given that the tech exists at all). The whole MAD principle seems very likely to be one reason the world has yet to descend into nuclear war. The main reason I prefer nuclear non-proliferation though, isn't because I genuinely expect something like a US/Russia nuclear war, it's more the possibility of terrorists or non-state actors getting their hands on a nuke. It's interesting though, because these various analogies between guns, nukes and AI's don't necessarily hold up. I was about to say a lot more, but on second thought, I want to think about this more.
  - jfoutz 11y ago
    I actually prefer japan's approach. They know exactly how to build nukes. They don't have any on hand. If they're backed into a corner, they'll produce as many as they feel they need. Unfortunately during the cold war, neither side could really rely on the other to just chill out for a couple of days. So now we can deliver hundreds (thousands?) anywhere in the world in about 45 minutes.
- deleted 11y ago
  [deleted]

athenot 11y ago

> If he believes in the potential harm of AI, then supporting its widespread use doesn't seem logical. If you take the quote above, and substitute the word "guns" for "AI", you basically have the NRA, and the NRA is not making the world a safer place. Guns are not exactly good at healing, making or creating. A better comparison would be knives. Knives can be used for stabbing and killing but also for sustenance (cooking), for healing (surgery), for arts (sculpture). So perhaps this is akin to National Cutlery Association (not sure if such an entity exists but you get the idea).

sombremesa 11y ago
Well, with that logic you can bring guns right back into the equation. Soldering guns, glue guns, nail guns, vaccination guns...
nicolashahn 11y ago
You're right, guns are pure evil. Clearly, we should take them out of the hands of cops, bodyguards, hunters, and civilians defending themselves.
- tempestn 11y ago
  That's a strawman argument. Parent is pointing out that unlike guns, AI (and knives) have purposes other than use as a weapon, and therefore it is possible that their widespread proliferation would be good, even if that of weapons is bad. Certainly weapons (including guns) have non-evil purposes as well, but that's beside the point.
  - Lawtonfogle 11y ago
    Guns also have uses other than a weapon. They shoot, but that doesn't mean they are inherently shooting living things. Just like knives stab and cut, but do not always target living things.
- gmac 11y ago
  Ironically, you're absolutely right. Cops, bodyguards and civilians defending themselves generally only need guns because their adversaries have guns. Just take them out of everyone's hands. I know this works, if you can make it happen, because I've seen the gun death statistics for countries with effective gun control. Hunters are a different case, but their weapons are rather different too. To be honest I wouldn't that much care about depriving them of a pastime if it meant turning US gun death figures into European ones. But that's probably unnecessary.

djokkataja 11y ago

> But if I adopt Elon's caution toward the technology, then I'm not sure if I agree with his reasoning. If he believes in the potential harm of AI, then supporting its widespread use doesn't seem logical. If you take the quote above, and substitute the word "guns" for "AI", you basically have the NRA, and the NRA is not making the world a safer place. They aren't interchangeable concepts, though: guns can only be used to harm or threaten harm. Artificial general intelligence could invent ways to harm but could also invent ways to anticipate, defend against, prevent, mitigate, and repair harm. > AI appears to be impossible to regulate. It could be regulated if there were extremely authoritarian restrictions on all computing. But such a state would be 1. impractical on a global scale, 2. probably undesirable by most people and 3. fuel for extremist responses and secretive AI development. > If an AGI is possible, then it is inevitable. The only thing that could preclude the possibility of creating AGI would be if there was something magical required for human-level reasoning and consciousness. If there's no magic, then everything "human" emerges from physical phenomena. Ie short of a sudden catastrophe that wipes humanity out or makes further technological development impossible, we are going to create AGI. Personally, I think that Musk and the OpenAI group may already have a vision for how to make it happen. Figuring out how to make neural networks work at human-comparable levels for tasks like machine vision was the hardest part IMO. Once you have that, if you break down how the brain would have to work (or could work) to perform various functions and limit yourself to using neural networks as building blocks, it's not that difficult to come up with a synthetic architecture that performs all of the same functions, provided you steer clear of magical thinking about things like free will.

eli_gottlieb 11y ago
>Figuring out how to make neural networks work at human-comparable levels for tasks like machine vision was the hardest part IMO. Once you have that, if you break down how the brain would have to work (or could work) to perform various functions and limit yourself to using neural networks as building blocks, it's not that difficult to come up with a synthetic architecture that performs all of the same functions, provided you steer clear of magical thinking Actually, you need a number of things other than neural networks, but... nevermind, everyone here is clearly fixated on pro-Musk-Bostrom-bloc vs anti rather than on the science.

nazgulnarsil 11y ago

As a declaration it also seems a little premature. We don't yet know whether narrow AIs (extrapolations of what we have now, autonomous drones etc.) will favor defense or offense. If it favors defense we would like to spread it as widely as possible, but if it favors offense we would like to contain it if possible.

Moshe_Silnorin 11y ago

Ecosystems are hellish malthusian processes rife with extinction. I'm unsure why they think an ecosystem of superhuman agents will be any different. What sort of selective pressures would exist in such a world that would ensure the survival of humanity? It's pretty naive to think a competitive ecology would select for anything more than very intelligent entities that value replication.

cwp 11y ago
Physics. You're right that ecosystems are brutal. That's exactly why I'm not worried about AI as an existential threat to humanity. A few years back Bill Joy was sounding the alarm on nanotechnology. He sounded a lot like Elon Musk does today. Nanobots could be a run-away technology that would reduce the world to "grey goo". But nothing like that will ever happen. The world is already awash in nanobots. We call them bacteria. Given the right conditions, they grow at an exponential rate. But they don't consume the entire earth in a couple of days, because "the right conditions" can't be sustained. They run out of energy. They drown in their own waste. AI will be the same. Yes, machines are better than us at some things, and that list is growing all the time. But biology is ferociously good at converting sunlight into pockets of low entropy. AI such as it exists today is terrible at dealing with the physical world, and only through a tremendous amount of effort are we able to keep it running. If the machines turn on us, we can just stop repairing them.
- Houshalter 11y ago
  The danger of nanotechnology is that it can be built better than biological life. It could outcompete it in it's own environment, or at least different environments. Solar panels can collect more energy than photosynthesis. Planes can fly faster than any bird. Guns are far more effective than any animal's weapons. Steam engines can run more efficiently than biological digestion. And we can get power from fuel sources biology doesn't touch, like fossil fuels or nuclear. We conquered the macro world before we even invented electricity. Now we are just starting to conquer the micro world. But AI is far more dangerous. It would take many many decades - perhaps centuries - of work to advance to that level. It's probably possible to build grey goo, it's just not easy or near. However AI could be much closer given the rapid rate of progress. If you make an unfriendly AI, you can't just shut it off. It could spread it's source code through the internet. And it won't tell you that it's dangerous. It will pretend to be benevolent until it no longer needs you.

astrofinch 11y ago

It might be useful to differentiate between technological equilibria that favor either attackers or defenders. * Two guys with pistols in a crowded bar: Attacker is favored. * Trench warfare during WWI: Defender is favored. * Nukes: Attacker is favored, although with the invention of nuclear submarines that could lurk under the ocean and offer credible retaliation even in the event of a devastating strike, the attacker became favored less. In general equilibria where the defender is favored seem better. Venice became one of the wealthier cities in the medieval world because it was situated in a lagoon that gave it a strong defensive advantage. The Chinese philosopher Mozi was one of the first consequentialist philosophers; during the Warring States period his followers went around advancing the state of the art in defensive siege warfare tactics: http://www.tor.com/2015/02/09/let-me-tell-you-a-little-bit-about-the-mohists/ http://www.tor.com/2015/02/09/let-me-tell-you-a-little-bit-a... Notably, I'm told that computer security current favors attackers in most areas: http://lesswrong.com/lw/dq9/work_on_security_instead_of_friendliness/ http://lesswrong.com/lw/dq9/work_on_security_instead_of_frie... (BTW the author of this post is a potential Satoshi Nakamoto candidate and knows his stuff.) In equilibria where the attacker is favored, the best solution is to have some kind of trusted central power with a monopoly on force that keeps the peace between everyone. That's what a modern state looks like. Even prisoners form their own semiformal governing structures, with designated ruling authorities, to deal with the fact that prison violence favors the attacker: http://www.econtalk.org/archives/2015/03/david_skarbek_o.html http://www.econtalk.org/archives/2015/03/david_skarbek_o.htm... Thought experiment: Let's say someone invents a personal force field that grants immunity to fists and bullets. In this world you can imagine that the need for a police force, and therefore the central state authority that manages the use of this police force, lessens considerably. The enforcement powers available to a central government also lessen considerably. This is somewhat similar to the situation with online communities. We don't have a central structure governing discussions on the web because online communities favor the defender. It's relatively easy to ban users based on their IP address or simply lock new users out of your forum entirely and therebye keep the troll mobs out. Hence the internet gives us something like Scott Alexander's idea of "Archipelago" where people get to be a part of the community they want (and deserve): http://slatestarcodex.com/2014/06/07/archipelago-and-atomic-communitarianism/ http://slatestarcodex.com/2014/06/07/archipelago-and-atomic-... Note the work done by the word "archipelago" which implies islands that are easy to defend and hard to attack (like Venice). Let's assume that superintelligent AI, when weaponized, shoves us into an entirely new and unexplored conflict equilibrium. If the new equilibrium favors defense we'd like to give everyone AIs so they can create their own atomic communities. If only a few people have AIs, they might be able to monopolize AI tech and prevent anyone else from getting one, though the info could leak eventually. If the new equilibrium favors offense we'd like to keep AIs in the hands of a small set of trusted, well-designed institutions--the same way we treat nuclear weapons. It could be that at the highest level of technological development, physics overwhelmingly favors offense. If everyone has AIs there's the possibility of destructive anarchy. In this world, research on the design of trustworthy, robust, inclusive institutions (to manage a monopoly on AI-created power) could be seen as AI safety research. The great filter http://waitbutwhy.com/2014/05/fermi-paradox.html http://waitbutwhy.com/2014/05/fermi-paradox.html weakly suggests that the new equilibrium favors offense. If the new equilibrium favors defense, even the "worst case scenario" autocratic regimes would have had plenty of time to colonize the galaxy by now. If the new equilibrium favors offense, it's entirely possible that civs reaching the superintelligent AI stage always destroy themselves in destructive anarchy and go no further. But the great filter is a very complicated topic and this line of reasoning has caveats, e.g. see http://lesswrong.com/lw/m2x/resolving_the_fermi_paradox_new_directions/ http://lesswrong.com/lw/m2x/resolving_the_fermi_paradox_new_... Anyway this entire comment is basically spitballing... point is that if $1B+ is going to be spent on this project, I would like to see at least a fraction of this capital go towards hammering issues like these out. (It'd be cool to set up an institute devoted to studying the great filter for instance.) As Enrico Fermi said: "History of science and technology has consistently taught us that scientific advances in basic understanding have sooner or later led to technical and industrial applications that have revolutionized our way of life... What is less certain, and what we all fervently hope, is that man will soon grow sufficiently adult to make good use of the powers that he acquires over nature." And I'm slightly worried that by virtue of choosing the name OpenAI, the team has committed themselves to a particular path without fully thinking it through.

chongli 11y ago

Two people in a basement might make significant advances in AI research. So from the start, AI appears to be impossible to regulate. If an AGI is possible, then it is inevitable. Not necessarily. AGI might be possible but it's not necessarily possible for two people in a basement. AGI might require some exotic computer architecture which hasn't been invented yet, for example. This would put it a lot closer to nuclear weapons in terms of barriers to existence.

Houshalter 11y ago
Computers are far more general purpose. Any computer can theoretically run any program, it would just be slower, at worst. Developing specialized hardware isn't out of reach, because of FPGAs.
- chongli 11y ago
  We're talking about AGI, an area where speed can represent a difference in kind, not merely degree. The same sort of distinction goes for cryptography too. One sort of exotic computer architecture I had in mind was a massively parallel (billions of "cores"), NUMA type machine. You can't really do that with an FPGA, can you?
  - p1esk 11y ago
    If all we needed was "billions of cores", we could have done it by now by simply putting together a million of GPU cards in a large cluster. No exotic architectures needed.
    - chongli 11y ago
      That's not the same thing though. All those GPU cards have to talk to their memory through a memory bus. I'm talking about a system where memory is divided up among the cores and they all communicate with one another by message passing. This is analogous to the architecture of the human brain.

rebootthesystem 11y ago

> If you take the quote above, and substitute the word "guns" for "AI", you basically have the NRA, and the NRA is not making the world a safer place. Sorry but this "guns > NRA > bad > evil" thing is getting pretty old. I don't even own a gun and I have trouble making the "gun > bad" connection when there are hundreds of millions of guns in the US. We should have rivers of blood running down every street. We don't. Just stop it. They are not the problem. Crazy people are the problem. Crazy people with drones are a problem. That does not make drones bad. Crazy people, well, intelligent crazy people, with AI are a problem. That does not make AI bad. If we are going to have intelligent arguments let's first be intelligent. The minute someone makes the "guns > NRA > bad > evil" argument in some form I know they are making it from a total ideological corner or from political indoctrination. I challenge anyone to put the numbers to Excel and make an argument to support "guns > NRA > bad > evil" or "drones > no rules > bad > evil" or "AI > watched Terminator too many times > bad evil" without having a crazy person as part of the equation.

ZenoArrow 11y ago
It's a commonly held belief that gun violence is linked to mental illness, but it doesn't appear to be true: http://news.vanderbilt.edu/2014/12/mental-illness-wrong-scapegoat-shootings/ http://news.vanderbilt.edu/2014/12/mental-illness-wrong-scap...
- rebootthesystem 11y ago
  Well, I don't know what term to use. I am using "crazy" to refer to the fact that some abnormal thought process is behind someone picking up a gun, knife, sword, axe, brick or driving a car to kill even one person or to engage in mass killings, to drive a car through a crowd, blow-up a building or use a gun or guns to mow down people in a theater. Normal people don't take a gun out of their safe, load it, throw a bunch of rounds in the back of their car, put on a bullet-proof vest and go shoot-up the neighborhood community center, church, school or mall. Those other people, the one's who would, those are the "crazies", not in a clinical sense but in that there's something seriously wrong with them that they would actually do the above. The down-votes on my original statement show I didn't do a good job of presenting my case. I do firmly believe we need to do something about access to guns. That does NOT mean taking guns away from law-abiding people. That means criminals or people who are living under circumstances that might compel them to commit crimes. The overwhelming majority of guns and gun owners do absolutely nothing to harm anyone. In fact, I'd be willing to bet most guns sit unused except for an occasional trip to the range or hunting. Some of us who would like to engage in a truly sensible conversation about guns or drones or green lasers pointed at planes and, yes, AI and robots. Yet if we come off the line making statements like "We have too many guns! The NRA is a terrorist organization!" we, in fact, have become the crazies. Because these statements are undeniably insane in the face of equally undeniable evidence. These statements only serve to instantly stop the conversation. The come back goes from "Guns don't kill people, people kill people" to "More guns in Paris would have saved lives". Both of which are undeniably factual statements. And, with that, the conversation stops. We can replace "AI", "knifes", "drones", "lasers" and more into these and similar statements. The end result is the same. Those advocating for some control become the crazies and the conversation goes nowhere. Because you are telling a perfectly harmless, responsible gun owner who might have a few guns in a safe that he is the problem. You are calling him a criminal. You are calling him "the problem". And, in his context, well, you are certifiably insane for saying so. The guy who believes he needs a gun to protect his home isn't going to take that gun and go shoot-up a theater, school, mall or community center. If we claim he is we are the crazies, not him. The fact that a number of us disagree with the need for such protection (I personally can't see the need) is irrelevant. Calling him a criminal is simply insane. I know people like that. I know people with over 20 guns in a safe. And I know they have not been out of that safe but for an occasional cleaning in ten or twenty years. And when those people hear the anti-gun, anti-NRA language spewing out they conclude "they" are insane. They are absolutely 100% correct in reaching that conclusion. Because he is not dangerous and his guns require a dangerous person in order to be loaded, carried to a destination and used to inflict harm. He is right and everyone else is crazy and the conversation stops. The right approach is to recognize that he isn't the problem. He is part of the solution. Because these types of gun owners --responsible and law abiding-- also happen to be the kind of people who abhor the use of guns to commit crimes. This is a powerful intersection of ideology gun control advocates have not woken up to. You acknowledge them as what they are, harmless law-abiding people, and ask them for help in figuring out how to reduce the incidence of guns being used to kill innocent people. Then you'll engage him, the community he represents and, yes, the NRA, in finding a solution. Becoming the crazy person who calls all of them dangerous criminals despite the overwhelming evidence to the contrary gets you nowhere. The conversation stops instantly, and rightly so. Let's not do the same with AI and technology in general. Let's not come off the line with statements that make us the crazies. Military use of AI and drones is very different subject, just like military use of guns is a different subject.

Houshalter 11y ago

The risk of AI comes from the control problem. We have a vague idea on how to build AIs. The work is just to optimize the algorithms extensively so they can run in real time. But we have no idea how to control those AIs. We can push a "reward button" every time it does something we like. But we can't prevent it from killing the programmer and stealing the button for itself. It doesn't matter at all who makes AI first. Without the control problem solved, no one will be able to use it for evil ends, even if they want to. But it's still incredibly dangerous if it gets loose in some uncontrolled setting. So this project is the exact opposite of what you would want to do if you wanted to minimize AI risk. This is rushing towards the risk as fast as possible.

chei0aiV 11y ago
Lets start by figuring out how to control humans before we start on the AIs.
- _yosefk 11y ago
  Actually, unlike controlling humans, whatever that means, it's very easy to limit the impact of AI - all you have to do is treat advanced chip fabrication processes the way nuclear technology is treated, and put a legal limit on processor power so it's somewhere where it was in the late 80s. (So you have office software and email but neither YouTube nor Doom 3; a reasonable price for our survival, I find.) Fewer organizations today can manufacture chips using the latest process nodes than there are governments that can make nuclear weapons, and it's probably harder to build a chip fab in secrecy and make a large enough number of chips to power an AI than it is to build nuclear weapons in secrecy. Berglas argues for this in http://berglas.org/Articles/AIKillGrandchildren/AIKillGrandchildren.html http://berglas.org/Articles/AIKillGrandchildren/AIKillGrandc... Of course the reason governments don't do this is because almost nobody sees the risks of AI "acting on its own" (and they're probably right), nor the risks of things like rogue drones made from cheap commodity hardware (which might be sensible as well - these are more like RPGs in the wrong hands than they are like nukes in the wrong hands.)
  - 11y ago

Lawtonfogle 11y ago

>If you take the quote above, and substitute the word "guns" for "AI", you basically have the NRA, and the NRA is not making the world a safer place. I agree that replacing AI with guns makes an interesting point to consider, but is safety a good metric to use? For example, banning alcohol makes the world much safer. And if you prioritize the safety of those who follow the ban over those who don't, things get really disturbing (poisoning alcohol, like the US government did in the past). A dictator who has a monopoly on weapons/AI can be pretty safe for everyone who falls in line. But it isn't very free. So perhaps the better question is what increases overall freedom, and I think that having equal AI for everyone is the best approach.