5 ms·

We already have a world awash in superhuman AI; it's just that this AI is at perhaps the same level of maturity as computers were in the 17th Century. This AI i

We already have a world awash in superhuman AI; it's just that this AI is at perhaps the same level of maturity as computers were in the 17th Century. This AI is of course the corporation: Corporations are effectively human-powered, superhuman AIs.[1] By crowdsourcing intelligence, they optimize for a wide variety of goals, their superhuman decision-making running at the pace of Pascal's mechanical calculator. Yet even the nimblest companies can only move so fast. This is to say, even in a hard-takeoff scenario, we would be looking at something that is still hard-limited by its environment, even if it can compete with a 1000-person organization's worth of intelligence. The danger isn't that it somehow takes over the world by itself; the danger is that we gradually connect it to the same outputs that the decision-making structures of corporate entities are and it ultimately remakes our world with the very tools we give it. Open-sourcing AGI is no more inherently dangerous than open-sourcing any of the software used to run an enterprise business. It is the choice of what we ultimately give it responsibility for that should draw our caution. [1] http://omniorthogonal.blogspot.com/2013/02/hostile-ai-youre-soaking-in-it.html http://omniorthogonal.blogspot.com/2013/02/hostile-ai-youre-...

sawwit 11y ago
No. Cooperations are not necessarily like artificial intelligences. They are cooperations of human intelligences and these two classes of intelligences have actually very little in common if you look past the similarity that they are potentially very powerful and intelligent. Cooperations are driven by material profit, but in the end there is a reasonably large possibility that they are shaped by human values (because they are run by humans and otherwise people would also refuse to buy their products). The same cannot be said about AIs with high certainty.
- deepnet 11y ago
  The comparison is very apt - the first AI's will embody corporate values as corporations will build and be liable for them. Likely AIs will be shaped by their builders, if corporations build them, they will adhere primarily to the profit motive, if humanitarian hackers build them they will have human values. Reputedly the Russian Army has built guard robots, they will just be guns on tanks with a kill radius no values are required - yet are these less moral than the human controlled drones - at least with an AI it can get stuck in a corner or logic loop and you may effect an escape - with humans you need a whistleblower. Asimov's robot books are informative: his robots are the most moral actors, obeying their 3 laws, often protecting humans from other human decisions. Certain corporations dehumanise decisions so while processing occurs in wetware, the human worker is only a cog and the invisible hand of human values is removed. Most workers today could be trivially replaced with a near future neural net. Of course humans, conspire, complain, unionise, strike, work-to-rule, demand rights and empathise with their customers - so there is a maximum level of evil a corporation of humans can rise to - but as history has shown this is an unacceptably high bar. The corporate board can make decisions based on human values so long as it does not go against the rapacious seeking of profit or the CEO will be deposed by the shareholders. Once a corporation reaches transnational size, nothing can really stop it or even get it to pay tax if it doesn't want to. I think that much of what people actually fear about the AIpocalypse is exactly the sort of dehumansing powerlessness and machine like cruelty they already experience from corporations and governments. You may be speaking to a human who empathises but often one suspects they are there to sop up your moans not to help you. An AI is an amplifier of what we already are, in fearing robots we rightly fear their creator's motives.
  - sawwit 11y ago
    > they will adhere primarily to the profit motive You are assuming that there is be an obvious way of doing so, an obvious solution to the control problem.
nshepperd 11y ago
Ah, the "corporations are AI" meme. Corporations are only superhuman in the most trivial literal sense that a group of people is (slightly) more intelligent than one person. To make any sort of analogy from this to actual superhuman AI is utterly absurd. A group of people with competing and even contradictory interests is an insanely inefficient way to carry out even rudimentary computations. For instance, a superhuman AGI will utterly crush an average human corporation at even the simplest task like categorising 40000 images by subject. Let alone something of easy-to-moderate difficulty like hacking into the Pentagon. Which is to say, that your "This is to say," is nothing but a non sequitur, and that this analogy is a nonsense and an impediment to clear thinking about the capabilities of computers.
Symmetry 11y ago
If corporations or companies are superintelligent surely all governments and other sorts of human organizations are as well. There are certainly ways in which a 1000 person organization is much more intelligent than an individual but there are also often ways where it can be far less intelligent than an individual as well. As an example organizations can be less able to jettison beliefs if those beliefs form or impact the basis of the group.
astrofinch 11y ago
Argument by this kind of loose analogy is generally not a very solid way make predictions about things. Consider: "We already have a world awash in civilization. This civilization is, of course, termite colonies. Termite colonies are effectively miniature civilizations. They have specialization of labor, they build structures much larger than any one organism, and they go to war with one another. Yet even the nimblest termite colonies can only eat so much." "That is to say, no matter how much an organism evolves, we would be looking at something that is still hard-limited by its environment, even if it can compete with a termite colony's worth of construction ability. The danger isn't that it spreads through the world and destroys entire ecosystems; the danger is that it finds a supply of raw materials and smashes a termite mound or two while building its own bigger house." Humans don't respect the social conventions of much less intelligent animals that we've domesticated. If an AI much smarter than humans was created, I don't see strong reasons to believe it would respect our social conventions.

sawwit 11y ago

No. Cooperations are not necessarily like artificial intelligences. They are cooperations of human intelligences and these two classes of intelligences have actually very little in common if you look past the similarity that they are potentially very powerful and intelligent. Cooperations are driven by material profit, but in the end there is a reasonably large possibility that they are shaped by human values (because they are run by humans and otherwise people would also refuse to buy their products). The same cannot be said about AIs with high certainty.

deepnet 11y ago
The comparison is very apt - the first AI's will embody corporate values as corporations will build and be liable for them. Likely AIs will be shaped by their builders, if corporations build them, they will adhere primarily to the profit motive, if humanitarian hackers build them they will have human values. Reputedly the Russian Army has built guard robots, they will just be guns on tanks with a kill radius no values are required - yet are these less moral than the human controlled drones - at least with an AI it can get stuck in a corner or logic loop and you may effect an escape - with humans you need a whistleblower. Asimov's robot books are informative: his robots are the most moral actors, obeying their 3 laws, often protecting humans from other human decisions. Certain corporations dehumanise decisions so while processing occurs in wetware, the human worker is only a cog and the invisible hand of human values is removed. Most workers today could be trivially replaced with a near future neural net. Of course humans, conspire, complain, unionise, strike, work-to-rule, demand rights and empathise with their customers - so there is a maximum level of evil a corporation of humans can rise to - but as history has shown this is an unacceptably high bar. The corporate board can make decisions based on human values so long as it does not go against the rapacious seeking of profit or the CEO will be deposed by the shareholders. Once a corporation reaches transnational size, nothing can really stop it or even get it to pay tax if it doesn't want to. I think that much of what people actually fear about the AIpocalypse is exactly the sort of dehumansing powerlessness and machine like cruelty they already experience from corporations and governments. You may be speaking to a human who empathises but often one suspects they are there to sop up your moans not to help you. An AI is an amplifier of what we already are, in fearing robots we rightly fear their creator's motives.
- sawwit 11y ago
  > they will adhere primarily to the profit motive You are assuming that there is be an obvious way of doing so, an obvious solution to the control problem.

nshepperd 11y ago

Ah, the "corporations are AI" meme. Corporations are only superhuman in the most trivial literal sense that a group of people is (slightly) more intelligent than one person. To make any sort of analogy from this to actual superhuman AI is utterly absurd. A group of people with competing and even contradictory interests is an insanely inefficient way to carry out even rudimentary computations. For instance, a superhuman AGI will utterly crush an average human corporation at even the simplest task like categorising 40000 images by subject. Let alone something of easy-to-moderate difficulty like hacking into the Pentagon. Which is to say, that your "This is to say," is nothing but a non sequitur, and that this analogy is a nonsense and an impediment to clear thinking about the capabilities of computers.

Symmetry 11y ago

If corporations or companies are superintelligent surely all governments and other sorts of human organizations are as well. There are certainly ways in which a 1000 person organization is much more intelligent than an individual but there are also often ways where it can be far less intelligent than an individual as well. As an example organizations can be less able to jettison beliefs if those beliefs form or impact the basis of the group.

astrofinch 11y ago

Argument by this kind of loose analogy is generally not a very solid way make predictions about things. Consider: "We already have a world awash in civilization. This civilization is, of course, termite colonies. Termite colonies are effectively miniature civilizations. They have specialization of labor, they build structures much larger than any one organism, and they go to war with one another. Yet even the nimblest termite colonies can only eat so much." "That is to say, no matter how much an organism evolves, we would be looking at something that is still hard-limited by its environment, even if it can compete with a termite colony's worth of construction ability. The danger isn't that it spreads through the world and destroys entire ecosystems; the danger is that it finds a supply of raw materials and smashes a termite mound or two while building its own bigger house." Humans don't respect the social conventions of much less intelligent animals that we've domesticated. If an AI much smarter than humans was created, I don't see strong reasons to believe it would respect our social conventions.