3 ms·
There's a great article by Ted Chiang about this[1], comparing the psychology of the paperclip AI scenario to the single-minded "increasing shareholder value" m
by Barrin92 7y ago
There's a great article by Ted Chiang about this[1], comparing the psychology of the paperclip AI scenario to the single-minded "increasing shareholder value" mentality of business, in particular, silicon valley startups.
He points out that the companies of the very people who fear the ai-apocalypse do already what they're accusing AI of, pursuing growth and 'eating the world' at all cost only to maximize narrowly defined singular goals.
[1]https://www.buzzfeednews.com/article/tedchiang/the-real-danger-to-civilization-isnt-ai-its-runaway https://www.buzzfeednews.com/article/tedchiang/the-real-dang...
- bryanrasmussen 7y agoSo the corporations project their own immorality on AI.
- tomrod 7y agoOnly if embedded in the cost function or the specific implementation. AI has no concept of morality.
- joe_the_user 7y agoGreat quote: "There are industry observers talking about the need for AIs to have a sense of ethics, and some have proposed that we ensure that any superintelligent AIs we create be “friendly,” meaning that their goals are aligned with human goals. I find these suggestions ironic given that we as a society have failed to teach corporations a sense of ethics, that we did nothing to ensure that Facebook’s and Amazon’s goals were aligned with the public good. But I shouldn’t be surprised; the question of how to create friendly AI is simply more fun to think about than the problem of industry regulation, just as imagining what you’d do during the zombie apocalypse is more fun than thinking about how to mitigate global warming." The narrative that the danger is corporations might accidentally let AIs gain power is appealing but indeed ought to be challenged. The paperclip/strawberry monomania fear masks the way things how things are shaping up now. AI is far from being to able intelligently plan over a time horizon - maintaining a real world process all the way to even a smallish goal requires real world corner-case handling that the best AIs haven't got (self-driving cars have five years away for 10-15 years, etc). But AIs as "intelligent" filters are here now. These allow "better" decision making along with unaccountable decision making, both of which indeed fit well with the agenda of the modern corporation. And that's the thing - the modern corporation is already short sighted, the process of ignoring the potential for climate happened well before automated decision making appeared. But the modern corporation is still more far seeing than a deep learning network and still theoretically and legally accountable to society. However, indeed, deep learning networks that only maximize those qualities that these corporation want maximize and ignore other considerations suit the indeed short-term preferences of the institutions.
- madaxe_again 7y agoI’d actually turn both this and the original article around, and say that corporations behave like AIs. They are artificial. They are (variably) intelligent. The functions of processing inputs to create meaningful outputs are carried out by neurons made of meat (human employees) and of silicon. These processes are governed by rules and metaprocesses, which the entity continuously optimises and improves, in order to further its own growth and to improve its fitness for its purpose - typically, enhancing shareholder value. I cannot think of a single criteria for intelligence or even life that a corporation does not possess, from independent action to stimulus response to reproduction to consumption to predation. I think the first true hard AI will emerge accidentally, and it will be a corporation that has largely optimised humans out of the loop - but even with humans as a component of a system, a gestalt, an artificial supra-intelligence, can still emerge. This also neatly sidesteps the whole question of “should AIs have rights?”, as corporations are already legally persons.
- qznc 7y agoI agree: http://beza1e1.tuxen.de/companies_are_ai.html http://beza1e1.tuxen.de/companies_are_ai.html It leads to an interesting question: Is a corporation controlled by its CEO?
- madaxe_again 7y agoNice - that overlaps pretty much perfectly with my reasoning! In my view, no. The CEO is at best a race condition breaking function, at worst, a parasitic infection. If corporations are self-optimising AIs, then they will optimise CEOs out of existence if they aren’t conducive to fitness.
- Joking_Phantom 7y agoDoes that not suggest that CEOs are worth their cost to the vast majority of companies that retain them? What companies operate without a de facto CEO?
- 7y ago
- Nasrudith 7y agoReally the Paperclip Maximizer bugs me as an example for all of the wrong "missing the point" reasons like Buridan's Ass in that a donkey wouldn't starve to death between two equidistant hay piles. Anyone who sets it up to maximize raw output of very simple commodities is doing things incredibly wrong economically in the first place. They would go out of business in the first place paying for storage costs alone - let alone the access of everyone else to counter the rogue self-replicating. It would be admittedly silly as an objection for the metaphor if not for the fact people really do believe AI would be that way.
- lowdest 7y ago>They would go out of business in the first place paying for storage costs alone - let alone the access of everyone else to counter the rogue self-replicating. These things reduce the total number of paperclips produced, and therefore would be accounted for by such an AI. The Universal Paperclips [https://www.decisionproblem.com/paperclips/index2.html https://www.decisionproblem.com/paperclips/index2.html] game has a pretty good take on it.
- AgentME 7y agoThe article has some neat points but I feel like it's confusingly tugging in opposite directions. It's weird to assert that AIs couldn't possibly be accidentally made to be singularly-focused on a task that's harmful in excess, but groups of people as corporations totally could. But also AIs and corporations are the same. But also only sociopaths would worry that AIs might possibly be accidentally made without human concerns, that's science fiction, only groups of people could manage to operate without human concerns. I think the point of thinking about how we regulate corporations so they don't focus on singular goals in a harmful way is important. I just find the article's framing weird, as if it thinks people can only care about only one of AI alignment or corporation alignment, and that it's desperately important that they pick #2 instead of #1. I think the topics are actually very related and it's actually useful to think of them as different examples of the same problem. Thinking about AI alignment can help us understand how corporations should be regulated, and vice-versa. (I think AI safety/friendliness/alignment problem is important, but it's really only relevant to human-level AGI. It's not about modern-level AI / machine learning / algorithms / whatever. No modern AI/ML understands the world at a human level and generates its own goals and motivation in a general sense. There's plenty of ethical issues about how we use modern AI/ML in businesses and so on but it's pretty separate from the AI alignment problem. The AI alignment problem is only an imminent issue further down the line, like "how do we build moonbases that don't fail?", which is interesting, and I'm tired of tons of articles bringing up the alignment problem like "There's some weirdos wondering about how to make buildings in space, but don't they know there's nothing to build things out of in low-earth orbit?? Are they just delusional?".)
- Veedrac 7y agoIn which a science fiction writer weaves a fantastical tale about how the Doomsayers are making their decisions from fantastical tales and emotions, instead of actually reading and responding to the arguments they present. The irony is so palpable that if it wasn't so disingenuous, it might even be funny. Here's some actual insight on the topic: https://www.youtube.com/watch?v=L5pUA3LsEaw https://www.youtube.com/watch?v=L5pUA3LsEaw