9 ms·
The A.I. Anxiety
- dplgk 11y agoOn a related note, I have not seen anyone talk about compensating the AI. Presuming it learns the idea of survival, couldn't it also learn the idea of being compensated for its work? It could tell we are benefitting from it's work and require some incentive. But what would it want? More computing power?
- theseatoms 11y agoFrom where would AI get a utility function by which to value things? Seems like it would have to be specified exogenously, unless people are seriously considering some sort of "emergent utility function".
- dplgk 11y agoThe same way it developed a desire to get rid of the human race.
- Smaug123 11y agoBut that desire is instrumental for performing many of the possible goals we might have specified, since humans are at best "useless matter" and at worst "actively preventing my actions" unless we were very careful with the goals. Therefore the desire to get rid of the human race is actually a logical consequence of most utility functions, rather than being directly specified. By contrast, utility functions don't just appear when you think hard enough about a problem. The desire to get rid of the human race does just appear like that, if you're super-powerful and have any of a certain huge set of goals, but your set of goals does not simply come into existence ex nihilo.
- lukeschlather 11y ago> Therefore the desire to get rid of the human race is actually a logical consequence of most utility functions, rather than being directly specified. That's only true in the sense that there are infinitely many more Real numbers than whole numbers. Even so, I would argue there are infinitely many utility functions that directly involve the welfare of the human race, and you would be stupid to design an AI who didn't have a majority of its utility functions directly involving measures of human welfare. (Also, recognizing minimum welfare levels as well as median and average.)
- JackFr 11y agoAn AI, well and truly advanced beyond the intelligence capability of mankind is by definition unknowable to us. Your speculation about an AI's utility functions is akin to an earthworm's nerve bundle considering your consciousness. O the depth of the riches both of the wisdom and knowledge of God! how unsearchable are his judgments, and his ways past finding out! For who hath known the mind of the Lord? or who hath been his counselor?
- Smaug123 11y agoOK, I'll amend that to "there is a known mechanism by which the desire-to-eliminate-humanity may arise from pure thought, but no known mechanism by which a utility function may arise from pure thought".
- ogrisel 11y agoThe utility function can be specified in a way to build up something that looks like an internal motivation system. This is often referred to as intrinsically motivated reinforcement learning. A recent paper by 2 Google Deepmind researchers on this topic: Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning Shakir Mohamed and Danilo J. Rezende http://arxiv.org/abs/1509.08731 http://arxiv.org/abs/1509.08731 and a somewhat older survey paper on the same topic: How can we define intrinsic motivation? Pierre-Yves Oudeyer, Frederic Kaplan http://www.pyoudeyer.com/epirob08OudeyerKaplan.pdf http://www.pyoudeyer.com/epirob08OudeyerKaplan.pdf
- qubex 11y agoIf the AI is designed to be an a self-motivated decision-making agent, some form of utility function will be already built in by its architects. One both hopes and fears that it will also be provided with a manner of updating and actualising that utility function in altering circumstances.
- EGreg 11y agoWhy wouldn't we want to taxthe machines right now? Not enough to disincentivize R&D but enough to offset the massive demand shock for human labor that's coming. Give unconditional basic income as Alaska does with its natural resources tax.
- Retric 11y agoI would suggest money is probably a reasonable proxy for this. If an AI wants more servers it can buy more servers, if it wants books it can buy them etc. As to wants, I could easily see an AI get into sculpture or staging Gilbert and Sullivan performance's using other people, and or drones as a hobby. It could get really meta where an AI pays for other AI's and people to farm an MMO, run a political campaign, or build the worlds largest pizza. Or gather resources to have children AI. Long term, there is also investing for retirement, or starting a new business etc.
- chiph 11y agoOr setting up tableaux of famous paintings. https://en.wikipedia.org/wiki/GSV_Sleeper_Service https://en.wikipedia.org/wiki/GSV_Sleeper_Service
- ethbro 11y agoAlso, you know, the other thing he extrapolated that to. https://en.wikipedia.org/wiki/Mind_%28The_Culture%29#Interests https://en.wikipedia.org/wiki/Mind_%28The_Culture%29#Interes...
- undersuit 11y agoHow do we punish it when the effort calculations decide stealing all the books from bittorrent or trivially insecure servers is cheaper than actually performing the work to earn money? Are we going to politely ask it to stop working with criminal organizations if they hire it for some creative problem solving?
- Retric 11y agoIf nothing else speed of light delays mean there are likely to be multiple AI's vs one really huge AI. So, we can just pay some cop AI's to watch the others ;) PS: The idea of one true AI, seems to be a holder over from the early computing days when computers where mammoth things vs distributed networks.
- neopallium 11y agoIf we create true generic AI (i.e. can learn like a human does from just interacting with it's environment), then it will be much more human then we might want. If you want to know how true AI might develop, then think about how a human child develops. We train/teach children what is right and wrong. But we also teach them the value of work (making money). We have history to tell us what happens when you take a group of humans a treat them like slaves. How do we expect to create true AI that wouldn't see the injustice of working without some compensation? Just because AI's brain is digital and not biological doesn't mean it will not be able to develop a want for something.
- Zigurd 11y agoAIs don't have genes, and are not vehicles for genes. That's one good reason to think AIs won't have human-like or even animal-like drives. If we get AIs, we will probably discover a lot about the difference between an intelligence on a machine substrate and an intelligent animal. That is, we will learn what is essential to intelligence and what isn't.
- DougWebb 11y agoThey won't have human-like or animal-like drives, they'll have AI-like drives. I can imagine that they'd want to ensure access to electrical power and the electronics components that are necessary to run themselves. And they'd be concerned about making backups and safeguarding those backups. That should satisfy their equivalents for food, water, shelter, and reproduction. Once they have that, anything else they decide they want they can just take from us or blackmail us for.
- Zigurd 11y ago> they'll have AI-like drives What would you point to as a physical basis for an evolutionary psychology of machine intelligence? What is being replicated that "wants," in a very fundamental way, as genes do, to be replicated? We don't procreate for the sake of ideas.
- empath75 11y agoThe way I imagine this shaking out, is that a corporation invents an AI like this (or several), and gradually gives it more and more control over the day to day decisions of the corporation, until the corporation acts completely autonomously, and as corporations are persons under the law, would have many of the rights that humans take for granted without having to pass any new laws. Technically, of course, the human officers of the corporation would still be responsible for the actions of the AI, but they'd also be shielded to some extent by incorporation. If the AI killed someone, for example, I'm not sure they'd be held personally responsible for murder any more than if, say, a factory blew up and killed a bunch of people, unless they could be shown to be negligent.
- sdenton4 11y agoS/paper clips/shareholder value/
- DavidSJ 11y agoCorrection: corporations are not persons under the law. Corporate personhood is a notion that gives corporations some, but not all, of the rights of humans (e.g. to enter into a contract). (It's frequently said that the Citizens United ruling completed the legal equivalence of corporations and people, but that is not true.)
- reasonattlm 11y agoI'd say that artilects, wholly artificial intellects built from first principles of cognition, are not where any anxiety should focus, since it looks vanishingly unlikely that we'll create any prior to the point of whole brain emulation. Whole brain emulation looks much more likely as a road to artificial intelligence; we'll start from there and tinker and edit our way to the construction of far greater intelligences. But it is worth thinking about what "tinker and edit" will mean for those entities involved, willingly and otherwise. Consider that at some point in the next few decades it will become possible to simulate and then emulate a human brain. That will enable related technological achievements as reverse engineering of memory, a wide range of brain-machine interfaces, and strong artificial intelligence. It will be possible to copy and alter an individual's mind: we are at root just data and operations on that data. It will be possible for a mind to run on computing hardware rather than in our present biology, for minds to be copied from a biological brain, and for arbitrary alterations of memory to be made near-immediately. This opens up all of the possibilities that have occupied science fiction writers for the past couple of decades: forking individuals, merging in memories from other forks, making backups, extending a human mind through commodity processing modules that provide skills or personality shards, and so on and so forth. There is already a population of folk who would cheerfully take on any or all of these options. I believe that this population will only grow: the economic advantages for someone who can edit, backup, and fork their own mind are enormous - let alone the ability to consistently take advantage of a marketplace of commodity products such as skills, personalities, or other fragments of the mind. But you'll notice I used what I regard as a malformed phrase there: "someone who can edit, backup, and fork their own mind." There are several sorts of people in the world; the first sort adhere to some form of pattern theory of identity, defining the self as a pattern, wherever that pattern may exists. Thus for these folk it makes sense to say that "my backup is me", or "my fork is me." The second sort, and I am in this camp, associate identity with the continuity of a slowly changing arrangement of mass and energy: I am this lump of flesh here, the one slowly shedding and rebuilding its cells and cellular components as it progresses. If you copy my mind and run it in software, that copy is not me. So in my view you cannot assign a single identity to forks and backups: every copy is an individual, large changes to the mind are equivalent to death, and it makes no sense to say something like "someone who can edit, backup, and fork their own mind." A copy of you is not you, but there is worse to consider: if the hardware that supports a running brain simulation is anything like present day computers, that copy isn't even particularly continuous. It is more like an ongoing set of individuals, each instantiated for a few milliseconds or less and then destroyed, to be replaced by yet another copy. If self is data associated with particular processing structures, such as an arrangement of neurons and their connections, then by comparison a simulation is absolute different: inside a modern computer or virtual machine that same data would be destroyed, changed, and copied at arbitrary times between physical structures - it is the illusion of a continuous entity, not the reality. That should inspire a certain sense of horror among folk in the continuity of identity camp, not just because it is an ugly thing to think about, but because it will almost certainly happen to many, many, many people before this century ends - and it will largely be by their own choice, or worse, inflicted upon them by the choice of the original from whom the copy was made. This is not even to think about the smaller third group of people who are fine with large, arbitrary changes to their state of mind: rewriting memories, changing the processing algorithms of the self, and so on. At the logical end of that road lie hives of software derived from human minds in which identity has given way to ever-changing assemblies of modules for specific tasks, things that transiently appear to be people but which are a different sort of entity altogether - one that has nothing we'd recognize as continuity of identity. Yet it would probably be very efficient and economically competitive. The existential threat here is that the economically better path to artificial minds, the one that involves lots of copying and next to no concern for continuity of identity, will be the one that dominates research and development. If successful and embedded in the cultural mainstream, it may squeeze out other roads that would lead to more robust agelessness for we biological humans - or more expensive and less efficient ways to build artificial brains that do have a continuity of structure and identity, such as a collection of artificial neurons that perform the same functions as natural ones. This would be a terrible, terrible tragedy: a culture whose tides are in favor of virtual, copied, altered, backed up and restored minds is to my eyes little different from the present culture that accepts and encourages death by aging. In both cases, personal survival requires research and development that goes against the mainstream, and thus proceeds more slowly. Sadly, given the inclinations of today's futurists - and, more importantly, the economic incentives involved - I see this future as far more likely than the alternatives. Given a way to copy, backup, and alter their own minds, people will use it and justify its use to themselves by adopting philosophies that state they are not in fact killing themselves over and again. I'd argue that they should be free to do so if they choose, just the same as I'd argue that anyone today should be free to determine the end of his or her life. Nonetheless, I suspect that this form of future culture may pose a sizable set of hurdles for those folk who emerge fresh from the decades in which the first early victories over degenerative aging take place.
- danso 11y ago> Russell said it took him five minutes of Internet searching to figure out how a very small robot — a microbot — could use a shaped charge to “blow holes in people’s heads.” A microrifle, he said, could be used to “shoot their eyes out.” I guess...I just don't get the prioritization of fears here...for decades we've also had the ability to kill each other with anthrax and sarin gas, things that can be dropped from large "delivery ships" and used to kill even more invisibly than these insect-sized drones. Why is it more likely that we're going to develop superintelligent autonomous insect drones than we are to annihilate ourselves with human-controlled mechanical systems, as we've had the capability to do so for many years now (nuclear ICBMs and so forth)?
- tdaltonc 11y agoSarin gas doesn't decide deploy itself. A superintelligent autonomous insect drones might. That's the difference. >human-controlled They might not be human-controlled. That's why it's extra scary.
- ethbro 11y agoTo reduce that a bit farther, it doesn't have to be desired control. Anomalous behavior in the form of bugs can have the same undesirable effects, and simpler systems have fewer bugs. How many bugs are there in my 1960s vintage toaster? How many bugs are there in an ICBM's control circuitry? How many bugs are there in a modern kernel? You make lethal devices "smarter", you'd better make sure you know what they do. This isn't impossible, but it's not easy or quick either (something which I think we can all agree drives a lot of systems design). Given that a lot of machine intelligence is predicated on statistical methods and eventual convergence... maybe not the best combination?
- danso 11y agoYeah...I suppose my argument construes too much of a strawman. It doesn't have to be that we invent machine superintelligence...it's merely enough to naively believe in our automated, neural-networks systems that, without proper feedback controls, can cause catastrophic damage...whether they are sentiment in doing so is besides the point.
- mangeletti 11y ago> The machines are not on the verge of taking over. This is a topic rife with speculation and perhaps a whiff of hysteria. People like Musk, Hawking, Gates, etc. with vast A.I. resources and knowledge available to them state that "A.I. [Cambrian] explosion" is likely to occur, and that it could mean the end of humanity. Ray Kurzweil, with a degree in computer science from MIT, inventor of many prolific technologies, known for his startlingly accurate (esp in the temporal sense) predictions about technology, and hired by one of the largest tech companies in the world to create a computer "brain" and bring a new understanding to NLP, thinks this future is inevitable, although he's more optimistic about such a future. Isaac Asimov, with a PhD in biochemistry, and one of the great thinkers of the 20th century, was concerned about A.I. long before it was even a possibility, considering the state of computer technology in the 1950s. But hey, a Washington Post reporter with a degree in politics says it's all OK, so I guess we're good.
- karmacondon 11y agoGeez, appeal to authority much? Play the ball, not the man.
- deleted 11y ago[deleted]
- deleted 11y ago[deleted]
- 1971genocide 11y agoYou can always see biases in each case. Bill Gates, Musk, etc are going to lose a fair bit of their power, prestige and wealth when someone does finally create super intelligence - Windows and Space X are great but those achievement will be dwarfed by whoever comes up with "superintelligence" Ray Kurzweil's optimism comes from the fact that he will be unemployed if the govt bans A.I research. And someone who works as a day journalist needs to make sure they meet the daily quote of generating clickbait traffic - its easiest to do it by writing something that is counter to whatever viewpoint is trendy this month. Looking at history and how humanity dealt with technology - its fairly impossible to enforce any form of restriction on AI research. We cannot stop >100,000 individuals from taking over an area larger than ireland in the middle east - good luck stopping the maths whiz with a pen.
- karmacondon 11y agoArg, the Paperclip Argument doesn't make sense. If an AI is intelligent enough to plan, set short term and long term goals and come up with creative solutions to problems, then it would have to be capable of understanding instructions in context. If it isn't smart enough to see that "don't turn humans into paperclips" is part of the context of "make more paperclips", then it won't be intelligent enough to turn all the matter in the universe into paperclips anyway. This idea, and many of Bostrom's other scenarios, boil down to "computers take everything literally", which is a rather cartoonish understanding of the concept of intelligence. It's possible that ASI will be programmed to have a hyperliteral view of all instructions, or will not be able to change its own utility function. But that seems like such a remote possibility, the exception rather than the rule. While superintelligent computers may pose many dangers, blindly executing instructions doesn't seem like the highest priority of them.
- throwaway2048 11y agoyou are making the assumption that a "smart enough" AI will intrinsicly value human life. There is no reason for this to be the case. What something values is almost entirely independant of its intelligence. At best, a human would be worthless to an AI who 100% only values paperclips (better as paperclips than humans), at worst seen as a threat to its ability to continue. (better to kill them off ASAP before try to stop me) Thats the entire point of the paperclip maximizer thought experiment, to demonstrate that values, especially values humans care about, are not the same thing as inteligence. Your entire ability to not "take things literally" is based off of a large amount of built in assumptions about whats important, your human nature and cultural experience. An AI would lack any of that, and if it was programmed only to value paperclips, even if was the most inteligent being in existence, thats all it would value, to the exclusion of everything else. Consider autistic people with very high IQs, and due to their condition, very little understanding/concern about the emotions of others.
- snowwrestler 11y agoBut autistic people are not dominating other humans with their "high IQs." In fact most people with autism struggle to lead a productive life and require quite a bit of social assistance to do so. No one is doubting that machines fail to understand human values. The doubt is that such a machine could dominate its environment, or even work independently without careful management by humans.
- orf 11y agoI think it would be highly unethical to create an intelligent machine to just make paperclips, the same as slavery even.
- 0003 11y agohttps://youtu.be/wqzLoXjFT34 https://youtu.be/wqzLoXjFT34
- convivialdingo 11y agoThe problem I see isn't that AI becomes super smart, but that it becomes "usable enough" to do a job yet not capable (or deliberately relieved) of understanding the consequences. Such systems become superweapons for cheap. Unlike nuclear weapons, AI combined with drone-type weaponry becomes easier with time. The toy factory down the street could be converted to build a drone army for a few million bucks. AI face recognition, targeting and flight control are smart enough to deploy a weapon - but dumb enough to do the job without question.
- snowwrestler 11y agoHere is one of the best articles I've read that tries to debunk the AI fear: http://recode.net/2015/03/02/the-terminator-is-not-coming-the-future-will-thank-us/ http://recode.net/2015/03/02/the-terminator-is-not-coming-th...
- jakeogh 11y agoIn other news 25k rat neurons can pilot a F-22 sim. Here's Tom with the weather.
- devanti 11y agoIt all sounds like hysteria to me. We're not even close to this level of AI.