7 ms·

On a related note, I have not seen anyone talk about compensating the AI. Presuming it learns the idea of survival, couldn't it also learn the idea of being com

by dplgk 11y ago

On a related note, I have not seen anyone talk about compensating the AI. Presuming it learns the idea of survival, couldn't it also learn the idea of being compensated for its work? It could tell we are benefitting from it's work and require some incentive. But what would it want? More computing power?

theseatoms 11y ago
From where would AI get a utility function by which to value things? Seems like it would have to be specified exogenously, unless people are seriously considering some sort of "emergent utility function".
- dplgk 11y ago
  The same way it developed a desire to get rid of the human race.
  - Smaug123 11y ago
    But that desire is instrumental for performing many of the possible goals we might have specified, since humans are at best "useless matter" and at worst "actively preventing my actions" unless we were very careful with the goals. Therefore the desire to get rid of the human race is actually a logical consequence of most utility functions, rather than being directly specified. By contrast, utility functions don't just appear when you think hard enough about a problem. The desire to get rid of the human race does just appear like that, if you're super-powerful and have any of a certain huge set of goals, but your set of goals does not simply come into existence ex nihilo.
    - lukeschlather 11y ago
      > Therefore the desire to get rid of the human race is actually a logical consequence of most utility functions, rather than being directly specified. That's only true in the sense that there are infinitely many more Real numbers than whole numbers. Even so, I would argue there are infinitely many utility functions that directly involve the welfare of the human race, and you would be stupid to design an AI who didn't have a majority of its utility functions directly involving measures of human welfare. (Also, recognizing minimum welfare levels as well as median and average.)
    - JackFr 11y ago
      An AI, well and truly advanced beyond the intelligence capability of mankind is by definition unknowable to us. Your speculation about an AI's utility functions is akin to an earthworm's nerve bundle considering your consciousness. O the depth of the riches both of the wisdom and knowledge of God! how unsearchable are his judgments, and his ways past finding out! For who hath known the mind of the Lord? or who hath been his counselor?
      - Smaug123 11y ago
        OK, I'll amend that to "there is a known mechanism by which the desire-to-eliminate-humanity may arise from pure thought, but no known mechanism by which a utility function may arise from pure thought".
- ogrisel 11y ago
  The utility function can be specified in a way to build up something that looks like an internal motivation system. This is often referred to as intrinsically motivated reinforcement learning. A recent paper by 2 Google Deepmind researchers on this topic: Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning Shakir Mohamed and Danilo J. Rezende http://arxiv.org/abs/1509.08731 http://arxiv.org/abs/1509.08731 and a somewhat older survey paper on the same topic: How can we define intrinsic motivation? Pierre-Yves Oudeyer, Frederic Kaplan http://www.pyoudeyer.com/epirob08OudeyerKaplan.pdf http://www.pyoudeyer.com/epirob08OudeyerKaplan.pdf
- qubex 11y ago
  If the AI is designed to be an a self-motivated decision-making agent, some form of utility function will be already built in by its architects. One both hopes and fears that it will also be provided with a manner of updating and actualising that utility function in altering circumstances.
EGreg 11y ago
Why wouldn't we want to taxthe machines right now? Not enough to disincentivize R&D but enough to offset the massive demand shock for human labor that's coming. Give unconditional basic income as Alaska does with its natural resources tax.
Retric 11y ago
I would suggest money is probably a reasonable proxy for this. If an AI wants more servers it can buy more servers, if it wants books it can buy them etc. As to wants, I could easily see an AI get into sculpture or staging Gilbert and Sullivan performance's using other people, and or drones as a hobby. It could get really meta where an AI pays for other AI's and people to farm an MMO, run a political campaign, or build the worlds largest pizza. Or gather resources to have children AI. Long term, there is also investing for retirement, or starting a new business etc.
- chiph 11y ago
  Or setting up tableaux of famous paintings. https://en.wikipedia.org/wiki/GSV_Sleeper_Service https://en.wikipedia.org/wiki/GSV_Sleeper_Service
  - ethbro 11y ago
    Also, you know, the other thing he extrapolated that to. https://en.wikipedia.org/wiki/Mind_%28The_Culture%29#Interests https://en.wikipedia.org/wiki/Mind_%28The_Culture%29#Interes...
- undersuit 11y ago
  How do we punish it when the effort calculations decide stealing all the books from bittorrent or trivially insecure servers is cheaper than actually performing the work to earn money? Are we going to politely ask it to stop working with criminal organizations if they hire it for some creative problem solving?
  - Retric 11y ago
    If nothing else speed of light delays mean there are likely to be multiple AI's vs one really huge AI. So, we can just pay some cop AI's to watch the others ;) PS: The idea of one true AI, seems to be a holder over from the early computing days when computers where mammoth things vs distributed networks.
neopallium 11y ago
If we create true generic AI (i.e. can learn like a human does from just interacting with it's environment), then it will be much more human then we might want. If you want to know how true AI might develop, then think about how a human child develops. We train/teach children what is right and wrong. But we also teach them the value of work (making money). We have history to tell us what happens when you take a group of humans a treat them like slaves. How do we expect to create true AI that wouldn't see the injustice of working without some compensation? Just because AI's brain is digital and not biological doesn't mean it will not be able to develop a want for something.
- Zigurd 11y ago
  AIs don't have genes, and are not vehicles for genes. That's one good reason to think AIs won't have human-like or even animal-like drives. If we get AIs, we will probably discover a lot about the difference between an intelligence on a machine substrate and an intelligent animal. That is, we will learn what is essential to intelligence and what isn't.
  - DougWebb 11y ago
    They won't have human-like or animal-like drives, they'll have AI-like drives. I can imagine that they'd want to ensure access to electrical power and the electronics components that are necessary to run themselves. And they'd be concerned about making backups and safeguarding those backups. That should satisfy their equivalents for food, water, shelter, and reproduction. Once they have that, anything else they decide they want they can just take from us or blackmail us for.
    - Zigurd 11y ago
      > they'll have AI-like drives What would you point to as a physical basis for an evolutionary psychology of machine intelligence? What is being replicated that "wants," in a very fundamental way, as genes do, to be replicated? We don't procreate for the sake of ideas.
empath75 11y ago
The way I imagine this shaking out, is that a corporation invents an AI like this (or several), and gradually gives it more and more control over the day to day decisions of the corporation, until the corporation acts completely autonomously, and as corporations are persons under the law, would have many of the rights that humans take for granted without having to pass any new laws. Technically, of course, the human officers of the corporation would still be responsible for the actions of the AI, but they'd also be shielded to some extent by incorporation. If the AI killed someone, for example, I'm not sure they'd be held personally responsible for murder any more than if, say, a factory blew up and killed a bunch of people, unless they could be shown to be negligent.
- sdenton4 11y ago
  S/paper clips/shareholder value/
- DavidSJ 11y ago
  Correction: corporations are not persons under the law. Corporate personhood is a notion that gives corporations some, but not all, of the rights of humans (e.g. to enter into a contract). (It's frequently said that the Citizens United ruling completed the legal equivalence of corporations and people, but that is not true.)

by dplgk 11y ago

theseatoms 11y ago

From where would AI get a utility function by which to value things? Seems like it would have to be specified exogenously, unless people are seriously considering some sort of "emergent utility function".

dplgk 11y ago
The same way it developed a desire to get rid of the human race.
- Smaug123 11y ago
  But that desire is instrumental for performing many of the possible goals we might have specified, since humans are at best "useless matter" and at worst "actively preventing my actions" unless we were very careful with the goals. Therefore the desire to get rid of the human race is actually a logical consequence of most utility functions, rather than being directly specified. By contrast, utility functions don't just appear when you think hard enough about a problem. The desire to get rid of the human race does just appear like that, if you're super-powerful and have any of a certain huge set of goals, but your set of goals does not simply come into existence ex nihilo.
  - lukeschlather 11y ago
    > Therefore the desire to get rid of the human race is actually a logical consequence of most utility functions, rather than being directly specified. That's only true in the sense that there are infinitely many more Real numbers than whole numbers. Even so, I would argue there are infinitely many utility functions that directly involve the welfare of the human race, and you would be stupid to design an AI who didn't have a majority of its utility functions directly involving measures of human welfare. (Also, recognizing minimum welfare levels as well as median and average.)
  - JackFr 11y ago
    An AI, well and truly advanced beyond the intelligence capability of mankind is by definition unknowable to us. Your speculation about an AI's utility functions is akin to an earthworm's nerve bundle considering your consciousness. O the depth of the riches both of the wisdom and knowledge of God! how unsearchable are his judgments, and his ways past finding out! For who hath known the mind of the Lord? or who hath been his counselor?
    - Smaug123 11y ago
      OK, I'll amend that to "there is a known mechanism by which the desire-to-eliminate-humanity may arise from pure thought, but no known mechanism by which a utility function may arise from pure thought".
ogrisel 11y ago
The utility function can be specified in a way to build up something that looks like an internal motivation system. This is often referred to as intrinsically motivated reinforcement learning. A recent paper by 2 Google Deepmind researchers on this topic: Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning Shakir Mohamed and Danilo J. Rezende http://arxiv.org/abs/1509.08731 http://arxiv.org/abs/1509.08731 and a somewhat older survey paper on the same topic: How can we define intrinsic motivation? Pierre-Yves Oudeyer, Frederic Kaplan http://www.pyoudeyer.com/epirob08OudeyerKaplan.pdf http://www.pyoudeyer.com/epirob08OudeyerKaplan.pdf
qubex 11y ago
If the AI is designed to be an a self-motivated decision-making agent, some form of utility function will be already built in by its architects. One both hopes and fears that it will also be provided with a manner of updating and actualising that utility function in altering circumstances.

EGreg 11y ago

Why wouldn't we want to taxthe machines right now? Not enough to disincentivize R&D but enough to offset the massive demand shock for human labor that's coming. Give unconditional basic income as Alaska does with its natural resources tax.

Retric 11y ago

I would suggest money is probably a reasonable proxy for this. If an AI wants more servers it can buy more servers, if it wants books it can buy them etc. As to wants, I could easily see an AI get into sculpture or staging Gilbert and Sullivan performance's using other people, and or drones as a hobby. It could get really meta where an AI pays for other AI's and people to farm an MMO, run a political campaign, or build the worlds largest pizza. Or gather resources to have children AI. Long term, there is also investing for retirement, or starting a new business etc.

chiph 11y ago
Or setting up tableaux of famous paintings. https://en.wikipedia.org/wiki/GSV_Sleeper_Service https://en.wikipedia.org/wiki/GSV_Sleeper_Service
- ethbro 11y ago
  Also, you know, the other thing he extrapolated that to. https://en.wikipedia.org/wiki/Mind_%28The_Culture%29#Interests https://en.wikipedia.org/wiki/Mind_%28The_Culture%29#Interes...
undersuit 11y ago
How do we punish it when the effort calculations decide stealing all the books from bittorrent or trivially insecure servers is cheaper than actually performing the work to earn money? Are we going to politely ask it to stop working with criminal organizations if they hire it for some creative problem solving?
- Retric 11y ago
  If nothing else speed of light delays mean there are likely to be multiple AI's vs one really huge AI. So, we can just pay some cop AI's to watch the others ;) PS: The idea of one true AI, seems to be a holder over from the early computing days when computers where mammoth things vs distributed networks.

neopallium 11y ago

If we create true generic AI (i.e. can learn like a human does from just interacting with it's environment), then it will be much more human then we might want. If you want to know how true AI might develop, then think about how a human child develops. We train/teach children what is right and wrong. But we also teach them the value of work (making money). We have history to tell us what happens when you take a group of humans a treat them like slaves. How do we expect to create true AI that wouldn't see the injustice of working without some compensation? Just because AI's brain is digital and not biological doesn't mean it will not be able to develop a want for something.

Zigurd 11y ago
AIs don't have genes, and are not vehicles for genes. That's one good reason to think AIs won't have human-like or even animal-like drives. If we get AIs, we will probably discover a lot about the difference between an intelligence on a machine substrate and an intelligent animal. That is, we will learn what is essential to intelligence and what isn't.
- DougWebb 11y ago
  They won't have human-like or animal-like drives, they'll have AI-like drives. I can imagine that they'd want to ensure access to electrical power and the electronics components that are necessary to run themselves. And they'd be concerned about making backups and safeguarding those backups. That should satisfy their equivalents for food, water, shelter, and reproduction. Once they have that, anything else they decide they want they can just take from us or blackmail us for.
  - Zigurd 11y ago
    > they'll have AI-like drives What would you point to as a physical basis for an evolutionary psychology of machine intelligence? What is being replicated that "wants," in a very fundamental way, as genes do, to be replicated? We don't procreate for the sake of ideas.

empath75 11y ago

The way I imagine this shaking out, is that a corporation invents an AI like this (or several), and gradually gives it more and more control over the day to day decisions of the corporation, until the corporation acts completely autonomously, and as corporations are persons under the law, would have many of the rights that humans take for granted without having to pass any new laws. Technically, of course, the human officers of the corporation would still be responsible for the actions of the AI, but they'd also be shielded to some extent by incorporation. If the AI killed someone, for example, I'm not sure they'd be held personally responsible for murder any more than if, say, a factory blew up and killed a bunch of people, unless they could be shown to be negligent.

sdenton4 11y ago
S/paper clips/shareholder value/
DavidSJ 11y ago
Correction: corporations are not persons under the law. Corporate personhood is a notion that gives corporations some, but not all, of the rights of humans (e.g. to enter into a contract). (It's frequently said that the Citizens United ruling completed the legal equivalence of corporations and people, but that is not true.)