5 ms·

It's hard to design any system (or research program for that matter) if the goal isn't well defined. If the research is aimed at ensuring that AI is aligned wi

by natch 12y ago

It's hard to design any system (or research program for that matter) if the goal isn't well defined. If the research is aimed at ensuring that AI is aligned with human interests, another priority should be researching what we deem those interests to be. The Law and Ethics research section of the research priorities document is centered on mechanisms, with a couple of nods to policy, but with the seeming assumption that that question of what is in the human interest has already been answered. But the answer certainly isn't obvious. Different sets of humans have wildly diverging and fundamental disagreements over what is right or wrong for humans and our future. I doubt we're going to find a consensus in the time we have. It will probably make more sense to just have smart people define a set of good principles. This is hard enough that it should be considered a research task, and I would think it would be at the top of the priority list. Otherwise, how do we know that the research we are doing on other problems even makes sense? If you just stub out the "human wishes" component of all this as one simple "human approves" button, then you haven't solved anything. We need a robust definition of what "aligned with human interests" means.

jimrandomh 12y ago
The current most-widely-accepted answer to this is "indirect normativity". This is basically saying that human values are complex enough that we should instruct an AI to study humans to figure out what they are, rather than try to program values in directly and risk getting them wrong. I'd be a lot more comfortable if metaethics and related philosophy advanced to the point where we didn't have to rely on indirect normativity. But I don't think this is something you can just throw money and people at, because this is a field which is sufficiently difficult and hard to judge, that work which isn't first-rate tends to detract from progress by adding noise rather than help.
- yanazendo 12y ago
  Indirect normativity is a losing strategy because you don't know what the resultant ethical system will be. https://ordinaryideas.wordpress.com/2012/04/21/indirect-normativity-write-up/ https://ordinaryideas.wordpress.com/2012/04/21/indirect-norm... Humanity has already solved ethics, so there's no reason for us to give up and ask AI to define ethics for us.
  - Houshalter 12y ago
    >Indirect normativity is a losing strategy because you don't know what the resultant ethical system will be. That's the point... >Humanity has already solved ethics Where on earth do you get this idea?
    - yanazendo 12y ago
      Awareness of ethical systems. What makes you think ethics isn't solved? Here's a relevant essay http://medium.com/@yanazendo/how-to-be-lazy-and-successful-at-the-same-time-1c283c6880b3 http://medium.com/@yanazendo/how-to-be-lazy-and-successful-a...
      - Houshalter 12y ago
        That's not even close to a formalization of human morality. Such a thing probably isn't even possible. See this: http://lesswrong.com/lw/ld/the_hidden_complexity_of_wishes/ http://lesswrong.com/lw/ld/the_hidden_complexity_of_wishes/
- vegedor 12y ago
  Isn't the addition of a little noise one way to train neural networks?
- TheOtherHobbes 12y ago
  >This is basically saying that human values are complex enough that we should instruct an AI to study humans to figure out what they are, rather than try to program values in directly and risk getting them wrong. Oh god no. A thousand times no. Humans are masters at hiding sociopathy behind cultural and political obfuscation. A machine that looks at human leaders and generalises human values from them is absolutely guaranteed to be a monster. >I'd be a lot more comfortable if metaethics and related philosophy advanced to the point where we didn't have to rely on indirect normativity. We already have any number of perfectly sustainable ethical systems. What we don't have is the ability to follow them in practice - partly because all political systems tend to reward power and influence with more power and influence, and partly because even when there are token checks and balances (e.g. in the form of representative democracy) it's so easy to game them with appeals to irrationality and emotion. This rarely works out well for most of the population. But I don't much like the content of the proposal. I think you can have extremely useful intelligence amplification without any need for agency or personality. Much better modelling along the lines of 'If you do this, this will be the outcome' would be a game changer on its own.
yanazendo 12y ago
Use ethic monoids to combine seemingly disparate ethical systems. https://medium.com/@yanazendo/ethic-monoids-913c3046079c https://medium.com/@yanazendo/ethic-monoids-913c3046079c
- Lambdanaut 12y ago
  Intuitively I get it, but it doesn't seem all that useful. I'd love to see an example of it in action.
  - yanazendo 12y ago
    The ACM and the IEEE want to do business with each other. Business partnerships cannot exist unless all parties agree to follow the same ethical system. Unfortunately, ACM and IEEE have different Codes of Ethics. To do business, they must define an ethical system that they both agree upon. The simplest monoid combines ethical systems by only including codes that appear in both Codes. So, the ACM+IEEE Code of Ethics can simply be defined as the intersection of the ACM+IEEE Code of Ethics. Then ACM knows what to expect from IEEE, and vice versa. If we want AI to be ethical, we must be able to express ethics computationally, and we must be able to formally verify the correctness ethical systems. So, we should use monoids.
- dborg 12y ago
  "Monoids", you keep using that word. I don't think it means what you think it means. You've basically said "use monoids" without a word of elaboration how to interpret ethics, or really anything, in this formalism.
deleted 12y ago
[deleted]

by natch 12y ago

jimrandomh 12y ago

The current most-widely-accepted answer to this is "indirect normativity". This is basically saying that human values are complex enough that we should instruct an AI to study humans to figure out what they are, rather than try to program values in directly and risk getting them wrong. I'd be a lot more comfortable if metaethics and related philosophy advanced to the point where we didn't have to rely on indirect normativity. But I don't think this is something you can just throw money and people at, because this is a field which is sufficiently difficult and hard to judge, that work which isn't first-rate tends to detract from progress by adding noise rather than help.

yanazendo 12y ago
Indirect normativity is a losing strategy because you don't know what the resultant ethical system will be. https://ordinaryideas.wordpress.com/2012/04/21/indirect-normativity-write-up/ https://ordinaryideas.wordpress.com/2012/04/21/indirect-norm... Humanity has already solved ethics, so there's no reason for us to give up and ask AI to define ethics for us.
- Houshalter 12y ago
  >Indirect normativity is a losing strategy because you don't know what the resultant ethical system will be. That's the point... >Humanity has already solved ethics Where on earth do you get this idea?
  - yanazendo 12y ago
    Awareness of ethical systems. What makes you think ethics isn't solved? Here's a relevant essay http://medium.com/@yanazendo/how-to-be-lazy-and-successful-at-the-same-time-1c283c6880b3 http://medium.com/@yanazendo/how-to-be-lazy-and-successful-a...
    - Houshalter 12y ago
      That's not even close to a formalization of human morality. Such a thing probably isn't even possible. See this: http://lesswrong.com/lw/ld/the_hidden_complexity_of_wishes/ http://lesswrong.com/lw/ld/the_hidden_complexity_of_wishes/
vegedor 12y ago
Isn't the addition of a little noise one way to train neural networks?
TheOtherHobbes 12y ago
>This is basically saying that human values are complex enough that we should instruct an AI to study humans to figure out what they are, rather than try to program values in directly and risk getting them wrong. Oh god no. A thousand times no. Humans are masters at hiding sociopathy behind cultural and political obfuscation. A machine that looks at human leaders and generalises human values from them is absolutely guaranteed to be a monster. >I'd be a lot more comfortable if metaethics and related philosophy advanced to the point where we didn't have to rely on indirect normativity. We already have any number of perfectly sustainable ethical systems. What we don't have is the ability to follow them in practice - partly because all political systems tend to reward power and influence with more power and influence, and partly because even when there are token checks and balances (e.g. in the form of representative democracy) it's so easy to game them with appeals to irrationality and emotion. This rarely works out well for most of the population. But I don't much like the content of the proposal. I think you can have extremely useful intelligence amplification without any need for agency or personality. Much better modelling along the lines of 'If you do this, this will be the outcome' would be a game changer on its own.

yanazendo 12y ago

Use ethic monoids to combine seemingly disparate ethical systems. https://medium.com/@yanazendo/ethic-monoids-913c3046079c https://medium.com/@yanazendo/ethic-monoids-913c3046079c

Lambdanaut 12y ago
Intuitively I get it, but it doesn't seem all that useful. I'd love to see an example of it in action.
- yanazendo 12y ago
  The ACM and the IEEE want to do business with each other. Business partnerships cannot exist unless all parties agree to follow the same ethical system. Unfortunately, ACM and IEEE have different Codes of Ethics. To do business, they must define an ethical system that they both agree upon. The simplest monoid combines ethical systems by only including codes that appear in both Codes. So, the ACM+IEEE Code of Ethics can simply be defined as the intersection of the ACM+IEEE Code of Ethics. Then ACM knows what to expect from IEEE, and vice versa. If we want AI to be ethical, we must be able to express ethics computationally, and we must be able to formally verify the correctness ethical systems. So, we should use monoids.
dborg 12y ago
"Monoids", you keep using that word. I don't think it means what you think it means. You've basically said "use monoids" without a word of elaboration how to interpret ethics, or really anything, in this formalism.

deleted 12y ago

[deleted]