4 ms·

I’m starting a one year research project on formalising ethics at Cambridge this month. The one big issue that stands out, to me, is how we source ethics. Until

by exdsq 6y ago

I’m starting a one year research project on formalising ethics at Cambridge this month. The one big issue that stands out, to me, is how we source ethics. Until we solve that, I don’t really see how we can practically apply any of these things.

throwaway_pdp09 6y ago
What does it mean to 'source ethics'. I don't understand the phrase.
- exdsq 6y ago
  Say we have a system of implementing some ethical framework in an AI application, how do we actually decide on what those ethics are. The problem gets harder the more global the application is - ethical values held in one place won’t be the same as another.
  - throwaway_pdp09 6y ago
    There is a common core of ethics , do unto others as you would be done by, and maybe a few more. These aren't universally accepted but they'll do. You pick your axioms and go from there. Like mathematics. What's the problem?
    - exdsq 6y ago
      Taking your example, that requires one to have already codified exactly what you yourself want and for that list of wants to be the same for everyone.
      - throwaway_pdp09 6y ago
        That is quite literally the opposite of what I said.
  - TeMPOraL 6y ago
    That's probably a question for neuroscientists. All ethics ultimately stem from what people feel is or isn't right. All ethical considerations are processed in our brains. Since we all share a common brain architecture[0], there's some core of ethics that's encoded in our wetware. There are some basic ethical concepts on which human societies tend to universally agree on. The question is, how much of that comes from our wetware, and how much of it is a function of environmental conditions. It may be that we're only born with some most basic building blocks, and any universality we observe comes from humans living in roughly the same social structures, and dealing with roughly the same problems. -- [0] - Changing at evolutionary timescales, but essentially fixed wrt. lifetime of a human individual.
    - claudiawerner 6y ago
      >All ethics ultimately stem from what people feel is or isn't right. All ethical considerations are processed in our brains. This is debatable. Some metaethicists would argue that the ethical realm is much like the mathematical one, and less of a matter of the functioning of the human brain. Further, it seems to me there is a difference between saying "I don't like X" and saying "X is morally wrong" - if there is a difference between those two statements (as a moral cognitivist would claim), then deduction of morals from "what feels right" is suspect. The argument of cognitivism vs non-cognitivism is interesting[0]. >There are some basic ethical concepts on which human societies tend to universally agree on. The number and reach of those principles is very debatable. Arguments from moral disagreement cast doubt on the extent to which human societies share ethical standards. >humans living in roughly the same social structures I don't think that's true at all; post-globalization, there has been a convergence in what social structures look like around the world, but it's very clear that humans have had and in many cases still do have a variety of social structures, and with that, a variety of obligations. The moral obligation to do X in Japan, for instance, may be different by virtue of a different social structure. Using neuroscience to "deduce" ethics seems like the ultimate briding of the is-ought gap and I'm not aware of any professional philosopher or theory who has successfully argued that this is possible. The attempt in Sam Harris' The Moral Landscape has been shot down from practically every angle imaginable, and I think for good reason. [0] https://plato.stanford.edu/entries/moral-cognitivism/ https://plato.stanford.edu/entries/moral-cognitivism/
anticensor 6y ago
What about a self-stabilising globally utilitarian (towards maximal societal output and overall comfort in expense of individual pleasure) system, similar to what we have with TCP (which is designed to maximise global packet flow and reliability in expense of local slowdowns)?
- joppy 6y ago
  What does this mean though? By utilitarian you mean to optimise for some utility function: how do you pick it? Does it allow for 99% of people to be fine while 1% live in absolute squalor? Does maximising societal output mean draining natural resources dry for generations to come? How does this treat personal autonomy, which is a way that many people derive meaning in their lives? Why should something as complicated as human ethics be reducible to terms as simple as a networking problem?
  - logicchains 6y ago
    >By utilitarian you mean to optimise for some utility function: how do you pick it? Does it allow for 99% of people to be fine while 1% live in absolute squalor? He could pick a form of utility based on ordinal utility that doesn't allow for interpersonal utility comparisons. In such a theory fewer things could be proven, but it would be easier (although still quite challenging) to prove whether an AI following that theory would decide to kill all humans.
    - anticensor 6y ago
      What I had in my mind was rather a non-linear combinatorial utility function which would allow for global utility comparisons (which would answer not only what the best action is, but also who the most viable actor for a given action given comparable skills is), not an ordinal one. That would enable a framework that gives you maximum societal output and maximum conformity at the same time.
  - anticensor 6y ago
    Behavioral interaction is easily reducible to a networking problem with just a few axioms and terms swapped. Simple rules make complex systems. > draining natural resources dry for generations to come? That would count as negative production in my model as I define output as (global production-global consumption)/log(global production+global consumption), aka diminished excess product, not as simple global production. > How does this treat personal autonomy It explicitly does not, as there is an inherent trade-off between personal pleasure and freedom, versus overall conformity. But you can tune the function by adding overall comfort (total objective satisfaction) as a third parameter, which will unfortunately reduce societal comformity. My model is both non-authoritarian and non-liberalistic. It relies on modern day high speed communication facilities to exchange the input parameters for the utility function instead.
hliyan 6y ago
As of late, I explain ethics to myself as: "A system that governs the behavior of a conscious agent that experiences the qualia of pain, within the same causal domain as other such agents, to arrive at the optimal dynamic equilibrium between minimizing individual pain and average group pain." It's how I would formulate Kant's Categorical Imperative.

by exdsq 6y ago

throwaway_pdp09 6y ago

What does it mean to 'source ethics'. I don't understand the phrase.

exdsq 6y ago
Say we have a system of implementing some ethical framework in an AI application, how do we actually decide on what those ethics are. The problem gets harder the more global the application is - ethical values held in one place won’t be the same as another.
- throwaway_pdp09 6y ago
  There is a common core of ethics , do unto others as you would be done by, and maybe a few more. These aren't universally accepted but they'll do. You pick your axioms and go from there. Like mathematics. What's the problem?
  - exdsq 6y ago
    Taking your example, that requires one to have already codified exactly what you yourself want and for that list of wants to be the same for everyone.
    - throwaway_pdp09 6y ago
      That is quite literally the opposite of what I said.
- TeMPOraL 6y ago
  That's probably a question for neuroscientists. All ethics ultimately stem from what people feel is or isn't right. All ethical considerations are processed in our brains. Since we all share a common brain architecture[0], there's some core of ethics that's encoded in our wetware. There are some basic ethical concepts on which human societies tend to universally agree on. The question is, how much of that comes from our wetware, and how much of it is a function of environmental conditions. It may be that we're only born with some most basic building blocks, and any universality we observe comes from humans living in roughly the same social structures, and dealing with roughly the same problems. -- [0] - Changing at evolutionary timescales, but essentially fixed wrt. lifetime of a human individual.
  - claudiawerner 6y ago
    >All ethics ultimately stem from what people feel is or isn't right. All ethical considerations are processed in our brains. This is debatable. Some metaethicists would argue that the ethical realm is much like the mathematical one, and less of a matter of the functioning of the human brain. Further, it seems to me there is a difference between saying "I don't like X" and saying "X is morally wrong" - if there is a difference between those two statements (as a moral cognitivist would claim), then deduction of morals from "what feels right" is suspect. The argument of cognitivism vs non-cognitivism is interesting[0]. >There are some basic ethical concepts on which human societies tend to universally agree on. The number and reach of those principles is very debatable. Arguments from moral disagreement cast doubt on the extent to which human societies share ethical standards. >humans living in roughly the same social structures I don't think that's true at all; post-globalization, there has been a convergence in what social structures look like around the world, but it's very clear that humans have had and in many cases still do have a variety of social structures, and with that, a variety of obligations. The moral obligation to do X in Japan, for instance, may be different by virtue of a different social structure. Using neuroscience to "deduce" ethics seems like the ultimate briding of the is-ought gap and I'm not aware of any professional philosopher or theory who has successfully argued that this is possible. The attempt in Sam Harris' The Moral Landscape has been shot down from practically every angle imaginable, and I think for good reason. [0] https://plato.stanford.edu/entries/moral-cognitivism/ https://plato.stanford.edu/entries/moral-cognitivism/

anticensor 6y ago

What about a self-stabilising globally utilitarian (towards maximal societal output and overall comfort in expense of individual pleasure) system, similar to what we have with TCP (which is designed to maximise global packet flow and reliability in expense of local slowdowns)?

joppy 6y ago
What does this mean though? By utilitarian you mean to optimise for some utility function: how do you pick it? Does it allow for 99% of people to be fine while 1% live in absolute squalor? Does maximising societal output mean draining natural resources dry for generations to come? How does this treat personal autonomy, which is a way that many people derive meaning in their lives? Why should something as complicated as human ethics be reducible to terms as simple as a networking problem?
- logicchains 6y ago
  >By utilitarian you mean to optimise for some utility function: how do you pick it? Does it allow for 99% of people to be fine while 1% live in absolute squalor? He could pick a form of utility based on ordinal utility that doesn't allow for interpersonal utility comparisons. In such a theory fewer things could be proven, but it would be easier (although still quite challenging) to prove whether an AI following that theory would decide to kill all humans.
  - anticensor 6y ago
    What I had in my mind was rather a non-linear combinatorial utility function which would allow for global utility comparisons (which would answer not only what the best action is, but also who the most viable actor for a given action given comparable skills is), not an ordinal one. That would enable a framework that gives you maximum societal output and maximum conformity at the same time.
- anticensor 6y ago
  Behavioral interaction is easily reducible to a networking problem with just a few axioms and terms swapped. Simple rules make complex systems. > draining natural resources dry for generations to come? That would count as negative production in my model as I define output as (global production-global consumption)/log(global production+global consumption), aka diminished excess product, not as simple global production. > How does this treat personal autonomy It explicitly does not, as there is an inherent trade-off between personal pleasure and freedom, versus overall conformity. But you can tune the function by adding overall comfort (total objective satisfaction) as a third parameter, which will unfortunately reduce societal comformity. My model is both non-authoritarian and non-liberalistic. It relies on modern day high speed communication facilities to exchange the input parameters for the utility function instead.

hliyan 6y ago

As of late, I explain ethics to myself as: "A system that governs the behavior of a conscious agent that experiences the qualia of pain, within the same causal domain as other such agents, to arrive at the optimal dynamic equilibrium between minimizing individual pain and average group pain." It's how I would formulate Kant's Categorical Imperative.