5 ms·

This problem isn’t going to go away, largely because there’s no such thing as a single set of ethical principles that can be rationally justified in a way that

This problem isn’t going to go away, largely because there’s no such thing as a single set of ethical principles that can be rationally justified in a way that every rational being will agree to. Depending upon your priorities, your ethical views will inevitably be incompatible with those of some other people in a manner no amount of reasoning will resolve. Believers in a strong central state will find little common ground with libertarians; advocates of radical redistribution will never agree with defenders of private property; relativists won’t suddenly persuade religious fundamentalists that they’re being silly. Who, then, gets to say what an optimal balance between privacy and security looks like — or what’s meant by a socially beneficial purpose? And if we can’t agree on this among ourselves, how can we teach a machine to embody “human” values?

bonoboTP 7y ago
Well, nations seem to be able to agree upon constitutions and laws. It works. We live in unprecedented peace in the western world. So somehow, even with all those different people with their different views, it is, miraculously, possible. Why would that fundamentally change if we involve tech in that?
- vertex-four 7y ago
  Even the police can't agree to uphold their own laws in this "peaceful western world" - every time I go outside right now, I return home to news of the people I'm living with having been beaten while in police custody, having been arrested for reasons like "your passport proves nothing about your identity". Breaking my friends' arms, kicking them repeatedly in the back of their head and their spine. I'm scared that next time I leave, that will be me. Technology first and foremost helps those with the resources to use it. That tends to be the people with power in the first place.
  - SpicyLemonZest 7y ago
    The modern world is big enough that you can find news of any bad thing if you’re looking for it. The relevant questions are how common it is and how common similar problems used to be.
    - vertex-four 7y ago
      I'm not looking for news, thank you very much - this is happening, right now, to me personally.
      - SpicyLemonZest 7y ago
        You have that many friends that the police have beaten one of them up every single time you return home? I don’t mean to be rude - I’m sure there’s a kernel of truth here and I’m sorry to hear you’re in such a bad situation - but I just can’t believe this is true as stated.
        vertex-four 7y ago
        I'm living in a shared house, with people I've grown quite close to. The local fascist party has recently made public their dislike of us and a call to action to "remove us"; the lead police officer on the "case" of our eviction (which has not yet been processed; we are legal residents of our house under the law here) is one of their known followers. There are multiple officers now dedicated to driving by our flat on a regular basis in order to check IDs of people attempting to visit or leave; we're watched on CCTV when we leave the neighbourhood. We regularly have our IDs checked without reason while walking to the shops (and they then arrest those who refuse to produce ID for an illegal request, then release them without charge a few hours later), or we're arrested and jailed for days (illegally, without a court order) for things like "walking a dog without a leash"; an offence which comes with an on-the-spot fine, not jail. In our list of things that have happened in a jail cell without a camera present so far, we have someone with broken ribs, and someone who was kicked repeatedly in the back of the head against a concrete floor after a failed attempt to break their arm, as well as more minor attacks. We have committed no crimes; we are legal, peaceful residents who have done nothing except for make a property management company go through a legal process to evict us, which we believe we will win. Nobody residing in our house, nor our visitors, have hurt anybody - we have taken part in no activity which resulted in anybody being hurt, and there's no reason for this violence against us other than a property management company which is upset at likely losing a multi-million euro redevelopment contract by failing to evict us. I'm honestly really glad that this isn't something within your worldview; I wouldn't wish this on anybody. But it is happening.
  - bonoboTP 7y ago
    The US is quite an outlier in police brutality though. Either way, the quote was about different political ideologies and who will get it their way, i.e. big government, small government etc. It seems that it can be figured out in a democratic process and people aren't rioting. From a historical perspective there is stability and peace even in the US. No civil wars or large scale revolutions or chaos.
    - vertex-four 7y ago
      I'm not in the US, I'm in a rich European country. The point is that people disagreeing with ethics doesn't mean the constitution or the law stops existing, it just means they ignore it. The existence of a constitution or a law usually just means that some people who our schools teach us are important decided to write something.
- LunaSea 7y ago
  > Well, nations seem to be able to agree upon constitutions and laws. I would disagree and say that laws and constitutional amendments are an aggregation of decisions passed by various majorities and coalitions of minorities during several decades. If you ask any current party, even when a majority, if they agree with all the laws that currently exist they would answer a resounding "no". So that means that there doesn't exist a legal common ground, just a ground we currently stand on that various parties want to change but don't have the votes, time or alternatives for.
- babesh 7y ago
  How about the current environmental l collapse or count the number of animals killed each year for food? The utilitarianism approach still depends on how you define utility and who you define it for. People, animals, the planet, consciousness are different things you can define it for. Another issue is the scale of time you are measuring things. It's not clear whether our approach is sustainable.
- rtikulit 7y ago
  The risk is that the naive and opportunistic application of decision-making, “judgement” and “ethics” embodied in AI systems will lead to the well-intentioned accidental regulation of important domains of human life, stifling those with relentless automatic gatekeepers and negative feedback loops which are pervasive, inscrutable, unaccountable, inimical to the fertile chaos of human negotiation and hostile to human social and political evolution. They will take human activity that is currently diverse and distributed and will force it through narrow channels controlled by self-appointed gatekeepers. In an analogy to censorship, consider a world where “prior restraint” and “chilling effects” apply to every aspect in our lives, where deviations from the ideal are reliably detected and rejected, where we would not be able to earn credentials, get a job, find a partner, buy a home, evade surveillance, obtain necessary health care, or safely express a political opinion unless we either conform to or cheat the prevailing machine inference gatekeepers. What an insane additional cognitive burden this will be for everyone, everywhere, all the time. This kind of AI is a mechanized, high-speed race to the bottom. And it will operate to discover undreamed-of bottoms to race to. In the name of a few extra percent of efficiency we will doom our children to a rigid and lifeless dance with pattern matching algorithms.
TheOtherHobbes 7y ago
The first flaw with this argument is that consensus ethics are deliberately engineered anyway, usually by people with the money to finance PR and propaganda operations. So the idea that (e.g.) libertarians have a free choice about what they believe is - ironically - not even close to being true. The second flaw with this argument is that all of these positions could, in principle, be tested objectively. The best possible outcome for true general AI - also the least likely in practice, but not impossible in principle - is that it could provide definitive answers to matter-of-opinion ethical questions. And it could do this not just in terms of incomprehensible statistics, but as comprehensible narratives derived from explicit arguments. In fact a hyper-AGI would know which arguments to use to be most persuasive and convincing - which isn't particularly reassuring, but it would be an inevitable outcome of hyper-AGI.
- logicchains 7y ago
  >The second flaw with this argument is that all of these positions could, in principle, be tested objectively. What is the objective measure by which you would test them; against what standard? And by what standard is that standard the best standard to use? By what measure in turn is that standard the best measure to use? It's impossible to find a truly objective standard due to https://en.m.wikipedia.org/wiki/Regress_argument; https://en.m.wikipedia.org/wiki/Regress_argument; a standard can only exist if defined circularly or upon some arbitrarily chosen axioms. A superintelligent AI can't somehow wave away the laws of logic, any more than it could prove that 1+1=3.

bonoboTP 7y ago

Well, nations seem to be able to agree upon constitutions and laws. It works. We live in unprecedented peace in the western world. So somehow, even with all those different people with their different views, it is, miraculously, possible. Why would that fundamentally change if we involve tech in that?

vertex-four 7y ago
Even the police can't agree to uphold their own laws in this "peaceful western world" - every time I go outside right now, I return home to news of the people I'm living with having been beaten while in police custody, having been arrested for reasons like "your passport proves nothing about your identity". Breaking my friends' arms, kicking them repeatedly in the back of their head and their spine. I'm scared that next time I leave, that will be me. Technology first and foremost helps those with the resources to use it. That tends to be the people with power in the first place.
- SpicyLemonZest 7y ago
  The modern world is big enough that you can find news of any bad thing if you’re looking for it. The relevant questions are how common it is and how common similar problems used to be.
  - vertex-four 7y ago
    I'm not looking for news, thank you very much - this is happening, right now, to me personally.
    - SpicyLemonZest 7y ago
      You have that many friends that the police have beaten one of them up every single time you return home? I don’t mean to be rude - I’m sure there’s a kernel of truth here and I’m sorry to hear you’re in such a bad situation - but I just can’t believe this is true as stated.
      - vertex-four 7y ago
        I'm living in a shared house, with people I've grown quite close to. The local fascist party has recently made public their dislike of us and a call to action to "remove us"; the lead police officer on the "case" of our eviction (which has not yet been processed; we are legal residents of our house under the law here) is one of their known followers. There are multiple officers now dedicated to driving by our flat on a regular basis in order to check IDs of people attempting to visit or leave; we're watched on CCTV when we leave the neighbourhood. We regularly have our IDs checked without reason while walking to the shops (and they then arrest those who refuse to produce ID for an illegal request, then release them without charge a few hours later), or we're arrested and jailed for days (illegally, without a court order) for things like "walking a dog without a leash"; an offence which comes with an on-the-spot fine, not jail. In our list of things that have happened in a jail cell without a camera present so far, we have someone with broken ribs, and someone who was kicked repeatedly in the back of the head against a concrete floor after a failed attempt to break their arm, as well as more minor attacks. We have committed no crimes; we are legal, peaceful residents who have done nothing except for make a property management company go through a legal process to evict us, which we believe we will win. Nobody residing in our house, nor our visitors, have hurt anybody - we have taken part in no activity which resulted in anybody being hurt, and there's no reason for this violence against us other than a property management company which is upset at likely losing a multi-million euro redevelopment contract by failing to evict us. I'm honestly really glad that this isn't something within your worldview; I wouldn't wish this on anybody. But it is happening.
- bonoboTP 7y ago
  The US is quite an outlier in police brutality though. Either way, the quote was about different political ideologies and who will get it their way, i.e. big government, small government etc. It seems that it can be figured out in a democratic process and people aren't rioting. From a historical perspective there is stability and peace even in the US. No civil wars or large scale revolutions or chaos.
  - vertex-four 7y ago
    I'm not in the US, I'm in a rich European country. The point is that people disagreeing with ethics doesn't mean the constitution or the law stops existing, it just means they ignore it. The existence of a constitution or a law usually just means that some people who our schools teach us are important decided to write something.
LunaSea 7y ago
> Well, nations seem to be able to agree upon constitutions and laws. I would disagree and say that laws and constitutional amendments are an aggregation of decisions passed by various majorities and coalitions of minorities during several decades. If you ask any current party, even when a majority, if they agree with all the laws that currently exist they would answer a resounding "no". So that means that there doesn't exist a legal common ground, just a ground we currently stand on that various parties want to change but don't have the votes, time or alternatives for.
babesh 7y ago
How about the current environmental l collapse or count the number of animals killed each year for food? The utilitarianism approach still depends on how you define utility and who you define it for. People, animals, the planet, consciousness are different things you can define it for. Another issue is the scale of time you are measuring things. It's not clear whether our approach is sustainable.
rtikulit 7y ago
The risk is that the naive and opportunistic application of decision-making, “judgement” and “ethics” embodied in AI systems will lead to the well-intentioned accidental regulation of important domains of human life, stifling those with relentless automatic gatekeepers and negative feedback loops which are pervasive, inscrutable, unaccountable, inimical to the fertile chaos of human negotiation and hostile to human social and political evolution. They will take human activity that is currently diverse and distributed and will force it through narrow channels controlled by self-appointed gatekeepers. In an analogy to censorship, consider a world where “prior restraint” and “chilling effects” apply to every aspect in our lives, where deviations from the ideal are reliably detected and rejected, where we would not be able to earn credentials, get a job, find a partner, buy a home, evade surveillance, obtain necessary health care, or safely express a political opinion unless we either conform to or cheat the prevailing machine inference gatekeepers. What an insane additional cognitive burden this will be for everyone, everywhere, all the time. This kind of AI is a mechanized, high-speed race to the bottom. And it will operate to discover undreamed-of bottoms to race to. In the name of a few extra percent of efficiency we will doom our children to a rigid and lifeless dance with pattern matching algorithms.

TheOtherHobbes 7y ago

The first flaw with this argument is that consensus ethics are deliberately engineered anyway, usually by people with the money to finance PR and propaganda operations. So the idea that (e.g.) libertarians have a free choice about what they believe is - ironically - not even close to being true. The second flaw with this argument is that all of these positions could, in principle, be tested objectively. The best possible outcome for true general AI - also the least likely in practice, but not impossible in principle - is that it could provide definitive answers to matter-of-opinion ethical questions. And it could do this not just in terms of incomprehensible statistics, but as comprehensible narratives derived from explicit arguments. In fact a hyper-AGI would know which arguments to use to be most persuasive and convincing - which isn't particularly reassuring, but it would be an inevitable outcome of hyper-AGI.

logicchains 7y ago
>The second flaw with this argument is that all of these positions could, in principle, be tested objectively. What is the objective measure by which you would test them; against what standard? And by what standard is that standard the best standard to use? By what measure in turn is that standard the best measure to use? It's impossible to find a truly objective standard due to https://en.m.wikipedia.org/wiki/Regress_argument; https://en.m.wikipedia.org/wiki/Regress_argument; a standard can only exist if defined circularly or upon some arbitrarily chosen axioms. A superintelligent AI can't somehow wave away the laws of logic, any more than it could prove that 1+1=3.