3 ms·

Is my reading of this law correct? It sounds like it outlaws all use of algorithms to evaluate people. The right to explanation is just for the rare exceptions

Is my reading of this law correct? It sounds like it outlaws all use of algorithms to evaluate people. The right to explanation is just for the rare exceptions where a "member state" authorizes it. But the legalese is difficult to parse, and no one else seems to get this impression. I think this is really bad. That's the majority of uses of machine learning. It also has a lot of economic value to predict things like how likely someone is to pay back a loan, or even who is a spammer. Most importantly, most these applications will go back to human judgement. And humans are far worse. Human predictions about candidates are really bad. We are incredibly biased by things like race, gender, and especially attractiveness. Unattractive people get twice the sentences of attractive people. Not to mention random stuff, like people being judged more harshly when the judge is hungry before lunch. They also rarely give explanations for their decisions, if they are even aware of the true reasons for them. Going back to humans is a huge step backwards and will hurt a lot more people than it helps. I think the same regulations that apply to algorithms, should apply to humans. That would show the absurdity of these laws. But humans are algorithms after all. And particularly bad ones (for this purpose anyway.)

germanier 10y ago
> It sounds like it outlaws all use of algorithms to evaluate people. The right to explanation is just for the rare exceptions where a "member state" authorizes it. Who says that this is rare? That's a pretty standard way EU regulations are written to mean "Action A can only be allowed if conditions B are met". If the regulation is approved, member states take a look and change their laws accordingly. They could outlaw algorithmic decision making if they wanted to (but they wouldn't need that regulation to do that). Much more likely is that they impose those conditions on the companies that use such algorithms just like they already impose a lot of conditions (such as "ethnicity is not allowed to be a direct input variable" currently is).
- ddebernardy 10y ago
  > If the regulation is approved, member states take a look and change their laws accordingly. Small nitpick id I may: you seem to be confusing Regulations with Directives, which must get transcribed within a certain timeframe in national law. Regulations apply directly - no transcription needed. Also: either way, EU law has primacy over national law.
  - germanier 10y ago
    Thanks for pointing this out, my post was not entirely clear on that matter. This proposed regulation does in fact contain some directive-like clauses with regards to the issue at hand. In this specific case they require the member states to disallow certain practices in their national law. It's also not uncommon for member states to enact laws dealing with the effects of a (directly applicable) regulation.
radarsat1 10y ago
> Most importantly, most these applications will go back to human judgement. And humans are far worse. I think you have to be careful there. Leaving decisions to machines embeds an assumption that all information related a decision can (and has been) encoded in a machine-interpretable manner. How often are circumstances and context taken into account during sentencing hearings? How often do judges actually try to consider things like whether the offender feels remorse? Sure, these kinds of things can be encoded into numbers, but should they be? What does it mean to encode remorse into e.g. a 7-valued point system? How much does that depend on the person entering the number, who must judge this, or should it be detected automatically using sensors? Should we be modifying sentences according to an offender's level of galvanic skin response? This is just an example of course, but my point in general is that you make a huge assumption that factors related to decision making can be encoded properly and in an unbiased manner. I'd even say that many of the factors that would need to be entered into a computer would have to be judged by a human, and therefore leaving the actual decision-making to a computer is a bit of a symbolic gesture in that sense. I would guess that, given the same data, a computer could/would be programmed to make the same judgement as a human. It is the data extraction process itself that incurs bias, not the algorithm. Furthermore, if you really leave things up to only externally and easily measurable data, you might not get the results you want or expect. A computer might see that black people are much more likely to commit crime, because hey, more black people are in jail, and therefore judge that the black person was more likely to be the criminal, given a choice. But is this the correct way to think? Indeed, a human would have to think really hard to correct this kind of bias. In other words, saying that an algorithm is less biased than a human judge is somewhat of a non-statement, since the judge actually follows fairly strict rules regarding what the facts imply -- it is more how he judges what those facts are that changes the sentencing. Meanwhile, on a computer, the programmer has a lot more power than you might assume in controlling how facts are weighted in order to come up with what is seen as a "fair" decision. And that fairness is a human judgement, fundamentally speaking, since it is about the relationship between a community and an individual. In short, I don't think you can replace humans with machines when it comes to making decisions about how to deal with humans. The machine can only provide inferences from facts, but the values that those facts imply are a fundamentally human concept. It is a social thing, not an algorithm thing. Another way to put it: if laws could be perfectly encoded by logic, we would have no need for lawyers and judges. These roles exist because as a society, we are constantly re-evaluating the logic of our society. The rules are not something that can be written statically. They must change as new ideas, problems, and circumstances change.

germanier 10y ago

> It sounds like it outlaws all use of algorithms to evaluate people. The right to explanation is just for the rare exceptions where a "member state" authorizes it. Who says that this is rare? That's a pretty standard way EU regulations are written to mean "Action A can only be allowed if conditions B are met". If the regulation is approved, member states take a look and change their laws accordingly. They could outlaw algorithmic decision making if they wanted to (but they wouldn't need that regulation to do that). Much more likely is that they impose those conditions on the companies that use such algorithms just like they already impose a lot of conditions (such as "ethnicity is not allowed to be a direct input variable" currently is).

ddebernardy 10y ago
> If the regulation is approved, member states take a look and change their laws accordingly. Small nitpick id I may: you seem to be confusing Regulations with Directives, which must get transcribed within a certain timeframe in national law. Regulations apply directly - no transcription needed. Also: either way, EU law has primacy over national law.
- germanier 10y ago
  Thanks for pointing this out, my post was not entirely clear on that matter. This proposed regulation does in fact contain some directive-like clauses with regards to the issue at hand. In this specific case they require the member states to disallow certain practices in their national law. It's also not uncommon for member states to enact laws dealing with the effects of a (directly applicable) regulation.

radarsat1 10y ago

> Most importantly, most these applications will go back to human judgement. And humans are far worse. I think you have to be careful there. Leaving decisions to machines embeds an assumption that all information related a decision can (and has been) encoded in a machine-interpretable manner. How often are circumstances and context taken into account during sentencing hearings? How often do judges actually try to consider things like whether the offender feels remorse? Sure, these kinds of things can be encoded into numbers, but should they be? What does it mean to encode remorse into e.g. a 7-valued point system? How much does that depend on the person entering the number, who must judge this, or should it be detected automatically using sensors? Should we be modifying sentences according to an offender's level of galvanic skin response? This is just an example of course, but my point in general is that you make a huge assumption that factors related to decision making can be encoded properly and in an unbiased manner. I'd even say that many of the factors that would need to be entered into a computer would have to be judged by a human, and therefore leaving the actual decision-making to a computer is a bit of a symbolic gesture in that sense. I would guess that, given the same data, a computer could/would be programmed to make the same judgement as a human. It is the data extraction process itself that incurs bias, not the algorithm. Furthermore, if you really leave things up to only externally and easily measurable data, you might not get the results you want or expect. A computer might see that black people are much more likely to commit crime, because hey, more black people are in jail, and therefore judge that the black person was more likely to be the criminal, given a choice. But is this the correct way to think? Indeed, a human would have to think really hard to correct this kind of bias. In other words, saying that an algorithm is less biased than a human judge is somewhat of a non-statement, since the judge actually follows fairly strict rules regarding what the facts imply -- it is more how he judges what those facts are that changes the sentencing. Meanwhile, on a computer, the programmer has a lot more power than you might assume in controlling how facts are weighted in order to come up with what is seen as a "fair" decision. And that fairness is a human judgement, fundamentally speaking, since it is about the relationship between a community and an individual. In short, I don't think you can replace humans with machines when it comes to making decisions about how to deal with humans. The machine can only provide inferences from facts, but the values that those facts imply are a fundamentally human concept. It is a social thing, not an algorithm thing. Another way to put it: if laws could be perfectly encoded by logic, we would have no need for lawyers and judges. These roles exist because as a society, we are constantly re-evaluating the logic of our society. The rules are not something that can be written statically. They must change as new ideas, problems, and circumstances change.