3 ms·

> Most importantly, most these applications will go back to human judgement. And humans are far worse. I think you have to be careful there. Leaving decisions

> Most importantly, most these applications will go back to human judgement. And humans are far worse. I think you have to be careful there. Leaving decisions to machines embeds an assumption that all information related a decision can (and has been) encoded in a machine-interpretable manner. How often are circumstances and context taken into account during sentencing hearings? How often do judges actually try to consider things like whether the offender feels remorse? Sure, these kinds of things can be encoded into numbers, but should they be? What does it mean to encode remorse into e.g. a 7-valued point system? How much does that depend on the person entering the number, who must judge this, or should it be detected automatically using sensors? Should we be modifying sentences according to an offender's level of galvanic skin response? This is just an example of course, but my point in general is that you make a huge assumption that factors related to decision making can be encoded properly and in an unbiased manner. I'd even say that many of the factors that would need to be entered into a computer would have to be judged by a human, and therefore leaving the actual decision-making to a computer is a bit of a symbolic gesture in that sense. I would guess that, given the same data, a computer could/would be programmed to make the same judgement as a human. It is the data extraction process itself that incurs bias, not the algorithm. Furthermore, if you really leave things up to only externally and easily measurable data, you might not get the results you want or expect. A computer might see that black people are much more likely to commit crime, because hey, more black people are in jail, and therefore judge that the black person was more likely to be the criminal, given a choice. But is this the correct way to think? Indeed, a human would have to think really hard to correct this kind of bias. In other words, saying that an algorithm is less biased than a human judge is somewhat of a non-statement, since the judge actually follows fairly strict rules regarding what the facts imply -- it is more how he judges what those facts are that changes the sentencing. Meanwhile, on a computer, the programmer has a lot more power than you might assume in controlling how facts are weighted in order to come up with what is seen as a "fair" decision. And that fairness is a human judgement, fundamentally speaking, since it is about the relationship between a community and an individual. In short, I don't think you can replace humans with machines when it comes to making decisions about how to deal with humans. The machine can only provide inferences from facts, but the values that those facts imply are a fundamentally human concept. It is a social thing, not an algorithm thing. Another way to put it: if laws could be perfectly encoded by logic, we would have no need for lawyers and judges. These roles exist because as a society, we are constantly re-evaluating the logic of our society. The rules are not something that can be written statically. They must change as new ideas, problems, and circumstances change.

Houshalter 10y ago
> Leaving decisions to machines embeds an assumption that all information related a decision can (and has been) encoded in a machine-interpretable manner. Im not assuming that at all. Even with less information , algorithms generally outperform humans. E.g. a hiring algorithm accurately predict job performance from just info on an application, while someone who actually interviews them is barely better than chance. I'm on mobile right now, but I have some links to a lot of research like this, when just one or two simple features is enough to beat human experts. > Sure, these kinds of things can be encoded into numbers, but should they be? What does it mean to encode remorse into e.g. a 7-valued point system? How much does that depend on the person entering the number, who must judge this, or should it be detected automatically using sensors? Should we be modifying sentences according to an offender's level of galvanic skin response? First of all, the optimal way to do it would be to have the human give a recommendation, then have the algorithm use that as a feature. So it can take human input into account, but also improve on it. Second, encoding subjective things as real values just lays bare the silliness of it. A human factoring this things in his head is not magically more objective or less biased. Trying to rate subjective features can sometimes improve performance for humans too, since it makes them actually think about it consciously . > A computer might see that black people are much more likely to commit crime, because hey, more black people are in jail, and therefore judge that the black person was more likely to be the criminal, given a choice. But is this the correct way to think? But I'm not talking about predicting if someone is a criminal . I'm taking about things like loan defaults and health risk. Here the company has pretty objective ground truth. If women really are less likely to get into car accidents, then it's just a fact. Theres no hidden bias here.
- radarsat1 10y ago
  > A human factoring this things in his head is not magically more objective or less biased. I get your point, and I think you make some good arguments, especially regarding the small number of data points needed. In fact, in this regard I think that machine learning (or statistics anyway) can be a very useful tool for informing decisions. I think it should be regarded similarly to how experts are used in court. But I think you're missing mine. It is not that humans are better or worse at having bias when considering factors. It is that some factors are inherently biased. And it is the choice of what we do with those biases that we call "values". Now, do you think "values" are things that can be encoded in a machine-precise fashion? Well, it's what we call "law". However, law is obviously _not_ something that can be applied "by algorithm," because otherwise we would have nothing to discuss when it comes to hearings. > But I'm not talking about predicting if someone is a criminal . I'm taking about things like loan defaults and health risk. Here the company has pretty objective ground truth. If women really are less likely to get into car accidents, then it's just a fact. Theres no hidden bias here. That is true. I think that categorizing people is something that will never go away. Whether it feels "right" or not is certainly debatable. People with pre-existing conditions aren't exactly happy with the current state of insurance. Do you think black people like it when they can't get a loan because they are black? Do we want society to work that way? You say that the machine can be used to make a better decision -- simple, remove the "skin colour" category. But perhaps that leads to a decrease in the confidence interval. Insurance/banks don't do that, because they consider it important information -- hence, (maybe) the need for regulation. I think this EU regulation is going to be a catalyser for some very interesting discussions around these issues. Basically what I'm saying is, talking about algorithms vs. humans is a side issue, when the real discussion is, on what kind of data do want decisions to be made?

Houshalter 10y ago

> Leaving decisions to machines embeds an assumption that all information related a decision can (and has been) encoded in a machine-interpretable manner. Im not assuming that at all. Even with less information , algorithms generally outperform humans. E.g. a hiring algorithm accurately predict job performance from just info on an application, while someone who actually interviews them is barely better than chance. I'm on mobile right now, but I have some links to a lot of research like this, when just one or two simple features is enough to beat human experts. > Sure, these kinds of things can be encoded into numbers, but should they be? What does it mean to encode remorse into e.g. a 7-valued point system? How much does that depend on the person entering the number, who must judge this, or should it be detected automatically using sensors? Should we be modifying sentences according to an offender's level of galvanic skin response? First of all, the optimal way to do it would be to have the human give a recommendation, then have the algorithm use that as a feature. So it can take human input into account, but also improve on it. Second, encoding subjective things as real values just lays bare the silliness of it. A human factoring this things in his head is not magically more objective or less biased. Trying to rate subjective features can sometimes improve performance for humans too, since it makes them actually think about it consciously . > A computer might see that black people are much more likely to commit crime, because hey, more black people are in jail, and therefore judge that the black person was more likely to be the criminal, given a choice. But is this the correct way to think? But I'm not talking about predicting if someone is a criminal . I'm taking about things like loan defaults and health risk. Here the company has pretty objective ground truth. If women really are less likely to get into car accidents, then it's just a fact. Theres no hidden bias here.

radarsat1 10y ago
> A human factoring this things in his head is not magically more objective or less biased. I get your point, and I think you make some good arguments, especially regarding the small number of data points needed. In fact, in this regard I think that machine learning (or statistics anyway) can be a very useful tool for informing decisions. I think it should be regarded similarly to how experts are used in court. But I think you're missing mine. It is not that humans are better or worse at having bias when considering factors. It is that some factors are inherently biased. And it is the choice of what we do with those biases that we call "values". Now, do you think "values" are things that can be encoded in a machine-precise fashion? Well, it's what we call "law". However, law is obviously _not_ something that can be applied "by algorithm," because otherwise we would have nothing to discuss when it comes to hearings. > But I'm not talking about predicting if someone is a criminal . I'm taking about things like loan defaults and health risk. Here the company has pretty objective ground truth. If women really are less likely to get into car accidents, then it's just a fact. Theres no hidden bias here. That is true. I think that categorizing people is something that will never go away. Whether it feels "right" or not is certainly debatable. People with pre-existing conditions aren't exactly happy with the current state of insurance. Do you think black people like it when they can't get a loan because they are black? Do we want society to work that way? You say that the machine can be used to make a better decision -- simple, remove the "skin colour" category. But perhaps that leads to a decrease in the confidence interval. Insurance/banks don't do that, because they consider it important information -- hence, (maybe) the need for regulation. I think this EU regulation is going to be a catalyser for some very interesting discussions around these issues. Basically what I'm saying is, talking about algorithms vs. humans is a side issue, when the real discussion is, on what kind of data do want decisions to be made?