47 ms·

> Any sufficiently complex programming project will end up reflecting some assumptions and biases on the behalf of the programmer. As more programmers contribut

> Any sufficiently complex programming project will end up reflecting some assumptions and biases on the behalf of the programmer. As more programmers contribute to a project, it won't reflect an individual's biases, but will still reflect those programmers as a group. Can you break this down more? It doesn't make sense to me. If you're writing a machine learning application to take a dataset and match future inputs to past results I don't see how these biases can sneak into the program. Unless the programmers are changing the datasets then I don't see how this makes sense.

theemathas 10y ago
Well... why can't the dataset be biased?
- amitdeshwar 10y ago
  That's not a bias of the programmer
  - chongli 10y ago
    Unless the programmer chose the dataset.
woodruffw 10y ago
In the case you presented, the biases are in the dataset itself. Arrest statistics in the US are heavily skewed by race. If you were to take a dataset of all US arrests between 1900 and 2000 and ask which populations are most likely to "commit crimes" (i.e., be arrested), you'd get racially biased model without recording why it's biased (discrimination, enduring poverty, minimum sentencing, &c).
- dante821 10y ago
  Can't that problem be solved by simply not telling the machine the race of the person who's being judged?
- mattnewton 10y ago
  Dead sibling comment asks a common questions I hear: why can't we just not tell the algorithm race? And the answer is that, with enough data points, if there is a statistical bias in the dataset, the algorithm will likely learn a substitute for race in the other features. For the sake of argument, perhaps an interaction between home address, annual income and model of car driven is predictive of race, and race is predictive of recidivism in the dataset- then the algorithm will learn this cross of features is predictive of recividism, even though we would like all of them to ideally be irrelevant to sentencing.
  - gravypod 10y ago
    > home address, annual income and model of car driven is predictive Why the hell would those be factors? The factors of the case should be things that actually matter in the case. A rich person who murders someone should get the same sentence as a poor person who murders someone. A black person who murders someone should get the same sentence as a white person who murders someone. Adding those factors would be insane in the first place. If you're adding crazy things like that you might as well add factors like "Can Juggle" and "Can Burp the Alphabet" because things like that should have just as much to do with sentencing as what kind of car you drive or where you live.
    - jcranmer 10y ago
      One example that's going to come up is "Has the person previously been convicted of a crime?" That question is strongly correlated with race, and yet is strongly arguable to be relevant. Model of car is hard to say that it's useful for predictive value, but home address (=which neighborhood did you grow up in, e.g., are you in Cabrini-Green or are you Gold Coast?) and annual income (poor people are more likely to commit crimes than rich people) definitely are going to be queried inputs. Particularly if the question is things like "what should I set bail at?"
      - gravypod 10y ago
        > "Has the person previously been convicted of a crime?" That question is strongly correlated with race, and yet is strongly arguable to be relevant. Past behavior is a good predictor of future behavior in my experience. That's more than doubled when it's recurring behavior. > home address (=which neighborhood did you grow up in, e.g., are you in Cabrini-Green or are you Gold Coast?) and annual income (poor people are more likely to commit crimes than rich people) definitely are going to be queried inputs Why? > Particularly if the question is things like "what should I set bail at?" That's still the job of the judge and there is law at hand for how those values are calculated. The computers won't be involved with that (I'd assume) just sentencing based on previous case decisions.
- dilemma 10y ago
  >In the case you presented, the biases are in the dataset itself. The code that is going to handle the data is also biased in some direction. Its actions is value-based, coming from that of the creators of the algorithm. This is why all code must be open and publicly auditable if used in law enforcement, health care and insurance, etc.
dogecoinbase 10y ago
How was the dataset produced?
deleted 10y ago
[deleted]
randomtask 10y ago
As with anything, it's hard to generalise without oversimplifying, but here goes. You don't generally just have a data set and a machine learning algorithm that somehow magics outputs from a data set. Usually decisions have to be made by people either in training the model, selecting variables that are included in a model, etc. Here's a simple example. Say you're trying to come up with an algorithm that decides whether articles in a data set are "fake news" (topical, I know). We have to tell the algorithm whether a given article in the training set is fake or legitimate, otherwise how would it know? Clearly this will reflect the views of whoever is tagging the articles. When we run the model on a training set we need to score how well it did, again this will reflect the opinion of the person doing the scoring. For a real example: https://mathbabe.org/2016/05/12/algorithms-are-as-biased-as-.. https://mathbabe.org/2016/05/12/algorithms-are-as-biased-as-....
- gravypod 10y ago
  I've done some machine learning in the past. I get how you tackle machine learning problems and the first thing that I would say that your proposed topic is not currently possible. "*Say you're trying to come up with an algorithm that decides whether articles in a data set are "fake news" (topical, I know).*" Current AI cannot do this. This would take finding sources, pulling data out of those sources, cross referencing multiple sources, and recurring for those articles to a certain depth. That's not a good facsimile for deciding sentencing. Sentencing is more like a linear regression classification. You have a history of previous cases where the defendant was found guilty. You then have a pile of factors that played into the judge's decision for sentencing. For example: * If they meant to do it * If they feel bad about doing it * If they did do it (Beyond a reasonable doubt) * If they have done it before * What severity this crime is * ... etc The judge then uses their experience in law and previous case law as well as statues to find a proper punishment. This is in the form of: * Time served * Fines * Privileges revoked * Community Service This would then be fed into a classification engine. You leave all of the existing infrastructure in place (Judge, Jury, Lawers) and just use their decision as input into the sentencing. Deciding the validity of claims is not within the scope of modern day machine learning (as of 2017). Classification engines are very much in the scope of machine learning of today. I don't see how case factors could be biased. I don't see how historical cases (when stripped of all identifying information) could be biased. I don't see why a system like this would be bad. All treatment of everyone would converge into a uniform handling of cases.

theemathas 10y ago

Well... why can't the dataset be biased?

amitdeshwar 10y ago
That's not a bias of the programmer
- chongli 10y ago
  Unless the programmer chose the dataset.

woodruffw 10y ago

In the case you presented, the biases are in the dataset itself. Arrest statistics in the US are heavily skewed by race. If you were to take a dataset of all US arrests between 1900 and 2000 and ask which populations are most likely to "commit crimes" (i.e., be arrested), you'd get racially biased model without recording why it's biased (discrimination, enduring poverty, minimum sentencing, &c).

dante821 10y ago
Can't that problem be solved by simply not telling the machine the race of the person who's being judged?
mattnewton 10y ago
Dead sibling comment asks a common questions I hear: why can't we just not tell the algorithm race? And the answer is that, with enough data points, if there is a statistical bias in the dataset, the algorithm will likely learn a substitute for race in the other features. For the sake of argument, perhaps an interaction between home address, annual income and model of car driven is predictive of race, and race is predictive of recidivism in the dataset- then the algorithm will learn this cross of features is predictive of recividism, even though we would like all of them to ideally be irrelevant to sentencing.
- gravypod 10y ago
  > home address, annual income and model of car driven is predictive Why the hell would those be factors? The factors of the case should be things that actually matter in the case. A rich person who murders someone should get the same sentence as a poor person who murders someone. A black person who murders someone should get the same sentence as a white person who murders someone. Adding those factors would be insane in the first place. If you're adding crazy things like that you might as well add factors like "Can Juggle" and "Can Burp the Alphabet" because things like that should have just as much to do with sentencing as what kind of car you drive or where you live.
  - jcranmer 10y ago
    One example that's going to come up is "Has the person previously been convicted of a crime?" That question is strongly correlated with race, and yet is strongly arguable to be relevant. Model of car is hard to say that it's useful for predictive value, but home address (=which neighborhood did you grow up in, e.g., are you in Cabrini-Green or are you Gold Coast?) and annual income (poor people are more likely to commit crimes than rich people) definitely are going to be queried inputs. Particularly if the question is things like "what should I set bail at?"
    - gravypod 10y ago
      > "Has the person previously been convicted of a crime?" That question is strongly correlated with race, and yet is strongly arguable to be relevant. Past behavior is a good predictor of future behavior in my experience. That's more than doubled when it's recurring behavior. > home address (=which neighborhood did you grow up in, e.g., are you in Cabrini-Green or are you Gold Coast?) and annual income (poor people are more likely to commit crimes than rich people) definitely are going to be queried inputs Why? > Particularly if the question is things like "what should I set bail at?" That's still the job of the judge and there is law at hand for how those values are calculated. The computers won't be involved with that (I'd assume) just sentencing based on previous case decisions.
dilemma 10y ago
>In the case you presented, the biases are in the dataset itself. The code that is going to handle the data is also biased in some direction. Its actions is value-based, coming from that of the creators of the algorithm. This is why all code must be open and publicly auditable if used in law enforcement, health care and insurance, etc.

dogecoinbase 10y ago

How was the dataset produced?

deleted 10y ago

[deleted]

randomtask 10y ago

As with anything, it's hard to generalise without oversimplifying, but here goes. You don't generally just have a data set and a machine learning algorithm that somehow magics outputs from a data set. Usually decisions have to be made by people either in training the model, selecting variables that are included in a model, etc. Here's a simple example. Say you're trying to come up with an algorithm that decides whether articles in a data set are "fake news" (topical, I know). We have to tell the algorithm whether a given article in the training set is fake or legitimate, otherwise how would it know? Clearly this will reflect the views of whoever is tagging the articles. When we run the model on a training set we need to score how well it did, again this will reflect the opinion of the person doing the scoring. For a real example: https://mathbabe.org/2016/05/12/algorithms-are-as-biased-as-.. https://mathbabe.org/2016/05/12/algorithms-are-as-biased-as-....

gravypod 10y ago
I've done some machine learning in the past. I get how you tackle machine learning problems and the first thing that I would say that your proposed topic is not currently possible. "*Say you're trying to come up with an algorithm that decides whether articles in a data set are "fake news" (topical, I know).*" Current AI cannot do this. This would take finding sources, pulling data out of those sources, cross referencing multiple sources, and recurring for those articles to a certain depth. That's not a good facsimile for deciding sentencing. Sentencing is more like a linear regression classification. You have a history of previous cases where the defendant was found guilty. You then have a pile of factors that played into the judge's decision for sentencing. For example: * If they meant to do it * If they feel bad about doing it * If they did do it (Beyond a reasonable doubt) * If they have done it before * What severity this crime is * ... etc The judge then uses their experience in law and previous case law as well as statues to find a proper punishment. This is in the form of: * Time served * Fines * Privileges revoked * Community Service This would then be fed into a classification engine. You leave all of the existing infrastructure in place (Judge, Jury, Lawers) and just use their decision as input into the sentencing. Deciding the validity of claims is not within the scope of modern day machine learning (as of 2017). Classification engines are very much in the scope of machine learning of today. I don't see how case factors could be biased. I don't see how historical cases (when stripped of all identifying information) could be biased. I don't see why a system like this would be bad. All treatment of everyone would converge into a uniform handling of cases.