4 ms·

That’s not really true. Say you want to know if you should approve someone for insurance. SELECT AVG(profit) WHERE feature == $user_feature; Say “feature” is

by mirker 5y ago

That’s not really true. Say you want to know if you should approve someone for insurance. SELECT AVG(profit) WHERE feature == $user_feature; Say “feature” is education level and then suppose education level is correlated with race or gender. Then you implicitly are writing a query to filter by race or gender. Humans often don’t consider these implicit correlations. An AI would do the same. When it’s human face detection, then sure, humans would find the mistakes quickly. But SQL data is very easy to implicitly bias, and like I said, you can slice the pie millions of ways for various reasons. On the other hand, if we had a simple definition, then the database could raise an exception when the condition for the query failed.

mattmcknight 5y ago
> Say “feature” is education level and then suppose education level is correlated with race or gender. Then you implicitly are writing a query to filter by race or gender. You aren't though. Because the predictive attribute is correlated with the irrelevant attributes, not determined by them. So you will find people with multiple values of your irrelevant attributes (race, gender) for the positive value of your predictive attribute (education). Thus, it is not implicitly filtering for the irrelevant attributes. It if is truly education that is predictive, using only that would be avoiding biasing your decision on the irrelevant attributes. Perhaps the risk comes when it is not really "education" that is causative, but some hidden personality trait, say conscientiousness, for which "education" is a possible proxy. In this case, trying to correct for some measure of race/gender access to education is still not going to work. For example, it is generally reported that females score higher on both measures of conscientiousness and educational achievement. Would you lower the value of your "education" score for females and raise them for males on this basis to correct for the bias in educational achievement against those with low conscientiousness? It would only seem to make the predictive value of the feature worse.
- mirker 5y ago
  It depends on the definition of fairness, which is what I’m trying to argue for. Practically speaking, heavy correlation and casual relationships are similarly problematic. If you correlate enough of the variables, you can limit your pool to a very particular group of people. And if your group of people is 99% of race/gender/age, then is that practically any different from doing a direct filter? The usual definitions of fairness actually have the strong constraint that the response (Y=f(X)) is independent of the fairness variables (f(X|Z) is the same for all fairness values of Z). So, concretely, if Z is gender, then you have the same insurance outcome regardless of gender. In this definition, the setting I described is unfair, since the correlation on education will bias the model to predict in a way that is correlated with race. The only practical way to satisfy such a strong definition is to do a correction on the data, which is a pretty straightforward optimization problem if you formalize the constraints. And yeah, usually you lose predictive power in doing so. For the record, the strong definition makes sense often times (to me). You wouldn’t want health insurance using your DNA as a predictor, likely, even if genetics are very predictive in that domain.
  - mattmcknight 5y ago
    What depends on the definition of fairness? Surely not whether the queries are the same. The queries are only the same if one factor determines the other. This is why you immediately retreat to a 99%, which still is not practically the same, and still doesn't matter if the correct features are being captured. If I am looking to predict who can run a 4 minute mile, and it turns out my model returns no people over 60, is that unfair? Hardly. The record for an over 60 person is 4:49 so most good models would not return anyone over that age. However, if my model is based on various physical features, not including age, if a 61 year old shows up with those physical features, "age bias" will not stop the model from predicting them as capable. You are trying to shift the discussion to a 99% case, which is quite different from most correlations in the first place. This is clearly not the case for predictive variables like education. If you have a graduation rate of 90% for one group and 80% for another group, your algorithm is favoring some of each group, and disfavoring some of each group based on education, not group membership. > In this definition, the setting I described is unfair, since the correlation on education will bias the model to predict in a way that is correlated with race. You must be leaving something out of the explanation or I missed it. Say p(insurance_claim) = f(education). If this relationship holds across groups, the model is not biased. The model would be biased only if education were predictive for some subgroups and not others, which would point to education not actually being the cause, and another latent variable driving the relationship (say conscientiousness). You are suggesting that even if the relationship holds across groups, that education is predictive for all groups, that the model is still unfair, because education is not equal across all groups. I think this is more of a case of education perhaps not being fair or life not being fair than the model not being fair? If evidence of education, let's a say passing a driver safety course, is strongly, negatively correlated with insurance claims, but you find that some people cannot afford a driver safety course, it is not the model which is unfair, but perhaps the economic circumstances that led to someone not being able to afford the course. Let's not burden models of insurance risk or expected cost by making them less correct in order to pretend that unfairness in society does not exist. Having a model that is correlated with race is not evidence of bias in the model, but may be evidence of bias in society. > The only practical way to satisfy such a strong definition is to do a correction on the data, which is a pretty straightforward optimization problem if you formalize the constraints. And yeah, usually you lose predictive power in doing so. If, as in your example, you had a feature that was 99% determined by group membership, you would lose 99% of the value of that feature. This is why the "strong" definition is not helpful. > For the record, the strong definition makes sense often times (to me). You wouldn’t want health insurance using your DNA as a predictor, likely, even if genetics are very predictive in that domain. This is a completely different argument, that a more predictive model might not benefit you. However, it does get me to understand your viewpoint a little bit. You seem to want a world where the luck of what DNA you are born with does not affect you. However, if I were to use a model to predict my own healthcare costs using DNA and decide whether to buy long term care insurance, I certainly wouldn't want a model that was modified to remove "bias" and gave me a less accurate prediction of the real costs.

by mirker 5y ago

mattmcknight 5y ago

> Say “feature” is education level and then suppose education level is correlated with race or gender. Then you implicitly are writing a query to filter by race or gender. You aren't though. Because the predictive attribute is correlated with the irrelevant attributes, not determined by them. So you will find people with multiple values of your irrelevant attributes (race, gender) for the positive value of your predictive attribute (education). Thus, it is not implicitly filtering for the irrelevant attributes. It if is truly education that is predictive, using only that would be avoiding biasing your decision on the irrelevant attributes. Perhaps the risk comes when it is not really "education" that is causative, but some hidden personality trait, say conscientiousness, for which "education" is a possible proxy. In this case, trying to correct for some measure of race/gender access to education is still not going to work. For example, it is generally reported that females score higher on both measures of conscientiousness and educational achievement. Would you lower the value of your "education" score for females and raise them for males on this basis to correct for the bias in educational achievement against those with low conscientiousness? It would only seem to make the predictive value of the feature worse.

mirker 5y ago
It depends on the definition of fairness, which is what I’m trying to argue for. Practically speaking, heavy correlation and casual relationships are similarly problematic. If you correlate enough of the variables, you can limit your pool to a very particular group of people. And if your group of people is 99% of race/gender/age, then is that practically any different from doing a direct filter? The usual definitions of fairness actually have the strong constraint that the response (Y=f(X)) is independent of the fairness variables (f(X|Z) is the same for all fairness values of Z). So, concretely, if Z is gender, then you have the same insurance outcome regardless of gender. In this definition, the setting I described is unfair, since the correlation on education will bias the model to predict in a way that is correlated with race. The only practical way to satisfy such a strong definition is to do a correction on the data, which is a pretty straightforward optimization problem if you formalize the constraints. And yeah, usually you lose predictive power in doing so. For the record, the strong definition makes sense often times (to me). You wouldn’t want health insurance using your DNA as a predictor, likely, even if genetics are very predictive in that domain.
- mattmcknight 5y ago
  What depends on the definition of fairness? Surely not whether the queries are the same. The queries are only the same if one factor determines the other. This is why you immediately retreat to a 99%, which still is not practically the same, and still doesn't matter if the correct features are being captured. If I am looking to predict who can run a 4 minute mile, and it turns out my model returns no people over 60, is that unfair? Hardly. The record for an over 60 person is 4:49 so most good models would not return anyone over that age. However, if my model is based on various physical features, not including age, if a 61 year old shows up with those physical features, "age bias" will not stop the model from predicting them as capable. You are trying to shift the discussion to a 99% case, which is quite different from most correlations in the first place. This is clearly not the case for predictive variables like education. If you have a graduation rate of 90% for one group and 80% for another group, your algorithm is favoring some of each group, and disfavoring some of each group based on education, not group membership. > In this definition, the setting I described is unfair, since the correlation on education will bias the model to predict in a way that is correlated with race. You must be leaving something out of the explanation or I missed it. Say p(insurance_claim) = f(education). If this relationship holds across groups, the model is not biased. The model would be biased only if education were predictive for some subgroups and not others, which would point to education not actually being the cause, and another latent variable driving the relationship (say conscientiousness). You are suggesting that even if the relationship holds across groups, that education is predictive for all groups, that the model is still unfair, because education is not equal across all groups. I think this is more of a case of education perhaps not being fair or life not being fair than the model not being fair? If evidence of education, let's a say passing a driver safety course, is strongly, negatively correlated with insurance claims, but you find that some people cannot afford a driver safety course, it is not the model which is unfair, but perhaps the economic circumstances that led to someone not being able to afford the course. Let's not burden models of insurance risk or expected cost by making them less correct in order to pretend that unfairness in society does not exist. Having a model that is correlated with race is not evidence of bias in the model, but may be evidence of bias in society. > The only practical way to satisfy such a strong definition is to do a correction on the data, which is a pretty straightforward optimization problem if you formalize the constraints. And yeah, usually you lose predictive power in doing so. If, as in your example, you had a feature that was 99% determined by group membership, you would lose 99% of the value of that feature. This is why the "strong" definition is not helpful. > For the record, the strong definition makes sense often times (to me). You wouldn’t want health insurance using your DNA as a predictor, likely, even if genetics are very predictive in that domain. This is a completely different argument, that a more predictive model might not benefit you. However, it does get me to understand your viewpoint a little bit. You seem to want a world where the luck of what DNA you are born with does not affect you. However, if I were to use a model to predict my own healthcare costs using DNA and decide whether to buy long term care insurance, I certainly wouldn't want a model that was modified to remove "bias" and gave me a less accurate prediction of the real costs.