2 ms·

The system developer has to look at race to de-bias a system, but the system shouldn't take race as input, unless it's somehow able to de-bias itself. In this e

The system developer has to look at race to de-bias a system, but the system shouldn't take race as input, unless it's somehow able to de-bias itself. In this example, I'd wonder why ZIP code is even an input.

godelski 4y ago
So I'd consider that part of the system. You could also train your model to debias by using a cost function that incorporates race and minimizes the KL divergence between different ethnicities. (Obviously it would be best to use interpretable models and/or causal ones) A Mahalanobis distance might also be good here. But when training your model you might want to PCA it and if race isn't an included factor then you can't see if it is a main contributing component or not. You should be training your model such that the racial variable contributes near nothing to the model's outcome (easier said than done). > In this example, I'd wonder why ZIP code is even an input. The school you go to is strongly correlated with zip code. So... if any of your schools are considered on your college app (hint: they are) then you have extracted the zip code (which again, points back to red lining[0]). In the crime example, the zip code is going to also correlate with the frequency and type of crime as crime (especially when broken down into types of crime) is not homogeneously distributed across a country/state/city (to be honest, not even a zip code! But the more you zoom in the more accurate you get. Aggregation bias is a pain). This stuff is a pet peeve of mine. Our tiny human brains weren't designed to figure out how all these things are connected. That's why we use data science and statistics. It is why data science and statistics is so fucking hard too! Even more so when you get into causal statistics. Contributions may frequently be obvious post hoc, but they rarely are a priori (which is why "novelty" is so absurd). The world is extremely complex and we have long chains of causality that we have to work with. The problem here is our brains are designed to simplify things. To take first or second order approximations. The problem with this is that these approximations are not sufficient for navigating the major issues we face in the modern world. These low order approximations often lead us in the wrong direction rather than being a small step in the right direction. If you're taking a gradient descent in a complex feature space (thousands of features/dimensions) then approximating it in 2D will get you nowhere near an equilibria. There's probably dozens, or hundreds, of features that are highly significant. All before we get into the mess of coupling and confounders. This is a problem because: 1) we fight over things we're not even qualified to have an opinion on; 2) the actions we take are frequently determined by popularity with a hint of analysis and the popularity is determined by low order modeling; 3) because we have a limited view of our actions we can easily convince ourselves that low order approximations are good because they work locally (spacial and temporal) but not over the global solution space (humans are notoriously terrible at long term thinking/planning and this too is highly correlated with income levels/life stability). Our modern world is highly dependent upon specialization but we have a culture that tries to convince us that expertise in an extremely narrow domain generalizes really well. It doesn't and that is destroying us. This isn't just a "you" problem, so you're not being targeted. These issues are highly common on HN and anywhere nerds congregate because that cultural pressure convinced us that our self-worth is dependent upon the generalization of our knowledge. Einstein nor Feynman said you don't understand something unless you can explain it to a simpleton/your grandma/a child. Most stuff you can't because domain expertise is required for even basic understanding. So I'm trying to put social pressure to solve this problem and I hope others will join. tldr: The world is fucking complex, treat it that way [0] One of the major issues with red lining (there are a lot) is that where you live determines the quality of education you get. School funding is often related to housing taxes. Cheaper houses? Less funding. Less funding? Lower education. Lower education? Increased difficulty in climbing the socioeconomic ladder. Which correlates with crime rates. Which correlates with a lot of things. These downstream effects aren't actually dependent upon race but if you correlate an upstream effect with race then the downstream ones do too.
- hot_gril 4y ago
  I don't assume this is easy or even possible to solve. If the model is trying to determine something like the applicant's level of adversity based on whatever info college admissions has, of course the result will have a divergence between races, and historical racism will have shaped some of your input variables (like high school) to some degree that we'll never fully understand. Using some statistical difference between races as a cost function wouldn't be satisfactory either. All I'm saying is at the bare minimum, don't feed race as an input. It's a vague factor that the applicant has zero control over, and a model (or human reviewer) making any assumptions based on race would be inherently racist, which is unjustifiable. Another bad example is how UCSD asks how feminine (rate 0-5) the applicant considers themselves. Only use factors that have some legitimate purpose; at least the high school matters because of grade inflation, but standardized tests are actually fairer. US hiring is this way, where many companies consider it an unwanted risk to even know about protected categories (age, race, religion, marital status, etc). It's far from an airtight solution. This isn't really about science, and the article is off-topic for HN.
  - godelski 4y ago
    > Using some statistical difference between races as a cost function wouldn't be satisfactory either. So how do you expect the expert to de-bias the system without some cost function?
    - hot_gril 4y ago
      I meant a cost function that the model trains on. The expert can look at the cost function to double-check whether the inputs are appropriate. Maybe the applicants' names got in there by mistake, and the names serve no purpose other than giving away their ethnicities.
      - godelski 4y ago
        > I meant a cost function that the model trains on. The expert can look at the cost function to double-check whether the inputs are appropriate. Okay, but how is the expert going to judge that the outputs of the model are appropriate? What techniques do they use?

godelski 4y ago

So I'd consider that part of the system. You could also train your model to debias by using a cost function that incorporates race and minimizes the KL divergence between different ethnicities. (Obviously it would be best to use interpretable models and/or causal ones) A Mahalanobis distance might also be good here. But when training your model you might want to PCA it and if race isn't an included factor then you can't see if it is a main contributing component or not. You should be training your model such that the racial variable contributes near nothing to the model's outcome (easier said than done). > In this example, I'd wonder why ZIP code is even an input. The school you go to is strongly correlated with zip code. So... if any of your schools are considered on your college app (hint: they are) then you have extracted the zip code (which again, points back to red lining[0]). In the crime example, the zip code is going to also correlate with the frequency and type of crime as crime (especially when broken down into types of crime) is not homogeneously distributed across a country/state/city (to be honest, not even a zip code! But the more you zoom in the more accurate you get. Aggregation bias is a pain). This stuff is a pet peeve of mine. Our tiny human brains weren't designed to figure out how all these things are connected. That's why we use data science and statistics. It is why data science and statistics is so fucking hard too! Even more so when you get into causal statistics. Contributions may frequently be obvious post hoc, but they rarely are a priori (which is why "novelty" is so absurd). The world is extremely complex and we have long chains of causality that we have to work with. The problem here is our brains are designed to simplify things. To take first or second order approximations. The problem with this is that these approximations are not sufficient for navigating the major issues we face in the modern world. These low order approximations often lead us in the wrong direction rather than being a small step in the right direction. If you're taking a gradient descent in a complex feature space (thousands of features/dimensions) then approximating it in 2D will get you nowhere near an equilibria. There's probably dozens, or hundreds, of features that are highly significant. All before we get into the mess of coupling and confounders. This is a problem because: 1) we fight over things we're not even qualified to have an opinion on; 2) the actions we take are frequently determined by popularity with a hint of analysis and the popularity is determined by low order modeling; 3) because we have a limited view of our actions we can easily convince ourselves that low order approximations are good because they work locally (spacial and temporal) but not over the global solution space (humans are notoriously terrible at long term thinking/planning and this too is highly correlated with income levels/life stability). Our modern world is highly dependent upon specialization but we have a culture that tries to convince us that expertise in an extremely narrow domain generalizes really well. It doesn't and that is destroying us. This isn't just a "you" problem, so you're not being targeted. These issues are highly common on HN and anywhere nerds congregate because that cultural pressure convinced us that our self-worth is dependent upon the generalization of our knowledge. Einstein nor Feynman said you don't understand something unless you can explain it to a simpleton/your grandma/a child. Most stuff you can't because domain expertise is required for even basic understanding. So I'm trying to put social pressure to solve this problem and I hope others will join. tldr: The world is fucking complex, treat it that way [0] One of the major issues with red lining (there are a lot) is that where you live determines the quality of education you get. School funding is often related to housing taxes. Cheaper houses? Less funding. Less funding? Lower education. Lower education? Increased difficulty in climbing the socioeconomic ladder. Which correlates with crime rates. Which correlates with a lot of things. These downstream effects aren't actually dependent upon race but if you correlate an upstream effect with race then the downstream ones do too.

hot_gril 4y ago
I don't assume this is easy or even possible to solve. If the model is trying to determine something like the applicant's level of adversity based on whatever info college admissions has, of course the result will have a divergence between races, and historical racism will have shaped some of your input variables (like high school) to some degree that we'll never fully understand. Using some statistical difference between races as a cost function wouldn't be satisfactory either. All I'm saying is at the bare minimum, don't feed race as an input. It's a vague factor that the applicant has zero control over, and a model (or human reviewer) making any assumptions based on race would be inherently racist, which is unjustifiable. Another bad example is how UCSD asks how feminine (rate 0-5) the applicant considers themselves. Only use factors that have some legitimate purpose; at least the high school matters because of grade inflation, but standardized tests are actually fairer. US hiring is this way, where many companies consider it an unwanted risk to even know about protected categories (age, race, religion, marital status, etc). It's far from an airtight solution. This isn't really about science, and the article is off-topic for HN.
- godelski 4y ago
  > Using some statistical difference between races as a cost function wouldn't be satisfactory either. So how do you expect the expert to de-bias the system without some cost function?
  - hot_gril 4y ago
    I meant a cost function that the model trains on. The expert can look at the cost function to double-check whether the inputs are appropriate. Maybe the applicants' names got in there by mistake, and the names serve no purpose other than giving away their ethnicities.
    - godelski 4y ago
      > I meant a cost function that the model trains on. The expert can look at the cost function to double-check whether the inputs are appropriate. Okay, but how is the expert going to judge that the outputs of the model are appropriate? What techniques do they use?