6 ms·

Whether or not the differences in society are the product of bias is it's own can of worms. But what you're writing is demonstrative of what I'm saying. People,

Whether or not the differences in society are the product of bias is it's own can of worms. But what you're writing is demonstrative of what I'm saying. People, unlike machines, tend to regard differences between groups with suspicion and are biased against acknowledging these differences. Machines, on the other hand, do not harbor these biases and build a model that reflects the reality we live in. As I wrote in my original comment, there are potential justifications as to why we might want to introduce bias in our machine learning models to prevent the from identifying things we don't want. But the point remains: building a model that, for example, associates technology with men and women at equal rates despite the disparity that exists in the reality is not the elimination of bias. It's the deliberate introduction of bias to make the model produce a result that is in line with what we consider a more ideal worldview.

crooked-v 7y ago
> building a model that, for example, associates technology with men and women at equal rates despite the disparity that exists in the reality is not the elimination of bias You're assuming that that disparity is not itself a result of bias in any way.
- manfredo 7y ago
  Maybe it is, maybe it isn't. But this is not relevant to what I'm saying. The goal of the AI is not to associate words in a hypothetical world where there is no discrimination or bias. The goal is to build a model that accurately reflects the real world. Again, there may be justifications to deliberately introduce bias in the model to produce higher rates of association between women and technology than would normally occur if the creators of the AI so desire. But this is not elimination of bias, this is deliberate introduction of bias to make th model match the creators' worldviews.
  - crooked-v 7y ago
    AI does not, in and of itself, have a goal. AI exists to be used for some human's goal. If the human's goal is to understand the level of association between women and technology without confounding factors like decades of marketing video games and computers to boys more than girls, the historical bias of such factors in the data fed to the AI becomes extremely important.
  - gbrown 7y ago
    > The goal is to build a model that accurately reflects the real world. This is incredibly reductive - we use models for MANY purposes - prediction, inference, description, evaluation. Until you outline a specific use case for the model, understand where the data its trained on comes from and how it may or may not reflect reality, and think about whether the phenomena which you're measuring are important/sufficient for your application you CANNOT possibly justify your broad claims.
    - manfredo 7y ago
      I'm well aware that models are used for a variety of applications. But regardless of the application, we usually want the model to reflect reality in some way. Say you have an AI that predicts probability of re-offending for prospective parolees. If we observe that this system is flagging men as high risk more frequently than women, then this is not indicative of bias. But!, one might say, men are socialized to be more violent and are more frequently recruited by gangs. These factors are being absorbed by the AI. Yes, they are, and this does not make the AI biased. The job of the AI to predict risk of re-offending in real world conditions. Not risk of re-offending in a hypothetical world where everyone has the same environment and experiences. If we were to manually adjust the model to make it display equal risks for both men and women then that is not elimination of bias. It is deliberate introduction of bias to make the output fit our ideal worldview.
      - crooked-v 7y ago
        > If we observe that this system is flagging men as high risk more frequently than women, then this is not indicative of bias. If all it's doing is associating being a man with a higher risk factor without taking into account anything else in that man's life then, yes, it's being biased. "Being a man" in this situation is a correlation to, not casuation of, actual factors making a person more likely to commit crime. This is a real problem that has been happening for years in Florida: the automated risk assessments used for a variety of decisions, including bail and sentencing factors, labels black people as inherently being more likely to reoffend than white people, even after controlling for actual crime rates and recidivism rates. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing https://www.propublica.org/article/machine-bias-risk-assessm...
  - loopz 7y ago
    This is a common misunderstanding. When models impact the world, what matters more are their outcomes and consequences.
    - manfredo 7y ago
      Right. And when you force a model to avoid disparities when disparities really do exist, the outcome is inaccurate data and the consequence is that decisions are made based on inaccurate data.
      - loopz 7y ago
        Sure, and there is also the assumption that good decisions are made on perfect/good data, which is also just an assumption. We can't even define good decisions well enough to be sure. Just because bad data may tend (in hindsight) to yield inaccurate decisions, or decisions based on faulty data, doesn't promote data as the main driver behind sound decisions. That would just be another belief system, and we don't have all possible data or omniscience to prove it!
        manfredo 7y ago
        No, you can test and gauge the quality of that data. At this point, you're not really even making a point about whether or not we should introduce biases in our models to make them match our worldview - you're saying the entire endeavor of trying to build a model is worthless. If what you're saying is true, that we can't measure what decisions are successful or unsuccessful, then there's no point in trying to build a model because we no way of knowing whether the model is working or not.
        loopz 7y ago
        It is the claim of vendors that their models works. It is irresponsible to just take their word for it, and such tests and critique need be ongoing.
      - alexknvl 7y ago
HBKXNCUO 7y ago
>People, unlike machines, tend to regard differences between groups with suspicion and are biased against acknowledging these differences. That is true for people in the western world that have been heavily dosed with the ideas coming out of the western social sciences in the past 100 years or so, and not really anyone anywhere else, ever.
empath75 7y ago
> People, unlike machines, tend to regard differences between groups with suspicion and are biased against acknowledging these differences. I would argue that the case is precisely the opposite. I was literally talking to someone from Colombia yesterday who was complaining to me about how all Venezuelans are lazy criminals that are stealing their jobs, when in an American context, people wouldn't even distinguish between them -- they'd both be 'hispanics' or 'latinos'. That person took a difference caused by a temporary political and economic situation and applied it to a population of people as an inherent property. I think that is actually the way that _most_ people think, and to make people think otherwise requires a lot of education.
- HBKXNCUO 7y ago
  >and to make people think otherwise requires a lot of education. With enough education, you can make most people think just about anything.
  - loopz 7y ago
    Education should be to make people think for themselves, come up with new ideas, inventions, better conventions or just enjoy life more.
    - HBKXNCUO 7y ago
      That would be nice. Too bad the people with the power to direct the education of the masses don't benefit from that and have no loyalty to the masses either.
- manfredo 7y ago
  Sure, I suppose I should have qualified that statement to refer to left leaning Americans (which are the primary demographic in US tech companies). Among these circles it is not socially acceptable to acknowledge disparities between groups, particularly when those disparities are related to demographics considered marginalized.
  - 7y ago

crooked-v 7y ago

> building a model that, for example, associates technology with men and women at equal rates despite the disparity that exists in the reality is not the elimination of bias You're assuming that that disparity is not itself a result of bias in any way.

manfredo 7y ago
Maybe it is, maybe it isn't. But this is not relevant to what I'm saying. The goal of the AI is not to associate words in a hypothetical world where there is no discrimination or bias. The goal is to build a model that accurately reflects the real world. Again, there may be justifications to deliberately introduce bias in the model to produce higher rates of association between women and technology than would normally occur if the creators of the AI so desire. But this is not elimination of bias, this is deliberate introduction of bias to make th model match the creators' worldviews.
- crooked-v 7y ago
  AI does not, in and of itself, have a goal. AI exists to be used for some human's goal. If the human's goal is to understand the level of association between women and technology without confounding factors like decades of marketing video games and computers to boys more than girls, the historical bias of such factors in the data fed to the AI becomes extremely important.
- gbrown 7y ago
  > The goal is to build a model that accurately reflects the real world. This is incredibly reductive - we use models for MANY purposes - prediction, inference, description, evaluation. Until you outline a specific use case for the model, understand where the data its trained on comes from and how it may or may not reflect reality, and think about whether the phenomena which you're measuring are important/sufficient for your application you CANNOT possibly justify your broad claims.
  - manfredo 7y ago
    I'm well aware that models are used for a variety of applications. But regardless of the application, we usually want the model to reflect reality in some way. Say you have an AI that predicts probability of re-offending for prospective parolees. If we observe that this system is flagging men as high risk more frequently than women, then this is not indicative of bias. But!, one might say, men are socialized to be more violent and are more frequently recruited by gangs. These factors are being absorbed by the AI. Yes, they are, and this does not make the AI biased. The job of the AI to predict risk of re-offending in real world conditions. Not risk of re-offending in a hypothetical world where everyone has the same environment and experiences. If we were to manually adjust the model to make it display equal risks for both men and women then that is not elimination of bias. It is deliberate introduction of bias to make the output fit our ideal worldview.
    - crooked-v 7y ago
      > If we observe that this system is flagging men as high risk more frequently than women, then this is not indicative of bias. If all it's doing is associating being a man with a higher risk factor without taking into account anything else in that man's life then, yes, it's being biased. "Being a man" in this situation is a correlation to, not casuation of, actual factors making a person more likely to commit crime. This is a real problem that has been happening for years in Florida: the automated risk assessments used for a variety of decisions, including bail and sentencing factors, labels black people as inherently being more likely to reoffend than white people, even after controlling for actual crime rates and recidivism rates. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing https://www.propublica.org/article/machine-bias-risk-assessm...
- loopz 7y ago
  This is a common misunderstanding. When models impact the world, what matters more are their outcomes and consequences.
  - manfredo 7y ago
    Right. And when you force a model to avoid disparities when disparities really do exist, the outcome is inaccurate data and the consequence is that decisions are made based on inaccurate data.
    - loopz 7y ago
      Sure, and there is also the assumption that good decisions are made on perfect/good data, which is also just an assumption. We can't even define good decisions well enough to be sure. Just because bad data may tend (in hindsight) to yield inaccurate decisions, or decisions based on faulty data, doesn't promote data as the main driver behind sound decisions. That would just be another belief system, and we don't have all possible data or omniscience to prove it!
      - manfredo 7y ago
        No, you can test and gauge the quality of that data. At this point, you're not really even making a point about whether or not we should introduce biases in our models to make them match our worldview - you're saying the entire endeavor of trying to build a model is worthless. If what you're saying is true, that we can't measure what decisions are successful or unsuccessful, then there's no point in trying to build a model because we no way of knowing whether the model is working or not.
        loopz 7y ago
        It is the claim of vendors that their models works. It is irresponsible to just take their word for it, and such tests and critique need be ongoing.
    - alexknvl 7y ago

HBKXNCUO 7y ago

>People, unlike machines, tend to regard differences between groups with suspicion and are biased against acknowledging these differences. That is true for people in the western world that have been heavily dosed with the ideas coming out of the western social sciences in the past 100 years or so, and not really anyone anywhere else, ever.

empath75 7y ago

> People, unlike machines, tend to regard differences between groups with suspicion and are biased against acknowledging these differences. I would argue that the case is precisely the opposite. I was literally talking to someone from Colombia yesterday who was complaining to me about how all Venezuelans are lazy criminals that are stealing their jobs, when in an American context, people wouldn't even distinguish between them -- they'd both be 'hispanics' or 'latinos'. That person took a difference caused by a temporary political and economic situation and applied it to a population of people as an inherent property. I think that is actually the way that _most_ people think, and to make people think otherwise requires a lot of education.

HBKXNCUO 7y ago
>and to make people think otherwise requires a lot of education. With enough education, you can make most people think just about anything.
- loopz 7y ago
  Education should be to make people think for themselves, come up with new ideas, inventions, better conventions or just enjoy life more.
  - HBKXNCUO 7y ago
    That would be nice. Too bad the people with the power to direct the education of the masses don't benefit from that and have no loyalty to the masses either.
manfredo 7y ago
Sure, I suppose I should have qualified that statement to refer to left leaning Americans (which are the primary demographic in US tech companies). Among these circles it is not socially acceptable to acknowledge disparities between groups, particularly when those disparities are related to demographics considered marginalized.
- 7y ago