19 ms·

I think Yann and his critics are focusing on different questions. IIUC Yann is looking at the narrow engineering perspective of how to build an fair ML system.

I think Yann and his critics are focusing on different questions. IIUC Yann is looking at the narrow engineering perspective of how to build an fair ML system. Gebru et al seem to be talking about a much more abstract, societal trend-oriented, "let's think about if we should not just if we can" type of question, though it's still pretty fuzzy to me. Anyway, the epistemological standards of those questions are basically incompatible (incommensurable?), so discussion is impossible unless they agree on which one they're trying to answer. I'm starting to notice this kind of thing a lot.

bhouston 6y ago
If that is the case they should notice they are talking past each other and reset the conversation... But I guess that is very hard/impossible once it gets personal. Thus I think the fact that it became personal is the main problem...
- dgellow 6y ago
  Twitter doesn’t help finding out the misunderstanding and do a reset, you have arguments/counter-arguments spread between thousands of messages with no overview of what is being said, what are the core ideas being discussed.
mlyle 6y ago
I think there's two different issues here. One, if datasets are biased --- if you build your system to only work on white males --- then it may have suboptimal results for other groups. This is a common problem: you use your company's faces, or college students enrolling in data gathering exercises, etc, who are not representative of the population at large. We can fix this by being careful about dataset bias. But the second issue hits right at the heart of a major societal problem/debate. When we use AI to make decisions about people, will the system become racist--- even with representative datasets? If you train something to predict, say, odds to default on a loan-- will it figure out things that correlate to race and be making mostly racial decisions? Different races do have different default rates, but we've decided as a society that it is unfair to use race to determine an individual's probability of default. But if we choose things that are correlates of both default rates and race, when is that fair and when is it just veiled racism (redlining)? What things are measuring a causal relationship and what things are just racism in disguise? This second problem is much worse with ML, because we have the ability to accept a whole bunch more things into our models and explaining the rationale of why decision are made is much harder. And of course, the first problem-- both bias in datasets during use and biases during research and development -- makes the second problem worse. It can't even necessarily be addressed by broadening the dataset and retraining: if, in this case, you do your research and training with just white faces, and report positive results, it may not generalize to work as well for everyone with a broader dataset.
- mc32 6y ago
  Is that attenuated by making the system "blind" to race in those decision-making processes?
  - mlyle 6y ago
    It's pretty hard to make the system "blind" to race. If you blind a system predicting recidivism to race, but it is able to see a lot of things that are correlates of race... it can end up being racist anyways. E.g. for a non-ML example -- redlining is theoretically "blind" to race, but makes extremely racist decisions.
  - compiler-guy 6y ago
    That is difficult to do because there are many things that correlate with race. So even if you remove race explicitly, things like neighborhoods, level of education and such end up being proxies. Removing race from the model is a very hard problem.
    - mlyle 6y ago
      One example: Credit card issuers have been known to take into account places that you have made purchases in assessing risk. If you shop at businesses that are primarily frequented by minorities, odds are you'll get a lower credit limit than if you didn't.
      - mc32 6y ago
        I guess the question is why would that be germane to issuing credit? Is it correlated to something that indicated actual credit risk? If it makes erroneous assumptions correlations, would it not behoove them to eliminate the contamination?
        mlyle 6y ago
        Sure: some of your purchase history definitely indicates behaviors that indicate you're a credit risk. Some of your purchase history indicates you're likely black (and if you're black, your credit risk is, empirically, a little higher-- even if it is not fair to not issue credit to people because they're black). ML will figure out both but can't explain the hidden variables: it'll figure out behaviors that directly indicate higher risk, and behaviors that indicate you're a minority that tends to be a higher risk.
aaanotherhnfolk 6y ago
A variant of this is that both ppl are answering the same question but on different time scales. So one is pro (on a generational timeline) and the other is con (on a 4-year timeline). Both might be right, for their respective time scales, which is why it's so hard for either one to back down. Many many ways to talk past one another. We need a small book on techniques to short circuit these situations and salvage conversations.
an_opabinia 6y ago
> incommensurable This is pretty much the issue at its core. The real question is, if Yann is so smart, why is he arguing with people on Twitter?
- mullingitover 6y ago
  He did leave Twitter though, so not a complete fail on his part.
jacquesm 6y ago
And that's the right way to look at it. If you can't then you shouldn't. It's not enough to build stuff and then to say 'oh, we can just fix these serious problems' afterwards if you're not able to foresee / structurally address them from the get go. Because it clearly shows a much more shallow understanding of the problem space than mere engineering can fix.
- mlyle 6y ago
  Surely this comment reaches a little too far? We have to be brave enough to try new things even if they have some risk of unknown/unintended consequences. No decision is completely safe and no system gets everything right in the first pass. All we can ask for is a serious effort to understand risks sufficiently well to reasonably believe that the system is safe.
  - jacquesm 6y ago
    You can try 'new things' in the lab, until you are sure they work and are safe for general application. To field these AI solutions in lots of places where negative side effects are a serious possibility knowing that this is the case is irresponsible. Seeing this as a simple engineering problem with a simple fix-as-you-go attitude is not the right approach.
    - mlyle 6y ago
      We're talking about a university research project to rescale photos here--- not a system that is fielded. Certainly treating PULSE as a simple engineering problem and fix-as-you-go isn't so bad, is it?
      - jacquesm 6y ago
        Just like 'temporary software' university research projects tend to eventually get fielded in some form or other. By not tackling this problem at the root we push the onus for correct implementation down to the people that field it who thought the academics had done a proper job in the first place with all their metrics pointing to 'success'. But success isn't a success when the dataset isn't representative and datasets used by academics as a rule are not representative or have built-in biases that are very hard to overcome by engineers aiming to field these solutions. This has already led to numerous troubles in the applications for loans, job evaluation, criminal matters and so on.

bhouston 6y ago

If that is the case they should notice they are talking past each other and reset the conversation... But I guess that is very hard/impossible once it gets personal. Thus I think the fact that it became personal is the main problem...

dgellow 6y ago
Twitter doesn’t help finding out the misunderstanding and do a reset, you have arguments/counter-arguments spread between thousands of messages with no overview of what is being said, what are the core ideas being discussed.

mlyle 6y ago

I think there's two different issues here. One, if datasets are biased --- if you build your system to only work on white males --- then it may have suboptimal results for other groups. This is a common problem: you use your company's faces, or college students enrolling in data gathering exercises, etc, who are not representative of the population at large. We can fix this by being careful about dataset bias. But the second issue hits right at the heart of a major societal problem/debate. When we use AI to make decisions about people, will the system become racist--- even with representative datasets? If you train something to predict, say, odds to default on a loan-- will it figure out things that correlate to race and be making mostly racial decisions? Different races do have different default rates, but we've decided as a society that it is unfair to use race to determine an individual's probability of default. But if we choose things that are correlates of both default rates and race, when is that fair and when is it just veiled racism (redlining)? What things are measuring a causal relationship and what things are just racism in disguise? This second problem is much worse with ML, because we have the ability to accept a whole bunch more things into our models and explaining the rationale of why decision are made is much harder. And of course, the first problem-- both bias in datasets during use and biases during research and development -- makes the second problem worse. It can't even necessarily be addressed by broadening the dataset and retraining: if, in this case, you do your research and training with just white faces, and report positive results, it may not generalize to work as well for everyone with a broader dataset.

mc32 6y ago
Is that attenuated by making the system "blind" to race in those decision-making processes?
- mlyle 6y ago
  It's pretty hard to make the system "blind" to race. If you blind a system predicting recidivism to race, but it is able to see a lot of things that are correlates of race... it can end up being racist anyways. E.g. for a non-ML example -- redlining is theoretically "blind" to race, but makes extremely racist decisions.
- compiler-guy 6y ago
  That is difficult to do because there are many things that correlate with race. So even if you remove race explicitly, things like neighborhoods, level of education and such end up being proxies. Removing race from the model is a very hard problem.
  - mlyle 6y ago
    One example: Credit card issuers have been known to take into account places that you have made purchases in assessing risk. If you shop at businesses that are primarily frequented by minorities, odds are you'll get a lower credit limit than if you didn't.
    - mc32 6y ago
      I guess the question is why would that be germane to issuing credit? Is it correlated to something that indicated actual credit risk? If it makes erroneous assumptions correlations, would it not behoove them to eliminate the contamination?
      - mlyle 6y ago
        Sure: some of your purchase history definitely indicates behaviors that indicate you're a credit risk. Some of your purchase history indicates you're likely black (and if you're black, your credit risk is, empirically, a little higher-- even if it is not fair to not issue credit to people because they're black). ML will figure out both but can't explain the hidden variables: it'll figure out behaviors that directly indicate higher risk, and behaviors that indicate you're a minority that tends to be a higher risk.

aaanotherhnfolk 6y ago

A variant of this is that both ppl are answering the same question but on different time scales. So one is pro (on a generational timeline) and the other is con (on a 4-year timeline). Both might be right, for their respective time scales, which is why it's so hard for either one to back down. Many many ways to talk past one another. We need a small book on techniques to short circuit these situations and salvage conversations.

an_opabinia 6y ago

> incommensurable This is pretty much the issue at its core. The real question is, if Yann is so smart, why is he arguing with people on Twitter?

mullingitover 6y ago
He did leave Twitter though, so not a complete fail on his part.

jacquesm 6y ago

And that's the right way to look at it. If you can't then you shouldn't. It's not enough to build stuff and then to say 'oh, we can just fix these serious problems' afterwards if you're not able to foresee / structurally address them from the get go. Because it clearly shows a much more shallow understanding of the problem space than mere engineering can fix.

mlyle 6y ago
Surely this comment reaches a little too far? We have to be brave enough to try new things even if they have some risk of unknown/unintended consequences. No decision is completely safe and no system gets everything right in the first pass. All we can ask for is a serious effort to understand risks sufficiently well to reasonably believe that the system is safe.
- jacquesm 6y ago
  You can try 'new things' in the lab, until you are sure they work and are safe for general application. To field these AI solutions in lots of places where negative side effects are a serious possibility knowing that this is the case is irresponsible. Seeing this as a simple engineering problem with a simple fix-as-you-go attitude is not the right approach.
  - mlyle 6y ago
    We're talking about a university research project to rescale photos here--- not a system that is fielded. Certainly treating PULSE as a simple engineering problem and fix-as-you-go isn't so bad, is it?
    - jacquesm 6y ago
      Just like 'temporary software' university research projects tend to eventually get fielded in some form or other. By not tackling this problem at the root we push the onus for correct implementation down to the people that field it who thought the academics had done a proper job in the first place with all their metrics pointing to 'success'. But success isn't a success when the dataset isn't representative and datasets used by academics as a rule are not representative or have built-in biases that are very hard to overcome by engineers aiming to field these solutions. This has already led to numerous troubles in the applications for loans, job evaluation, criminal matters and so on.