5 ms·

Why? It sounds like ideologically driven circular reasoning. If you train an AI on the largest dataset it's possible to obtain then you have, almost by definiti

Why? It sounds like ideologically driven circular reasoning. If you train an AI on the largest dataset it's possible to obtain then you have, almost by definition, done the most you can to avoid bias of any sort: the model will learn the most accurate representation of reality it can given the data available. Gebru is the type of person who defines "bias" as anything that isn't sufficiently positive towards people who look like herself, not the usual definition of a deviation from reality as exists. Having encountered AI "fairness" and "bias" papers (words quoted because the words aren't used with their dictionary definitions), it's not even clear to me they should count as research at all, let alone be worth reading. They take as the starting premise that anything a model learns about the world that is politically incorrect is a bug, and go downhill from there.

kevinventullo 6y ago
If you train an AI on the largest dataset it's possible to obtain then you have, almost by definition, done the most you can to avoid bias of any sort All politics aside, this is not even true for toy ML problems. If I’m trying to do digit recognition and “all the data I can find” is a billion hand-written 0’s and a million hand-written 1’s through 9’s, naively training on that data will yield a model that’s pretty close to guessing 0 every time.
- thu2111 6y ago
  We're talking about tech firms that have access to the entire internet and use it. Hypothetical examples involving imaginary datasets that nobody would use don't prove anything relevant. And note that my argument is not about whether you actually avoid bias, it's about whether you've done the most you can do to avoid it. If you used all the data you've got, then you've done the most you can, even if for some reason the data you've got isn't any good.
  - d_e_solomon 6y ago
    > We're talking about tech firms that have access to the entire internet and use it. I think you're trying to say that a large enough dataset will be free of bias. I don't see how that follows. If I train a model on home mortgage decisions, I will replicate the bias on that currently exists on that dataset - https://news.northwestern.edu/stories/2020/01/racial-discrimination-in-mortgage-market-persistent-over-last-four-decades/ https://news.northwestern.edu/stories/2020/01/racial-discrim... - unless there are conscientious choices to reduce that bias. Researchers in ethics in ML are specifically trying to enable tech companies to do a better job of not replicating bias and justifiably point out where that is occurring. Third, I would argue that applying an ML model to do something faster if it replicates the bias of a previously human decision is even worse. The bias has taken the human element completely out and systematized the bias and made it possible with even less friction.
    - bisectable 6y ago
      Given the current public discourse climate in academia, where 99% of the researchers are politically left or extreme left, I recommend a grain of salt for any research that makes claims racial discrimination. The report you quote has at least a couple of strange holes. The paper itself is behind a paywall, thus beyond mere mortals reach. > For example, in about 10% audits in which a white and an African-American auditor were sent to apply for the same unit after 2005, the white auditor was recommended more units than the African-American auditor. These trends hold in both the large HUD (Housing and Urban Development)-sponsored housing audits, which others have examined with similar findings to us, and in smaller correspondence studies They fail to mention how large is the gap. Is the white auditor recommended 102 vs the black auditor 97, or 150 vs 50, or 200 vs 3? Without such critical information it is hard to form an opinion, unless one already has a large bias in accepting discrimination narratives uncritically. > In the mortgage market the researchers found that racial gaps in loan denial have declined only slightly, and racial gaps in mortgage cost have not declined at all, suggesting persistent racial discrimination. Black and Hispanic borrowers are more likely to be rejected when they apply for a loan and are more likely to receive a high-cost mortgage. They fail to mention the magic words 'when controlled for income'. America has a huge income disparity problem, which is conveniently forgotten behind the ongoing race (and gender) hucksterism. Assuming we'd wave a magic wand and fix all disparities across visible populations tomorrow, it will still not fix the fact that huge income disparities exist between individuals. Google engineers and researchers get paid 5 times the median national income or more, and (senior) Google management in the 10x to 1000x range. The vast majority of the population is stuck in dead end precarious jobs, with little social mobility, one medical emergency from bankruptcy.
      - joehl 6y ago
        In causal modelling thou shallst not control by consequences of the causal treatment (the causal treatment here is being born black). Read your Rubin&Rosenbaum.
YeGoblynQueenne 6y ago
>> Gebru is the type of person who defines "bias" as anything that isn't sufficiently positive towards people who look like herself, not the usual definition of a deviation from reality as exists. I haven't seen this "usual definition" of bias as a "deviation from reality" that you give anywhere before. Can you say where it is coming from?
jart 6y ago
The greatest struggle facing the tech elite today is how to use the proletariat's data to train an artificial intelligence that doesn't inherit the proletariat's beliefs.
- thu2111 6y ago
  Pithy and concise, I like it. That's exactly it.
joehl 6y ago
If your data set is biased for the purpose of answering your question, then having more data doesn't make things better. In this case 'big data' just becomes 'big bias'. Go to wikipedia and read on Simpsons Paradox.

kevinventullo 6y ago

If you train an AI on the largest dataset it's possible to obtain then you have, almost by definition, done the most you can to avoid bias of any sort All politics aside, this is not even true for toy ML problems. If I’m trying to do digit recognition and “all the data I can find” is a billion hand-written 0’s and a million hand-written 1’s through 9’s, naively training on that data will yield a model that’s pretty close to guessing 0 every time.

thu2111 6y ago
We're talking about tech firms that have access to the entire internet and use it. Hypothetical examples involving imaginary datasets that nobody would use don't prove anything relevant. And note that my argument is not about whether you actually avoid bias, it's about whether you've done the most you can do to avoid it. If you used all the data you've got, then you've done the most you can, even if for some reason the data you've got isn't any good.
- d_e_solomon 6y ago
  > We're talking about tech firms that have access to the entire internet and use it. I think you're trying to say that a large enough dataset will be free of bias. I don't see how that follows. If I train a model on home mortgage decisions, I will replicate the bias on that currently exists on that dataset - https://news.northwestern.edu/stories/2020/01/racial-discrimination-in-mortgage-market-persistent-over-last-four-decades/ https://news.northwestern.edu/stories/2020/01/racial-discrim... - unless there are conscientious choices to reduce that bias. Researchers in ethics in ML are specifically trying to enable tech companies to do a better job of not replicating bias and justifiably point out where that is occurring. Third, I would argue that applying an ML model to do something faster if it replicates the bias of a previously human decision is even worse. The bias has taken the human element completely out and systematized the bias and made it possible with even less friction.
  - bisectable 6y ago
    Given the current public discourse climate in academia, where 99% of the researchers are politically left or extreme left, I recommend a grain of salt for any research that makes claims racial discrimination. The report you quote has at least a couple of strange holes. The paper itself is behind a paywall, thus beyond mere mortals reach. > For example, in about 10% audits in which a white and an African-American auditor were sent to apply for the same unit after 2005, the white auditor was recommended more units than the African-American auditor. These trends hold in both the large HUD (Housing and Urban Development)-sponsored housing audits, which others have examined with similar findings to us, and in smaller correspondence studies They fail to mention how large is the gap. Is the white auditor recommended 102 vs the black auditor 97, or 150 vs 50, or 200 vs 3? Without such critical information it is hard to form an opinion, unless one already has a large bias in accepting discrimination narratives uncritically. > In the mortgage market the researchers found that racial gaps in loan denial have declined only slightly, and racial gaps in mortgage cost have not declined at all, suggesting persistent racial discrimination. Black and Hispanic borrowers are more likely to be rejected when they apply for a loan and are more likely to receive a high-cost mortgage. They fail to mention the magic words 'when controlled for income'. America has a huge income disparity problem, which is conveniently forgotten behind the ongoing race (and gender) hucksterism. Assuming we'd wave a magic wand and fix all disparities across visible populations tomorrow, it will still not fix the fact that huge income disparities exist between individuals. Google engineers and researchers get paid 5 times the median national income or more, and (senior) Google management in the 10x to 1000x range. The vast majority of the population is stuck in dead end precarious jobs, with little social mobility, one medical emergency from bankruptcy.
    - joehl 6y ago
      In causal modelling thou shallst not control by consequences of the causal treatment (the causal treatment here is being born black). Read your Rubin&Rosenbaum.

YeGoblynQueenne 6y ago

>> Gebru is the type of person who defines "bias" as anything that isn't sufficiently positive towards people who look like herself, not the usual definition of a deviation from reality as exists. I haven't seen this "usual definition" of bias as a "deviation from reality" that you give anywhere before. Can you say where it is coming from?

jart 6y ago

The greatest struggle facing the tech elite today is how to use the proletariat's data to train an artificial intelligence that doesn't inherit the proletariat's beliefs.

thu2111 6y ago
Pithy and concise, I like it. That's exactly it.

joehl 6y ago

If your data set is biased for the purpose of answering your question, then having more data doesn't make things better. In this case 'big data' just becomes 'big bias'. Go to wikipedia and read on Simpsons Paradox.