7 ms·

I am not a native speaker and I don't think I understand the quote. What should "money laundering for bias mean"?

bayindirh 6y ago
Early machine learning / deep learning algorithms have shown bias towards some results or behavior due to problems in training dataset. These problems are tried to be remedied with better training sets but I'm not aware whether it's completely eliminated. "Money laundering for bias" in this context can be translated as "Finding a plausible excuse/cover for a specific bias (your computer/model says the same!)".
bombela 6y ago
Money laundering hides where the money comes from. Bias are subjective assumption you have of the world and people. So this quote is saying that machine learning hides the source of bias.
cpeterso 6y ago
Before machine learning, a system (such as resume screening) that was biased was the fault of the rules and people implementing them. With machine learning, people can blame the “The Algorithm”, a numerical black box that can’t be inspected or interviewed. The system and people now claim they are blameless because “the computer can’t be biased“, even though it was trained on data produced by the previous biased system.
MereInterest 6y ago
To add to the other answers, machine learning based on biased datasets results in biased models. For example, Word2Vec can be used to make analogies. Having it determine "Man is to woman as king is to ___." results in "queen", which is reasonable. On the other hand, determining "Man is to computer programmer as woman is to ___." results in "homemaker". The algorithm wasn't deliberately designed to have sexist bias in it, but there was implicit bias in the dataset from which it learned. https://www.technologyreview.com/2016/07/27/158634/how-vector-space-mathematics-reveals-the-hidden-sexism-in-language/ https://www.technologyreview.com/2016/07/27/158634/how-vecto... As another example, predictive policing tries to place police in places with higher crime rates. Those crime rates are determined by looking at past history of police reports and arrests. That past history has human bias already in it, with disproportionately higher arrest rates in places with racial minorities. The effect of the predictive policing is to justify overpolicing of minorities. https://en.wikipedia.org/wiki/Predictive_policing#Criticisms https://en.wikipedia.org/wiki/Predictive_policing#Criticisms
- mattkrause 6y ago
  The analogy thing didn’t entirely hold up: the original demonstration was constrained not to return the same word in the prompt, so “man is to woman as doctor is to ____” had to return something that’s close to, but not the same as “doctor” in the embedding space. Hence, it returns nurse. Ironically, this makes the original point nearly as well: we need to evaluate the hell out of machine learning systems to make sure that they’re doing what we think they are and that they’re not keying off something else instead, especially something biased. To date, the field has been...not great about this.
  - organsnyder 6y ago
    I wonder what it would return for "Woman is to doctor as man is to ___". I'd bet money on the result not being "nurse".
    - sqrt17 6y ago
      It highly depends on the corpus and you can check for yourself. I'm being lazy, but the results for Gutenberg books you can check online at http://labs.statsbiblioteket.dk/dsc/ http://labs.statsbiblioteket.dk/dsc/ - man is to woman as doctor is to reprovingly (nurse is the first noun, on position 4) - woman is to man as doctor is to snodgrass (after a couple nonsense/rare words) The most important thing that teaches us is that big corpora (bigger than PG) are essential for this method.
    - mattkrause 6y ago
      You can monkey around with it here: http://bionlp-www.utu.fi/wv_demo/ http://bionlp-www.utu.fi/wv_demo/ (choose the English Google News model, but this may not be exactly the same set/model as the original report). Man is to Woman as Doctor is to ___ gives 1) gynecologist 2) nurse 3) doctors 4) physician 5) pediatrician Woman is to Man as Doctor is to ___ gives: 1) physician 2) doctors 3) surgeon 4) dentist 5) cardiologist These are just generally near "Doctor" though: the ten nearest terms are physician, doctors, gynecologist, surgeon, dentist, pediatrician, pharmacist, neurologist, cardiologist, and nurse. Some gender differences may persist (nurse is #2 for `woman`, but #68 for `man`, but it's also near `woman` generally and you could imagine it gets a bit of a boost from the verb ("to feed a baby") being attached exclusively to women too. Anyway, my point is not that there's no bias (there certainly can be--seed GTP-3 with a prompt about Muslims) but that one should be wary of thinking they know what the model is doing.
donmcronald 6y ago
Since computers are unbiased, the idea is that machine learning is also unbiased. However, that's not true and, if anything, it actually re-enforces a lot of bias. I've never done any, but my understanding is that machine learning is just correlation. It's good at figuring out "what", but not "why". Consider training an algorithm to recognize horses by feeding it millions of pictures of horses. Eventually, the algorithm "learns" what a horse is, but it's definition of a horse is based on the inputs it was given by a human. So now, consider the scenario where the millions of pictures of horses were all brown. If you give the algorithm a picture of a white horse, it'll tell you it's not a horse. If you give it a picture of a brown donkey, it might think it's a horse because it's learned to put too much emphasis on the color brown. If that algorithm becomes relied on to define a horse, "the system" will insist there are no white horses even though you can walk outside and see them plain as day. Now, apply the same kind of idea and feed an algorithm mugshots of all criminals. It's going to develop the same bias and tell you that a black person is more likely to be a criminal than a white person. There's no nuance. The inputs used to train the AI were tainted by decades of systematic discrimination, but the AI doesn't know that. Of course you could try to take that input bias into account, but the whole sales pitch of machine learning is that you feed it tons of data and it gives you an objective result. As far as I know, no one is trying to quantify, and correct, the biases in the inputs. The phrase "money laundering for bias" means the machine learning algorithms are used to re-enforce incorrect opinions and assumptions because it gives the excuse that an "objective" computer used cold hard data to draw the same conclusion. Machine learning is one of the scariest parts of tech right now because it's the equivalent of an extremely stupid person that only understands correlation and not causality and the systems being built are going to be making a lot of decisions at scale.
- vngzs 6y ago
  I share your concerns. I am of the opinion that we shouldn't be using machine learning algorithms to make decisions of consequence in people's lives without filtering those decisions through human operators. You can use machine learning to assist these human operators and empower them to do much more work than without the technology. But at the end of the day, you still have someone to hold responsible, and you still have checks on the ML doing something profoundly unethical.
derbOac 6y ago
Money laundering is a crime in which one takes money or goods taken through criminal activities (e.g., theft) and sends it through some process, usually a cover, like a business or something, to make it appear legitimate. More generally, it refers to the process by which money or resources obtained through ethically questionable means is passed through layers of other activities so as to obscure its original source. https://en.wikipedia.org/wiki/Money_laundering https://en.wikipedia.org/wiki/Money_laundering So the idea is, you have some bias in some process (racial, religious, whatever). You set up some algorithm that relies on the bias to make predictions or classifications. Now you can say it's not you that's biased, it's just the algorithm. The algorithm is some process by which your bias is "made legitimate".
- bisectable 6y ago
  The solution to this conundrum is to aggressively encode explicit biases into the algorithm. The only requirement is that the explicit biases favor certain demographics that are politically deemed more equal than others.
  - dang 6y ago
    We've banned this account and a bunch of related accounts for abusing HN and breaking the site guidelines. If you do it again we will ban your main account as well. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- reader_mode 6y ago
  > You set up some algorithm that relies on the bias to make predictions or classifications. Now you can say it's not you that's biased, it's just the algorithm. It's not that you set up the algorithm to rely on the bias - ML trains on data produced by a biased system and ends up building a model containing the same biases - you don't need to "set-up" anything - if you're fine with the existing biases you can "launder" them through ML.
  - derbOac 6y ago
xwdv 6y ago
Concretely, imagine you have a bank, and you decide you don’t want to give loans to black people. You employ a machine learning algorithm, feed it some data that shows black people in poverty have high risk of default on loans and avoid giving it data that shows otherwise. Train your neural network hard. Now when people ask about why you won’t don’t give out many loans to blacks, throw your hands up and say “I would but our advanced machine learning algorithms say these people are high risk, we’re not racist we’re just following the results.”
ianhorn 6y ago
“Money laundering” is taking criminal money (maybe marked currency or something) and trading them out for “clean” currency that can’t be tracked. Imagine a dataset about healthcare outcomes. You want to know whether airlifting a patient is good or bad, so you train a machine learning model to predict mortality given airlifting a patient versus not airlifting them. Turns out a lot more of the airlifted patients died, and the model picks up on that, deciding that airlifting is dangerous. Obviously, we’re airlifting the patients who are in the most dire circumstances, and that’s why they die more, but maybe the model doesn’t have the context/circumstance variables to see that, or maybe it’s just regularized and thinks those context variables are noise. So the bias of “airlift = bad” gets stuck in the model, but then people defend it as mathematically precise so it can’t be biased like a person can. They say that the model just reflects reality, pretending the model or data is wrong is blindly rejecting reality for the sake of political correctness. It’s worse with really human stuff like recidivism because the context variables that might be useful (like the patient’s dire circumstances) tend to be very human and complex and they’re unlikely to be captured in a simple form. Even if they were, interpreting them might require human level AI. By analogy, it is frequently impossible (or just too hard to be worth it) for current technology’s ML to tell from the data we feed it that the patients being airlifted were most near death to begin with. So you end up with an algorithm biased against airlifting patients getting defended as mathematically bulletproof.

Early machine learning / deep learning algorithms have shown bias towards some results or behavior due to problems in training dataset. These problems are tried to be remedied with better training sets but I'm not aware whether it's completely eliminated. "Money laundering for bias" in this context can be translated as "Finding a plausible excuse/cover for a specific bias (your computer/model says the same!)".

bombela 6y ago

Money laundering hides where the money comes from. Bias are subjective assumption you have of the world and people. So this quote is saying that machine learning hides the source of bias.

cpeterso 6y ago

Before machine learning, a system (such as resume screening) that was biased was the fault of the rules and people implementing them. With machine learning, people can blame the “The Algorithm”, a numerical black box that can’t be inspected or interviewed. The system and people now claim they are blameless because “the computer can’t be biased“, even though it was trained on data produced by the previous biased system.

MereInterest 6y ago

To add to the other answers, machine learning based on biased datasets results in biased models. For example, Word2Vec can be used to make analogies. Having it determine "Man is to woman as king is to ___." results in "queen", which is reasonable. On the other hand, determining "Man is to computer programmer as woman is to ___." results in "homemaker". The algorithm wasn't deliberately designed to have sexist bias in it, but there was implicit bias in the dataset from which it learned. https://www.technologyreview.com/2016/07/27/158634/how-vector-space-mathematics-reveals-the-hidden-sexism-in-language/ https://www.technologyreview.com/2016/07/27/158634/how-vecto... As another example, predictive policing tries to place police in places with higher crime rates. Those crime rates are determined by looking at past history of police reports and arrests. That past history has human bias already in it, with disproportionately higher arrest rates in places with racial minorities. The effect of the predictive policing is to justify overpolicing of minorities. https://en.wikipedia.org/wiki/Predictive_policing#Criticisms https://en.wikipedia.org/wiki/Predictive_policing#Criticisms

mattkrause 6y ago
The analogy thing didn’t entirely hold up: the original demonstration was constrained not to return the same word in the prompt, so “man is to woman as doctor is to ____” had to return something that’s close to, but not the same as “doctor” in the embedding space. Hence, it returns nurse. Ironically, this makes the original point nearly as well: we need to evaluate the hell out of machine learning systems to make sure that they’re doing what we think they are and that they’re not keying off something else instead, especially something biased. To date, the field has been...not great about this.
- organsnyder 6y ago
  I wonder what it would return for "Woman is to doctor as man is to ___". I'd bet money on the result not being "nurse".
  - sqrt17 6y ago
    It highly depends on the corpus and you can check for yourself. I'm being lazy, but the results for Gutenberg books you can check online at http://labs.statsbiblioteket.dk/dsc/ http://labs.statsbiblioteket.dk/dsc/ - man is to woman as doctor is to reprovingly (nurse is the first noun, on position 4) - woman is to man as doctor is to snodgrass (after a couple nonsense/rare words) The most important thing that teaches us is that big corpora (bigger than PG) are essential for this method.
  - mattkrause 6y ago
    You can monkey around with it here: http://bionlp-www.utu.fi/wv_demo/ http://bionlp-www.utu.fi/wv_demo/ (choose the English Google News model, but this may not be exactly the same set/model as the original report). Man is to Woman as Doctor is to ___ gives 1) gynecologist 2) nurse 3) doctors 4) physician 5) pediatrician Woman is to Man as Doctor is to ___ gives: 1) physician 2) doctors 3) surgeon 4) dentist 5) cardiologist These are just generally near "Doctor" though: the ten nearest terms are physician, doctors, gynecologist, surgeon, dentist, pediatrician, pharmacist, neurologist, cardiologist, and nurse. Some gender differences may persist (nurse is #2 for `woman`, but #68 for `man`, but it's also near `woman` generally and you could imagine it gets a bit of a boost from the verb ("to feed a baby") being attached exclusively to women too. Anyway, my point is not that there's no bias (there certainly can be--seed GTP-3 with a prompt about Muslims) but that one should be wary of thinking they know what the model is doing.

donmcronald 6y ago

Since computers are unbiased, the idea is that machine learning is also unbiased. However, that's not true and, if anything, it actually re-enforces a lot of bias. I've never done any, but my understanding is that machine learning is just correlation. It's good at figuring out "what", but not "why". Consider training an algorithm to recognize horses by feeding it millions of pictures of horses. Eventually, the algorithm "learns" what a horse is, but it's definition of a horse is based on the inputs it was given by a human. So now, consider the scenario where the millions of pictures of horses were all brown. If you give the algorithm a picture of a white horse, it'll tell you it's not a horse. If you give it a picture of a brown donkey, it might think it's a horse because it's learned to put too much emphasis on the color brown. If that algorithm becomes relied on to define a horse, "the system" will insist there are no white horses even though you can walk outside and see them plain as day. Now, apply the same kind of idea and feed an algorithm mugshots of all criminals. It's going to develop the same bias and tell you that a black person is more likely to be a criminal than a white person. There's no nuance. The inputs used to train the AI were tainted by decades of systematic discrimination, but the AI doesn't know that. Of course you could try to take that input bias into account, but the whole sales pitch of machine learning is that you feed it tons of data and it gives you an objective result. As far as I know, no one is trying to quantify, and correct, the biases in the inputs. The phrase "money laundering for bias" means the machine learning algorithms are used to re-enforce incorrect opinions and assumptions because it gives the excuse that an "objective" computer used cold hard data to draw the same conclusion. Machine learning is one of the scariest parts of tech right now because it's the equivalent of an extremely stupid person that only understands correlation and not causality and the systems being built are going to be making a lot of decisions at scale.

vngzs 6y ago
I share your concerns. I am of the opinion that we shouldn't be using machine learning algorithms to make decisions of consequence in people's lives without filtering those decisions through human operators. You can use machine learning to assist these human operators and empower them to do much more work than without the technology. But at the end of the day, you still have someone to hold responsible, and you still have checks on the ML doing something profoundly unethical.

derbOac 6y ago

Money laundering is a crime in which one takes money or goods taken through criminal activities (e.g., theft) and sends it through some process, usually a cover, like a business or something, to make it appear legitimate. More generally, it refers to the process by which money or resources obtained through ethically questionable means is passed through layers of other activities so as to obscure its original source. https://en.wikipedia.org/wiki/Money_laundering https://en.wikipedia.org/wiki/Money_laundering So the idea is, you have some bias in some process (racial, religious, whatever). You set up some algorithm that relies on the bias to make predictions or classifications. Now you can say it's not you that's biased, it's just the algorithm. The algorithm is some process by which your bias is "made legitimate".

bisectable 6y ago
The solution to this conundrum is to aggressively encode explicit biases into the algorithm. The only requirement is that the explicit biases favor certain demographics that are politically deemed more equal than others.
- dang 6y ago
  We've banned this account and a bunch of related accounts for abusing HN and breaking the site guidelines. If you do it again we will ban your main account as well. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
reader_mode 6y ago
> You set up some algorithm that relies on the bias to make predictions or classifications. Now you can say it's not you that's biased, it's just the algorithm. It's not that you set up the algorithm to rely on the bias - ML trains on data produced by a biased system and ends up building a model containing the same biases - you don't need to "set-up" anything - if you're fine with the existing biases you can "launder" them through ML.
- derbOac 6y ago

xwdv 6y ago

Concretely, imagine you have a bank, and you decide you don’t want to give loans to black people. You employ a machine learning algorithm, feed it some data that shows black people in poverty have high risk of default on loans and avoid giving it data that shows otherwise. Train your neural network hard. Now when people ask about why you won’t don’t give out many loans to blacks, throw your hands up and say “I would but our advanced machine learning algorithms say these people are high risk, we’re not racist we’re just following the results.”

ianhorn 6y ago

“Money laundering” is taking criminal money (maybe marked currency or something) and trading them out for “clean” currency that can’t be tracked. Imagine a dataset about healthcare outcomes. You want to know whether airlifting a patient is good or bad, so you train a machine learning model to predict mortality given airlifting a patient versus not airlifting them. Turns out a lot more of the airlifted patients died, and the model picks up on that, deciding that airlifting is dangerous. Obviously, we’re airlifting the patients who are in the most dire circumstances, and that’s why they die more, but maybe the model doesn’t have the context/circumstance variables to see that, or maybe it’s just regularized and thinks those context variables are noise. So the bias of “airlift = bad” gets stuck in the model, but then people defend it as mathematically precise so it can’t be biased like a person can. They say that the model just reflects reality, pretending the model or data is wrong is blindly rejecting reality for the sake of political correctness. It’s worse with really human stuff like recidivism because the context variables that might be useful (like the patient’s dire circumstances) tend to be very human and complex and they’re unlikely to be captured in a simple form. Even if they were, interpreting them might require human level AI. By analogy, it is frequently impossible (or just too hard to be worth it) for current technology’s ML to tell from the data we feed it that the patients being airlifted were most near death to begin with. So you end up with an algorithm biased against airlifting patients getting defended as mathematically bulletproof.