11 ms·

More accurate isn't always good because creditworthiness is all about stereotyping people. Consider this example: lesbians are statistically more likely to div

More accurate isn't always good because creditworthiness is all about stereotyping people. Consider this example: lesbians are statistically more likely to divorce. The ML models decides that lesbians are a higher mortgage risk and raises their interest rates. It does this indirectly by raising interest rates on married couples with different last names and owners of Subaru vehicles. In another, the model sees that people who follow NBA players on twitter are more likely to default. It then assigns then higher interest rates. Well, it turns out that poor urban black people are far more likely to be interested in the NBA than the average person. Probably because it's one of the only sports you can play in a parking lot. The model decides to raise interest rates and cut lending to most black people. "Perfect" credit worthiness modelling is only going to be beneficial to rich white and asian people. Essentially favoring the stereotypically richest ethnic groups. This is extremely problematic for any sort of equality initiatives. Treating everyone the same regardless of means is definitely socialist. It's the same as removing the pre-existing conditions exclusions for health insurance. It favors the unfortunate at the expense of the fortunate. I'm not going to go into whether that is good or bad for society but I ask you to consider the extreme, historic level wealth inequality in the US where 1% of the population has over half the money.

thegrimmest 6y ago
A lot to unpack here, let me try. Are machine learning algorythms capable of stereotyping people? How accurate does a stereotype have to be before it's a useful predictor? If these models lead to good data, then what's wrong with them? If people who follow the NBA do in fact default on their loans more, shouldn't they pay higher interest? I agree that the healthy should not be compelled to subsidize the costs of healthcare for the unhealthy. Can you tell me why it's ethical to force my compliance? It seems you primarily see things along racial divides, grouping people into black, white, and asian. There are, more nuanced ways to group people, which is exactly what these agorythms are doing. Also, what's wrong with wealth inequality? I'd be much more concerned about absolute quality of life, which has increased dramatically for everyone in the last half century.
- dredmorbius 6y ago
  Algorithms: - May be based on bad, inaccurate, or nonrepresentative data. - May be based on factors intrinsic to individuals which they cannot change. - May mask underlying bias factors (as in the NBA example above.) - May reflect existing biases in behaviours. E.g., arrest, promotion, recruitment, or admissions data used to encode an AI selector, which merely codify biases reflected in the underlying historical data. - May reverse causality, implying A causes B, when in fact B results in A. - Amplify existing inequalities. Starting points in life matter, and an algorithm which simply amplifies the existing inequities of wealth, race, place of birth, religion, health, etc., compound rather than address these issues. - Fail to consider fairness or equity in decisionmaking. This is the underlying fundamental problem in credit-based resource allocation. People need access to resources regardless of their creditworthiness, though there might well be cases in which allocations are modified or managed given behavioural issues. Reinforcing standing biases does not address the underlying inequities.
  - thegrimmest 6y ago
    I would say that accuracy is a greater concern than "fairness" or "equity" since it's quite hard to agree on the meanings of the latter terms (and as pointed out above, these are political, not statistical concerns, and the distinction is meaningful). I would expect that inaccuracy in such models would be iteratively improved. If the models are already more accurate than existing methods, doesn't that make them better? Doesn't a more accurate model in fact ensure that fewer people are lumped into groups they don't belong to, and that more deserving people (ie. people who are likley to repay loans) have access to credit?
    - dredmorbius 6y ago
      All these criteria are value judgements and exquisitely political. I give you Karl Marx: As soon as land becomes private property, the landlord demands a share of almost all the produce which the labourer can either raise, or collect from it. His rent makes the first deduction from the produce of the labour which is employed upon land. ... Civil government, so far as it is instituted for the security of property, is in reality instituted for the defense of the rich against the poor, or of those who have some property against those who have none at all. ... The interest of [businessmen] is always in some respects different from, and even opposite to, that of the public ... The proposal of any new law or regulation of commerce which comes from this order ... ought never to be adopted, till after having been long and carefully examined ... with the most suspicious attention. It comes from an order of men ... who have generally an interest to deceive and even oppress the public. ... The masters, being fewer in number, can combine much more easily; and the law, besides, authorizes, or at least does not prohibit their combinations, while it prohibits those of the workmen. We have no acts of parliament against combining to lower the price of work; but many against combining to raise it. In all such disputes the masters can hold out much longer. ... Masters are always and everywhere in a sort of tacit, but constant and uniform combination, not to raise the wages of labour above their actual rate. To violate this combination is everywhere a most unpopular action, and a sort of reproach to a master among his neighbours and equals. We seldom, indeed, hear of this combination, because it is the usual, and one may say, the natural state of things, which nobody ever hears of. Masters, too, sometimes enter into particular combinations to sink the wages of labour even below this rate. ... A man must always live by his work, and his wages must at least be sufficient to maintain him. They must even upon most occasions be somewhat more; otherwise it would be impossible for him to bring up a family, and the race of such workmen could not last beyond the first generation. ... No society can surely be flourishing and happy, of which the far greater part of the members are poor and miserable. It is but equity, besides, that they who feed, clothe, and lodge the whole body of the people, should have such a share of the produce of their own labour as to be themselves tolerably well fed, clothed, and lodged. ... Wealth, as Mr Hobbes says, is power. ... POLITICAL œconomy, considered as a branch of the science of a statesman or legislator, proposes two distinct objects: first, to provide a plentiful revenue or subsistence for the people, or more properly to enable them to provide such a revenue or subsistence for themselves; and secondly, to supply the state or commonwealth with a revenue sufficient for the public services. It proposes to enrich both the people and the sovereign. The first object of political economy is to provide subsistence for the people. Oh, silly me, that's Adam Smith. So hard to tell them apart. https://en.wikisource.org/wiki/The_Wealth_of_Nations/ https://en.wikisource.org/wiki/The_Wealth_of_Nations/ http://oll.libertyfund.org/titles/smith-an-inquiry-into-the-nature-and-causes-of-the-wealth-of-nations-cannan-ed-vol-1#lf0206-01_head_052 http://oll.libertyfund.org/titles/smith-an-inquiry-into-the-... As well as your politics and economics, your morals and statistical understanding are deeply flawed.
dredmorbius 6y ago
Cathy O'Neil's done some excellent work in this area. "Algorithms are opinions embedded in code." There's a summary of it on the TED Radio Hour here: https://www.npr.org/2018/01/26/580617998/cathy-oneil-do-algorithms-perpetuate-human-bias https://www.npr.org/2018/01/26/580617998/cathy-oneil-do-algo... TED Talk https://embed-ssl.ted.com/talks/cathy_o_neil_the_era_of_blind_faith_in_big_data_must_end.html https://embed-ssl.ted.com/talks/cathy_o_neil_the_era_of_blin... (video) Cathy O’Neil Is Unimpressed by Your AI Bias Removal Tool (A RedTail Q&A) https://redtailmedia.org/2018/10/29/redtail-talks-about-flipping-the-script-on-how-we-value-algorithims-with-the-weapons-of-math-destruction-author/ https://redtailmedia.org/2018/10/29/redtail-talks-about-flip... Weapons of Math Destruction: Cathy O'Neil adds up the damage of algorithms https://www.theguardian.com/books/2016/oct/27/cathy-oneil-weapons-of-math-destruction-algorithms-big-data https://www.theguardian.com/books/2016/oct/27/cathy-oneil-we... Human Insights missing from Big Data: https://www.ted.com/talks/tricia_wang_the_human_insights_missing_from_big_data https://www.ted.com/talks/tricia_wang_the_human_insights_mis... Further references: Weapons of Math Detruction outlines dangers of relying on data analytics: https://www.npr.org/2016/09/12/493654950/weapons-of-math-destruction-outlines-dangers-of-relying-on-data-analytics https://www.npr.org/2016/09/12/493654950/weapons-of-math-des... Can Big Data Really Help Screen Immigrants? https://www.npr.org/2017/12/15/571199955/dhs-wants-to-build-a-computer-system-to-help-determine-who-gets-to-visit-the-u-s https://www.npr.org/2017/12/15/571199955/dhs-wants-to-build-...

thegrimmest 6y ago

A lot to unpack here, let me try. Are machine learning algorythms capable of stereotyping people? How accurate does a stereotype have to be before it's a useful predictor? If these models lead to good data, then what's wrong with them? If people who follow the NBA do in fact default on their loans more, shouldn't they pay higher interest? I agree that the healthy should not be compelled to subsidize the costs of healthcare for the unhealthy. Can you tell me why it's ethical to force my compliance? It seems you primarily see things along racial divides, grouping people into black, white, and asian. There are, more nuanced ways to group people, which is exactly what these agorythms are doing. Also, what's wrong with wealth inequality? I'd be much more concerned about absolute quality of life, which has increased dramatically for everyone in the last half century.

dredmorbius 6y ago
Algorithms: - May be based on bad, inaccurate, or nonrepresentative data. - May be based on factors intrinsic to individuals which they cannot change. - May mask underlying bias factors (as in the NBA example above.) - May reflect existing biases in behaviours. E.g., arrest, promotion, recruitment, or admissions data used to encode an AI selector, which merely codify biases reflected in the underlying historical data. - May reverse causality, implying A causes B, when in fact B results in A. - Amplify existing inequalities. Starting points in life matter, and an algorithm which simply amplifies the existing inequities of wealth, race, place of birth, religion, health, etc., compound rather than address these issues. - Fail to consider fairness or equity in decisionmaking. This is the underlying fundamental problem in credit-based resource allocation. People need access to resources regardless of their creditworthiness, though there might well be cases in which allocations are modified or managed given behavioural issues. Reinforcing standing biases does not address the underlying inequities.
- thegrimmest 6y ago
  I would say that accuracy is a greater concern than "fairness" or "equity" since it's quite hard to agree on the meanings of the latter terms (and as pointed out above, these are political, not statistical concerns, and the distinction is meaningful). I would expect that inaccuracy in such models would be iteratively improved. If the models are already more accurate than existing methods, doesn't that make them better? Doesn't a more accurate model in fact ensure that fewer people are lumped into groups they don't belong to, and that more deserving people (ie. people who are likley to repay loans) have access to credit?
  - dredmorbius 6y ago
    All these criteria are value judgements and exquisitely political. I give you Karl Marx: As soon as land becomes private property, the landlord demands a share of almost all the produce which the labourer can either raise, or collect from it. His rent makes the first deduction from the produce of the labour which is employed upon land. ... Civil government, so far as it is instituted for the security of property, is in reality instituted for the defense of the rich against the poor, or of those who have some property against those who have none at all. ... The interest of [businessmen] is always in some respects different from, and even opposite to, that of the public ... The proposal of any new law or regulation of commerce which comes from this order ... ought never to be adopted, till after having been long and carefully examined ... with the most suspicious attention. It comes from an order of men ... who have generally an interest to deceive and even oppress the public. ... The masters, being fewer in number, can combine much more easily; and the law, besides, authorizes, or at least does not prohibit their combinations, while it prohibits those of the workmen. We have no acts of parliament against combining to lower the price of work; but many against combining to raise it. In all such disputes the masters can hold out much longer. ... Masters are always and everywhere in a sort of tacit, but constant and uniform combination, not to raise the wages of labour above their actual rate. To violate this combination is everywhere a most unpopular action, and a sort of reproach to a master among his neighbours and equals. We seldom, indeed, hear of this combination, because it is the usual, and one may say, the natural state of things, which nobody ever hears of. Masters, too, sometimes enter into particular combinations to sink the wages of labour even below this rate. ... A man must always live by his work, and his wages must at least be sufficient to maintain him. They must even upon most occasions be somewhat more; otherwise it would be impossible for him to bring up a family, and the race of such workmen could not last beyond the first generation. ... No society can surely be flourishing and happy, of which the far greater part of the members are poor and miserable. It is but equity, besides, that they who feed, clothe, and lodge the whole body of the people, should have such a share of the produce of their own labour as to be themselves tolerably well fed, clothed, and lodged. ... Wealth, as Mr Hobbes says, is power. ... POLITICAL œconomy, considered as a branch of the science of a statesman or legislator, proposes two distinct objects: first, to provide a plentiful revenue or subsistence for the people, or more properly to enable them to provide such a revenue or subsistence for themselves; and secondly, to supply the state or commonwealth with a revenue sufficient for the public services. It proposes to enrich both the people and the sovereign. The first object of political economy is to provide subsistence for the people. Oh, silly me, that's Adam Smith. So hard to tell them apart. https://en.wikisource.org/wiki/The_Wealth_of_Nations/ https://en.wikisource.org/wiki/The_Wealth_of_Nations/ http://oll.libertyfund.org/titles/smith-an-inquiry-into-the-nature-and-causes-of-the-wealth-of-nations-cannan-ed-vol-1#lf0206-01_head_052 http://oll.libertyfund.org/titles/smith-an-inquiry-into-the-... As well as your politics and economics, your morals and statistical understanding are deeply flawed.

dredmorbius 6y ago

Cathy O'Neil's done some excellent work in this area. "Algorithms are opinions embedded in code." There's a summary of it on the TED Radio Hour here: https://www.npr.org/2018/01/26/580617998/cathy-oneil-do-algorithms-perpetuate-human-bias https://www.npr.org/2018/01/26/580617998/cathy-oneil-do-algo... TED Talk https://embed-ssl.ted.com/talks/cathy_o_neil_the_era_of_blind_faith_in_big_data_must_end.html https://embed-ssl.ted.com/talks/cathy_o_neil_the_era_of_blin... (video) Cathy O’Neil Is Unimpressed by Your AI Bias Removal Tool (A RedTail Q&A) https://redtailmedia.org/2018/10/29/redtail-talks-about-flipping-the-script-on-how-we-value-algorithims-with-the-weapons-of-math-destruction-author/ https://redtailmedia.org/2018/10/29/redtail-talks-about-flip... Weapons of Math Destruction: Cathy O'Neil adds up the damage of algorithms https://www.theguardian.com/books/2016/oct/27/cathy-oneil-weapons-of-math-destruction-algorithms-big-data https://www.theguardian.com/books/2016/oct/27/cathy-oneil-we... Human Insights missing from Big Data: https://www.ted.com/talks/tricia_wang_the_human_insights_missing_from_big_data https://www.ted.com/talks/tricia_wang_the_human_insights_mis... Further references: Weapons of Math Detruction outlines dangers of relying on data analytics: https://www.npr.org/2016/09/12/493654950/weapons-of-math-destruction-outlines-dangers-of-relying-on-data-analytics https://www.npr.org/2016/09/12/493654950/weapons-of-math-des... Can Big Data Really Help Screen Immigrants? https://www.npr.org/2017/12/15/571199955/dhs-wants-to-build-a-computer-system-to-help-determine-who-gets-to-visit-the-u-s https://www.npr.org/2017/12/15/571199955/dhs-wants-to-build-...