3 ms·

Honestly I would rather that they not try. I don't understand why a computer tool has to be held to a political standard.

daemoens 4y ago
It's not a political standard though. There is actual diversity in this world. Why wouldn't you want that in your product?
- deleted 4y ago
  [deleted]
- mensetmanusman 4y ago
  Fix the data input side, not the data output side. The data input side is slowly being fixed in real time as the rest of the world gets online and learns these methods.
  - astrange 4y ago
    That wouldn't necessarily fix the issue or do anything. A model isn't a perfect average of all the data you throw into its training set. You have to actually try these things and see if they work.
  - throwaway4aday 4y ago
    In a sane world we would be able to tack on a disclaimer saying "This model was trained on data with a majority representation of caucasian males from Western English speaking countries and so results may skew in that direction" and people would read it and think "well, duh" and "hey let's train some more models with more data from around the world" instead of opining about systemic racism and sexism on the internet.
Jerrrry 4y ago
There are legitimate reasons to reduce externalizations of societies innate biases. A mortgage AI that calculates premiums for the public shouldn't bias against people with historically black names, for example. This problem is harder to tackle because it is difficult to expose and resign the "latent space" that results in these biases; it's difficult to massage the ML algo's to identify and remove the pathways that result in this bias. It's simply much easier to allow the robot to be bias/racist/reflective of "reality" (its training data), and add a filter / band-aid on top; which is what they've attempted. when this is appropriate is the more cultured question; I don't think we should attempt to band-aid these models, but for more socially-critical things, it is definitely appropriate. It's naive on either extreme: do we reject reality, and substitute or own? Or do we call our substitute reality, and hope the zeitgeist follows?
- mh- 4y ago
  > A mortgage AI that calculates premiums for the public shouldn't bias against people with historically black names, for example. That's a great example, thanks. Also, I hope the teams working on that come up with a different solution...
- ceeplusplus 4y ago
  That's great, but by doing so you are also inadvertently favoring, in your example, the people with black names. For example, Chinese people save on average, 50 times more than Americans according to the Fed [1]. That would mean they would generally be overrepresented in loan approvals because they have a better balance sheet. Does that necessarily mean that Americans are discriminated against in the approval process? No. My question to you is: is an algorithm that takes no racial inputs (name, race, address, etc) yet still produces disproportionate results biased or racist? I say no. [1] https://files.stlouisfed.org/files/htdocs/publications/es/08/ES0819.pdf https://files.stlouisfed.org/files/htdocs/publications/es/08...
  - Jerrrry 4y ago
    I would agree that it is not. The government, and many people, have moved the definition and goal posts; so that anything that has the end result of a non-proportional uniformity can be labeled and treated as bias. Ultimately it is a nuanced game; is discriminating against certain clothing or hair-styles racist? Of course. Yet, neither of those are explicitly tied to one's skin color or ethnicity, but are an indirect associative trait because of culture. In America, we have intentionally muddled the waters of demarcation between culture and race, and are starting to see the cost of that.
- deleted 4y ago
  [deleted]
- whywhywhywhy 4y ago
  Wouldn't the whole point of a "Mortgage AI" be to discriminate so the lenders hands could be clean. Not that I agree with that but I don't see why you would build one otherwise, if you wanted discrimination free mortgages wouldn't the whole process by anonymized and minimal personal information rather than the current system of having to hand over every detail of your life.
norwalkbear 4y ago
I agree, the trust is broken now. Im going to skip on any AI that pulls that crap.

daemoens 4y ago

It's not a political standard though. There is actual diversity in this world. Why wouldn't you want that in your product?

deleted 4y ago
[deleted]
mensetmanusman 4y ago
Fix the data input side, not the data output side. The data input side is slowly being fixed in real time as the rest of the world gets online and learns these methods.
- astrange 4y ago
  That wouldn't necessarily fix the issue or do anything. A model isn't a perfect average of all the data you throw into its training set. You have to actually try these things and see if they work.
- throwaway4aday 4y ago
  In a sane world we would be able to tack on a disclaimer saying "This model was trained on data with a majority representation of caucasian males from Western English speaking countries and so results may skew in that direction" and people would read it and think "well, duh" and "hey let's train some more models with more data from around the world" instead of opining about systemic racism and sexism on the internet.

Jerrrry 4y ago

There are legitimate reasons to reduce externalizations of societies innate biases. A mortgage AI that calculates premiums for the public shouldn't bias against people with historically black names, for example. This problem is harder to tackle because it is difficult to expose and resign the "latent space" that results in these biases; it's difficult to massage the ML algo's to identify and remove the pathways that result in this bias. It's simply much easier to allow the robot to be bias/racist/reflective of "reality" (its training data), and add a filter / band-aid on top; which is what they've attempted. when this is appropriate is the more cultured question; I don't think we should attempt to band-aid these models, but for more socially-critical things, it is definitely appropriate. It's naive on either extreme: do we reject reality, and substitute or own? Or do we call our substitute reality, and hope the zeitgeist follows?

mh- 4y ago
> A mortgage AI that calculates premiums for the public shouldn't bias against people with historically black names, for example. That's a great example, thanks. Also, I hope the teams working on that come up with a different solution...
ceeplusplus 4y ago
That's great, but by doing so you are also inadvertently favoring, in your example, the people with black names. For example, Chinese people save on average, 50 times more than Americans according to the Fed [1]. That would mean they would generally be overrepresented in loan approvals because they have a better balance sheet. Does that necessarily mean that Americans are discriminated against in the approval process? No. My question to you is: is an algorithm that takes no racial inputs (name, race, address, etc) yet still produces disproportionate results biased or racist? I say no. [1] https://files.stlouisfed.org/files/htdocs/publications/es/08/ES0819.pdf https://files.stlouisfed.org/files/htdocs/publications/es/08...
- Jerrrry 4y ago
  I would agree that it is not. The government, and many people, have moved the definition and goal posts; so that anything that has the end result of a non-proportional uniformity can be labeled and treated as bias. Ultimately it is a nuanced game; is discriminating against certain clothing or hair-styles racist? Of course. Yet, neither of those are explicitly tied to one's skin color or ethnicity, but are an indirect associative trait because of culture. In America, we have intentionally muddled the waters of demarcation between culture and race, and are starting to see the cost of that.
deleted 4y ago
[deleted]
whywhywhywhy 4y ago
Wouldn't the whole point of a "Mortgage AI" be to discriminate so the lenders hands could be clean. Not that I agree with that but I don't see why you would build one otherwise, if you wanted discrimination free mortgages wouldn't the whole process by anonymized and minimal personal information rather than the current system of having to hand over every detail of your life.

norwalkbear 4y ago

I agree, the trust is broken now. Im going to skip on any AI that pulls that crap.