12 ms·

it's a hard problem. at least they tried.

by mysore 4y ago

protonbob 4y ago
Honestly I would rather that they not try. I don't understand why a computer tool has to be held to a political standard.
- daemoens 4y ago
  It's not a political standard though. There is actual diversity in this world. Why wouldn't you want that in your product?
  - deleted 4y ago
    [deleted]
  - mensetmanusman 4y ago
    Fix the data input side, not the data output side. The data input side is slowly being fixed in real time as the rest of the world gets online and learns these methods.
    - astrange 4y ago
      That wouldn't necessarily fix the issue or do anything. A model isn't a perfect average of all the data you throw into its training set. You have to actually try these things and see if they work.
    - throwaway4aday 4y ago
      In a sane world we would be able to tack on a disclaimer saying "This model was trained on data with a majority representation of caucasian males from Western English speaking countries and so results may skew in that direction" and people would read it and think "well, duh" and "hey let's train some more models with more data from around the world" instead of opining about systemic racism and sexism on the internet.
- Jerrrry 4y ago
  There are legitimate reasons to reduce externalizations of societies innate biases. A mortgage AI that calculates premiums for the public shouldn't bias against people with historically black names, for example. This problem is harder to tackle because it is difficult to expose and resign the "latent space" that results in these biases; it's difficult to massage the ML algo's to identify and remove the pathways that result in this bias. It's simply much easier to allow the robot to be bias/racist/reflective of "reality" (its training data), and add a filter / band-aid on top; which is what they've attempted. when this is appropriate is the more cultured question; I don't think we should attempt to band-aid these models, but for more socially-critical things, it is definitely appropriate. It's naive on either extreme: do we reject reality, and substitute or own? Or do we call our substitute reality, and hope the zeitgeist follows?
  - mh- 4y ago
    > A mortgage AI that calculates premiums for the public shouldn't bias against people with historically black names, for example. That's a great example, thanks. Also, I hope the teams working on that come up with a different solution...
  - ceeplusplus 4y ago
    That's great, but by doing so you are also inadvertently favoring, in your example, the people with black names. For example, Chinese people save on average, 50 times more than Americans according to the Fed [1]. That would mean they would generally be overrepresented in loan approvals because they have a better balance sheet. Does that necessarily mean that Americans are discriminated against in the approval process? No. My question to you is: is an algorithm that takes no racial inputs (name, race, address, etc) yet still produces disproportionate results biased or racist? I say no. [1] https://files.stlouisfed.org/files/htdocs/publications/es/08/ES0819.pdf https://files.stlouisfed.org/files/htdocs/publications/es/08...
    - Jerrrry 4y ago
      I would agree that it is not. The government, and many people, have moved the definition and goal posts; so that anything that has the end result of a non-proportional uniformity can be labeled and treated as bias. Ultimately it is a nuanced game; is discriminating against certain clothing or hair-styles racist? Of course. Yet, neither of those are explicitly tied to one's skin color or ethnicity, but are an indirect associative trait because of culture. In America, we have intentionally muddled the waters of demarcation between culture and race, and are starting to see the cost of that.
- norwalkbear 4y ago
  I agree, the trust is broken now. Im going to skip on any AI that pulls that crap.
Jerrrry 4y ago
It's not a "problem," it's an unwanted shard of reality piercing through an ideological guise.
- ketzo 4y ago
  serious question: in what way is that not a “problem?”
  - bobcostas55 4y ago
    Well, it's a problem for the ideology.
  - TheFreim 4y ago
    It's not a problem in a few ways, let me know what you think (feel free to ask for clarification). 1. The training data would've been the best way to get organic results, the input is where it'd be necessary to have representative samples of populations. 2. If the reason the model needs to be manipulated to include more "diversity" is that there wasn't enough "diversity" in the training set then its likely the results will be lower quality 3. People should be free to manipulate the results how they wish, a base model without arbitrary manipulations of "diversity" would be the best starting point to allow users to get the appropriate results 4. A "diverse" group of people depends on a variety of different circumstances, if their method of increasing it is as naive as some of the are claiming this could result in absurdities when generating historical images or images relating to specific locations/cultures where things will be LESS representative
- gnulinux 4y ago
  How's it NOT a problem? If I'm trying to produce "stock people images", and if it only gives me white men, it's clearly broken because when I ask for "people" I'm actually asking for "people". I'm having difficulty understanding how it can be considered to be working as intended, when it literally doesn't. Clearly, the software has substantial bias that gets in way of it accomplishing its task. If I want to produce "animal images" but it only produces images of black cats, do you think there is any question whether it's a problem or not?
kache_ 4y ago
While their heart is in the right place, I'd like to challenge the idea that certain groups are so fragile that they don't understand that historically, there are more pictures of certain groups doing certain things. It's a hard problem for sure. But remember, the bias ends with the user using the tool. If I want a black scientist, I can just say "black scientist". Let me be mindful of the bias, until we have a generally intelligent system that can actually do it. I'm generally intelligent too, you know.
- UmYeahNo 4y ago
  >But remember, the bias ends with the user using the tool. If I want a black scientist, I can just say "black scientist". That is a really, really, narrow viewpoint. I think what people would prefer is that if you query "Scientist" that the images returned are as likely to be any combination of gender and race. It's not that a group is "fragile", it's that they have to specify race and gender at all, when that specificity is not part of the intention. It seems that they recognize that querying "Scientist" will predominantly skew a certain way, and they're trying in some way to unskew. Or, perhaps, you'd rather that the query be really, really specific? like: "an adult human of any gender and any race and skin color dressed in a laboratory coat...", but I would much rather just say "a scientist" and have the system recognize that anyone can be a scientist. And then if I need to be specific, then I would be happy to say "a black-haired scientist"
  - kache_ 4y ago
    This is a problem with generative models across the board. It's important that we don't skew our perceptions by GAN outputs as a society, so it's definitely good that we're thinking about it. I just wish that we had a solution that solved across the class of problems "Generative AI feeds into itself and society (which is in a way, a generative AI), creating a positive feedback loop that eventually leads to a cultural freeze" It's way bigger than just this narrow race issue the current zeitgeist is concerned about. But I agree, maybe I should skew to being optimistic that at least we're trying
  - numpad0 4y ago
    Kind of funny that NN tech is supposed to construct some upper dimensional understanding, yet realistically cannot be expected to be able to generate gender and race indeterminate portrayal of a scientist.
deleted 4y ago
[deleted]

it's a hard problem. at least they tried.

by mysore 4y ago

it's a hard problem. at least they tried.

Honestly I would rather that they not try. I don't understand why a computer tool has to be held to a political standard.

daemoens 4y ago
It's not a political standard though. There is actual diversity in this world. Why wouldn't you want that in your product?
- deleted 4y ago
  [deleted]
- mensetmanusman 4y ago
  Fix the data input side, not the data output side. The data input side is slowly being fixed in real time as the rest of the world gets online and learns these methods.
  - astrange 4y ago
    That wouldn't necessarily fix the issue or do anything. A model isn't a perfect average of all the data you throw into its training set. You have to actually try these things and see if they work.
  - throwaway4aday 4y ago
    In a sane world we would be able to tack on a disclaimer saying "This model was trained on data with a majority representation of caucasian males from Western English speaking countries and so results may skew in that direction" and people would read it and think "well, duh" and "hey let's train some more models with more data from around the world" instead of opining about systemic racism and sexism on the internet.
Jerrrry 4y ago
There are legitimate reasons to reduce externalizations of societies innate biases. A mortgage AI that calculates premiums for the public shouldn't bias against people with historically black names, for example. This problem is harder to tackle because it is difficult to expose and resign the "latent space" that results in these biases; it's difficult to massage the ML algo's to identify and remove the pathways that result in this bias. It's simply much easier to allow the robot to be bias/racist/reflective of "reality" (its training data), and add a filter / band-aid on top; which is what they've attempted. when this is appropriate is the more cultured question; I don't think we should attempt to band-aid these models, but for more socially-critical things, it is definitely appropriate. It's naive on either extreme: do we reject reality, and substitute or own? Or do we call our substitute reality, and hope the zeitgeist follows?
- mh- 4y ago
  > A mortgage AI that calculates premiums for the public shouldn't bias against people with historically black names, for example. That's a great example, thanks. Also, I hope the teams working on that come up with a different solution...
- ceeplusplus 4y ago
  That's great, but by doing so you are also inadvertently favoring, in your example, the people with black names. For example, Chinese people save on average, 50 times more than Americans according to the Fed [1]. That would mean they would generally be overrepresented in loan approvals because they have a better balance sheet. Does that necessarily mean that Americans are discriminated against in the approval process? No. My question to you is: is an algorithm that takes no racial inputs (name, race, address, etc) yet still produces disproportionate results biased or racist? I say no. [1] https://files.stlouisfed.org/files/htdocs/publications/es/08/ES0819.pdf https://files.stlouisfed.org/files/htdocs/publications/es/08...
  - Jerrrry 4y ago
    I would agree that it is not. The government, and many people, have moved the definition and goal posts; so that anything that has the end result of a non-proportional uniformity can be labeled and treated as bias. Ultimately it is a nuanced game; is discriminating against certain clothing or hair-styles racist? Of course. Yet, neither of those are explicitly tied to one's skin color or ethnicity, but are an indirect associative trait because of culture. In America, we have intentionally muddled the waters of demarcation between culture and race, and are starting to see the cost of that.
norwalkbear 4y ago
I agree, the trust is broken now. Im going to skip on any AI that pulls that crap.

Jerrrry 4y ago

It's not a "problem," it's an unwanted shard of reality piercing through an ideological guise.

ketzo 4y ago
serious question: in what way is that not a “problem?”
- bobcostas55 4y ago
  Well, it's a problem for the ideology.
- TheFreim 4y ago
  It's not a problem in a few ways, let me know what you think (feel free to ask for clarification). 1. The training data would've been the best way to get organic results, the input is where it'd be necessary to have representative samples of populations. 2. If the reason the model needs to be manipulated to include more "diversity" is that there wasn't enough "diversity" in the training set then its likely the results will be lower quality 3. People should be free to manipulate the results how they wish, a base model without arbitrary manipulations of "diversity" would be the best starting point to allow users to get the appropriate results 4. A "diverse" group of people depends on a variety of different circumstances, if their method of increasing it is as naive as some of the are claiming this could result in absurdities when generating historical images or images relating to specific locations/cultures where things will be LESS representative
gnulinux 4y ago
How's it NOT a problem? If I'm trying to produce "stock people images", and if it only gives me white men, it's clearly broken because when I ask for "people" I'm actually asking for "people". I'm having difficulty understanding how it can be considered to be working as intended, when it literally doesn't. Clearly, the software has substantial bias that gets in way of it accomplishing its task. If I want to produce "animal images" but it only produces images of black cats, do you think there is any question whether it's a problem or not?

kache_ 4y ago

While their heart is in the right place, I'd like to challenge the idea that certain groups are so fragile that they don't understand that historically, there are more pictures of certain groups doing certain things. It's a hard problem for sure. But remember, the bias ends with the user using the tool. If I want a black scientist, I can just say "black scientist". Let me be mindful of the bias, until we have a generally intelligent system that can actually do it. I'm generally intelligent too, you know.

UmYeahNo 4y ago
>But remember, the bias ends with the user using the tool. If I want a black scientist, I can just say "black scientist". That is a really, really, narrow viewpoint. I think what people would prefer is that if you query "Scientist" that the images returned are as likely to be any combination of gender and race. It's not that a group is "fragile", it's that they have to specify race and gender at all, when that specificity is not part of the intention. It seems that they recognize that querying "Scientist" will predominantly skew a certain way, and they're trying in some way to unskew. Or, perhaps, you'd rather that the query be really, really specific? like: "an adult human of any gender and any race and skin color dressed in a laboratory coat...", but I would much rather just say "a scientist" and have the system recognize that anyone can be a scientist. And then if I need to be specific, then I would be happy to say "a black-haired scientist"
- kache_ 4y ago
  This is a problem with generative models across the board. It's important that we don't skew our perceptions by GAN outputs as a society, so it's definitely good that we're thinking about it. I just wish that we had a solution that solved across the class of problems "Generative AI feeds into itself and society (which is in a way, a generative AI), creating a positive feedback loop that eventually leads to a cultural freeze" It's way bigger than just this narrow race issue the current zeitgeist is concerned about. But I agree, maybe I should skew to being optimistic that at least we're trying
- numpad0 4y ago
  Kind of funny that NN tech is supposed to construct some upper dimensional understanding, yet realistically cannot be expected to be able to generate gender and race indeterminate portrayal of a scientist.

deleted 4y ago

[deleted]