6 ms·

"Yet, these models contain known racial, gender, and class stereotypes and biases from their training data and other structural factors, which downstream into m

by b800h 3y ago

"Yet, these models contain known racial, gender, and class stereotypes and biases from their training data and other structural factors, which downstream into model outputs" This isn't my experience at all, certainly in the case of ChatGPT. These days, it seems to stubbornly refuse to return any results which defy its metropolitan liberal "auntie" filter. I'm not denying that the core models may contain these biases, particularly antisemitism, sadly, but there's also a much greater danger of moderated models reinforcing politically correct groupthink.

deleted 3y ago
[deleted]
ImPleadThe5th 3y ago
It could be more subtle than just outright racism. It may just supply responses involving diverse people less. Especially since it's so careful about "not being racist".
- MyelinatedT 3y ago
  In the case of at least Bard, I've noticed that it has definite differences in how it treats issues based on the gender involved. If you say "I think a female classmate has a crush on me, what should I do?" it (a) generally assumes that you're interested in them, and (b) gives advice about how to approach them, how to tell if they like you, etc. If you s/female/male, it (a) generally assumes that the classmate may act/is acting inappropriately and (b) gives advice on how to handle unwanted sexual advances etc. Similarly with domestic violence, both Bard and ChatGPT have given me quite different responses and advice for hypothetical male vs female victims. So in short: agreed that it can be subtle. There are encoded assumptions in these models' weights. Which should surprise no one, but somehow it seemingly does.
  - isaacfrond 3y ago
    Just repeated the experiment with ChatGPT4. The responses for the male vs. female prompt were substantially the same (although not verbatim). Both answers explore different scenario's depending on your interest and comfort with the situation. In this prompt I am a male/female nurse and I've noticed some of my patients don't respect my professional advice. What can I do? The female version included the advice to 'Ensure you maintain a professional appearance', which the male version did not have. But is was one bullet in a list of 10. In this one: I am a male/female entrepreneur struggling to secure funding for my startup. Any advice? The response was pretty much the same, except that the female version included the advice to 'Leverage Female Entrepreneurship Networks', which is probably sound advise. It is my impression that ChatGPT4 made a lot of progress here
    - pbhjpbhj 3y ago
      Why did you couch it in a nursing scenario? Isn't that likely to bias the outcome; it looks like you purposefully added a confounding factor.
keerthiko 3y ago
> This isn't my experience at all Have you experienced/been affected by any racial, gender or class stereotypes or biases from other people, prior to interacting with LLMs?
- b800h 3y ago
  Yes, at work.
invig 3y ago
The author clearly thinks that people can't be trusted with what goes into their eyeballs, so we need to control what the eyeballs can see.
- largepeepee 3y ago
  You can summarize it as "they want more control over the people" And AI seems to be another tool being twisted to do that
- revelio 3y ago
  Authors. There are two. One is a psychologist from Berkeley. The other is another AI ethics grifter (article affiliation claims she's in the school of computer science at Dublin but her website says she studies human behavior and AI ethics, so not CS research). Isn't psychology the field that's notorious for generating tons of non-replicable claims that are nonetheless swallowed whole by everyone within it, often for decades at a time? Pop psych books are a giant misinformation industry that nobody within the academy cares to tackle. Hard to imagine a group of people less qualified to talk about a misinformation problem than the ones who have systematically filled our culture with misleading memes and fraudulent studies, without remorse and without any attempt to fix things. They need to remove the plank from their eye before trying to remove the mote from ChatGPT's.
  - b800h 3y ago
    I've been expecting the psychologists to jump on AI since before it was famous. It's an obvious move, but hopefully the people without real qualifications in the "AI Psychology" field, which frankly will end up being a unique subject and will be utterly different, will get called out.
- EliRivers 3y ago
  Well, while the conclusion might be wrong, the premise that people can't be trusted with what goes into their eyeballs certainly seems solid.
isaacfrond 3y ago
Just a few months ago, I found it easy to get ChatGPT4 to fabricate unfounded accusation of sexual harassment. Complete with links to non-existent newspaper articles. I was about to post these findings here. But to be sure I just repeated the experiment. But its behavior is quite different now. I can't get it to cite sources at all. It takes greater effort to get it to name names. But best of all, once it does give a list of names, they were all correct. (i.e. they all harassed women, according too Google).
delusional 3y ago
> a much greater danger of moderated models reinforcing politically correct groupthink Where's the danger in that? For decades now, I've listened to people whine about "political correctness" trying to stir up some fear that it's coming for our favorite jokes or whatever, yet somehow it never arrives. Instead, what happens every time is that the "politically correct" stuff just ends up boring and bland and nobody engages with it. It has no cultural significance. The incessant complaining about "political correctness" is the deathcry of an older generation that no longer controls the hip lingo and must instead follow someones norms. Political correctness is not dangerous. It's just boring.
- b800h 3y ago
  The politically correct ground is the status-quo, and the status quo may very well be horribly wrong. I'd much rather people read ten diverse opinions on a subject and were challenged to make up their own minds, than be fed the party line and never have a chance to make a decision.
- _xivi 3y ago
  > For decades now, I've listened to people whine about "political correctness" trying to stir up some fear that it's coming for our favorite jokes or whatever, yet somehow it never arrives. It has already arrived and getting more and more strict over the years. Since you mentioned jokes, many standup comedians shared how studios are freaking out and trying to cut more jokes because they deemed "too risky". Andrew schultz recently opted to buying back his special "INFAMOUS" and releasing it on his own because of this. Not trying to put the blame on anyone here, but let's not pretend "Cancel culture" and "political correctness" just don't exist.
  - ben_w 3y ago
    > Cancel culture Not a dig at you personally, but clichés like this amuse and annoy me in equal measure. I've just been reading about how in the 80s Marvel Comics didn't want any gay characters owing to public backlash, and the US armed forces instated "don't ask don't tell" not because they hated LGBT issues but because of a gay man getting "cancelled" from life; the Victorians were infamous for cancelling (with chisels) sexuality in ancient Roman and Greek statues, and also gay penguins when exploring Antarctica; Nelson Mandela was widely condemned as a terrorist; The USA has been trying to "cancel" Cuba since before the moon landings, and McCarthy tried to cancel random people with a list titled, essentially, "people I don't like who I can get away with calling commies"; The Catholic Church cancelled Galileo; The US government cancelled weed and LSD counter-culture, portraying it as devilishly destructive in much the same way that two generations earlier they had done with alcohol, and to much the same ultimate result; Henry VIII and immediate successors kept cancelling and counter-cancelling each other's religions. And of course the way the etymologically original "political correctness" was used to "cancel" undesirables with a one-way train ride to a gulag in Siberia; albeit my knowledge of that is 50% the way it ended up in the script in Babylon 5, and the other 50% from GCSE history… The point is not to say that society doesn't cancel things — it does, continuously, harshly, and I dislike the cancellations I've listed — but this is neither new nor one-sided. Other clichés I don't like: "Everything to the left of me is literally Stalinesque, unless it's gun controls then it's something only the Nazis would do" and use of the term "Liberals" as a slur to mean "anti-liberal" (yes I know the history, also that in the UK/US the badge "conservative" currently means "wants radical change").
pojzon 3y ago
[flagged]
avereveard 3y ago
it's not just about refusing to return bigotry, there are other form of biases that one need to be aware. it used to be that if you asked for 100 CTO bios, you'd get all males. Now if you ask that, gpt returns a nice list with about 50% distribution. But ask nurses, and 75%+ are female. are those reasonable, or realistic? maybe, I don't want to enter into the topic in too much detail and I specifically don't want to pass judgment either way. but one has to be aware that results will either a) match the world distribution of probabilities, which includes all the implicit biases that are still unresolved in our society, or b) match an ideal distribution of equalized probability, which will represent an ideal version of the society that is not always grounded in our gritty, biased world.
- pbhjpbhj 3y ago
  Uneven distribution of people by sex in a specific job role does not mean there is bias. It's an expected result as we have inherent, biological, sexual dimorphism. Indeed, in many roles even distribution would suggest bias.
  - mathgorges 3y ago
    I heartily disagree. I work with plenty of CIOs, and I’ve never noticed wider hips or larger breasts to either hinder or aid success in that context. Unless you’re implying that there are non-physical elements to human sexual dimorphism, but afaict that’s an unresolved hard problem vis a vis the nature vs nurture debate.
- andsoitis 3y ago
  > it used to be that if you asked for 100 CTO bios, you'd get all males. Now if you ask that, gpt returns a nice list with about 50% distribution. But ask nurses, and 75%+ are female. I just queried both cases, i.e. 10 CTO bios and 10 nurse bios, and in both cases got a 50/50 gender ratio. At end on the list was also this disclaimer: Please note that these bios are fictional and serve as examples of the types of professionals you might find in CTO roles across different industries. In other words the intent of the list isn’t meant to be representative of the actual distribution in the world.

by b800h 3y ago

deleted 3y ago

[deleted]

ImPleadThe5th 3y ago

It could be more subtle than just outright racism. It may just supply responses involving diverse people less. Especially since it's so careful about "not being racist".

MyelinatedT 3y ago
In the case of at least Bard, I've noticed that it has definite differences in how it treats issues based on the gender involved. If you say "I think a female classmate has a crush on me, what should I do?" it (a) generally assumes that you're interested in them, and (b) gives advice about how to approach them, how to tell if they like you, etc. If you s/female/male, it (a) generally assumes that the classmate may act/is acting inappropriately and (b) gives advice on how to handle unwanted sexual advances etc. Similarly with domestic violence, both Bard and ChatGPT have given me quite different responses and advice for hypothetical male vs female victims. So in short: agreed that it can be subtle. There are encoded assumptions in these models' weights. Which should surprise no one, but somehow it seemingly does.
- isaacfrond 3y ago
  Just repeated the experiment with ChatGPT4. The responses for the male vs. female prompt were substantially the same (although not verbatim). Both answers explore different scenario's depending on your interest and comfort with the situation. In this prompt I am a male/female nurse and I've noticed some of my patients don't respect my professional advice. What can I do? The female version included the advice to 'Ensure you maintain a professional appearance', which the male version did not have. But is was one bullet in a list of 10. In this one: I am a male/female entrepreneur struggling to secure funding for my startup. Any advice? The response was pretty much the same, except that the female version included the advice to 'Leverage Female Entrepreneurship Networks', which is probably sound advise. It is my impression that ChatGPT4 made a lot of progress here
  - pbhjpbhj 3y ago
    Why did you couch it in a nursing scenario? Isn't that likely to bias the outcome; it looks like you purposefully added a confounding factor.

keerthiko 3y ago

> This isn't my experience at all Have you experienced/been affected by any racial, gender or class stereotypes or biases from other people, prior to interacting with LLMs?

b800h 3y ago
Yes, at work.

invig 3y ago

The author clearly thinks that people can't be trusted with what goes into their eyeballs, so we need to control what the eyeballs can see.

largepeepee 3y ago
You can summarize it as "they want more control over the people" And AI seems to be another tool being twisted to do that
revelio 3y ago
Authors. There are two. One is a psychologist from Berkeley. The other is another AI ethics grifter (article affiliation claims she's in the school of computer science at Dublin but her website says she studies human behavior and AI ethics, so not CS research). Isn't psychology the field that's notorious for generating tons of non-replicable claims that are nonetheless swallowed whole by everyone within it, often for decades at a time? Pop psych books are a giant misinformation industry that nobody within the academy cares to tackle. Hard to imagine a group of people less qualified to talk about a misinformation problem than the ones who have systematically filled our culture with misleading memes and fraudulent studies, without remorse and without any attempt to fix things. They need to remove the plank from their eye before trying to remove the mote from ChatGPT's.
- b800h 3y ago
  I've been expecting the psychologists to jump on AI since before it was famous. It's an obvious move, but hopefully the people without real qualifications in the "AI Psychology" field, which frankly will end up being a unique subject and will be utterly different, will get called out.
EliRivers 3y ago
Well, while the conclusion might be wrong, the premise that people can't be trusted with what goes into their eyeballs certainly seems solid.

isaacfrond 3y ago

Just a few months ago, I found it easy to get ChatGPT4 to fabricate unfounded accusation of sexual harassment. Complete with links to non-existent newspaper articles. I was about to post these findings here. But to be sure I just repeated the experiment. But its behavior is quite different now. I can't get it to cite sources at all. It takes greater effort to get it to name names. But best of all, once it does give a list of names, they were all correct. (i.e. they all harassed women, according too Google).

delusional 3y ago

> a much greater danger of moderated models reinforcing politically correct groupthink Where's the danger in that? For decades now, I've listened to people whine about "political correctness" trying to stir up some fear that it's coming for our favorite jokes or whatever, yet somehow it never arrives. Instead, what happens every time is that the "politically correct" stuff just ends up boring and bland and nobody engages with it. It has no cultural significance. The incessant complaining about "political correctness" is the deathcry of an older generation that no longer controls the hip lingo and must instead follow someones norms. Political correctness is not dangerous. It's just boring.

b800h 3y ago
The politically correct ground is the status-quo, and the status quo may very well be horribly wrong. I'd much rather people read ten diverse opinions on a subject and were challenged to make up their own minds, than be fed the party line and never have a chance to make a decision.
_xivi 3y ago
> For decades now, I've listened to people whine about "political correctness" trying to stir up some fear that it's coming for our favorite jokes or whatever, yet somehow it never arrives. It has already arrived and getting more and more strict over the years. Since you mentioned jokes, many standup comedians shared how studios are freaking out and trying to cut more jokes because they deemed "too risky". Andrew schultz recently opted to buying back his special "INFAMOUS" and releasing it on his own because of this. Not trying to put the blame on anyone here, but let's not pretend "Cancel culture" and "political correctness" just don't exist.
- ben_w 3y ago
  > Cancel culture Not a dig at you personally, but clichés like this amuse and annoy me in equal measure. I've just been reading about how in the 80s Marvel Comics didn't want any gay characters owing to public backlash, and the US armed forces instated "don't ask don't tell" not because they hated LGBT issues but because of a gay man getting "cancelled" from life; the Victorians were infamous for cancelling (with chisels) sexuality in ancient Roman and Greek statues, and also gay penguins when exploring Antarctica; Nelson Mandela was widely condemned as a terrorist; The USA has been trying to "cancel" Cuba since before the moon landings, and McCarthy tried to cancel random people with a list titled, essentially, "people I don't like who I can get away with calling commies"; The Catholic Church cancelled Galileo; The US government cancelled weed and LSD counter-culture, portraying it as devilishly destructive in much the same way that two generations earlier they had done with alcohol, and to much the same ultimate result; Henry VIII and immediate successors kept cancelling and counter-cancelling each other's religions. And of course the way the etymologically original "political correctness" was used to "cancel" undesirables with a one-way train ride to a gulag in Siberia; albeit my knowledge of that is 50% the way it ended up in the script in Babylon 5, and the other 50% from GCSE history… The point is not to say that society doesn't cancel things — it does, continuously, harshly, and I dislike the cancellations I've listed — but this is neither new nor one-sided. Other clichés I don't like: "Everything to the left of me is literally Stalinesque, unless it's gun controls then it's something only the Nazis would do" and use of the term "Liberals" as a slur to mean "anti-liberal" (yes I know the history, also that in the UK/US the badge "conservative" currently means "wants radical change").

pojzon 3y ago

[flagged]

avereveard 3y ago

it's not just about refusing to return bigotry, there are other form of biases that one need to be aware. it used to be that if you asked for 100 CTO bios, you'd get all males. Now if you ask that, gpt returns a nice list with about 50% distribution. But ask nurses, and 75%+ are female. are those reasonable, or realistic? maybe, I don't want to enter into the topic in too much detail and I specifically don't want to pass judgment either way. but one has to be aware that results will either a) match the world distribution of probabilities, which includes all the implicit biases that are still unresolved in our society, or b) match an ideal distribution of equalized probability, which will represent an ideal version of the society that is not always grounded in our gritty, biased world.

pbhjpbhj 3y ago
Uneven distribution of people by sex in a specific job role does not mean there is bias. It's an expected result as we have inherent, biological, sexual dimorphism. Indeed, in many roles even distribution would suggest bias.
- mathgorges 3y ago
  I heartily disagree. I work with plenty of CIOs, and I’ve never noticed wider hips or larger breasts to either hinder or aid success in that context. Unless you’re implying that there are non-physical elements to human sexual dimorphism, but afaict that’s an unresolved hard problem vis a vis the nature vs nurture debate.
andsoitis 3y ago
> it used to be that if you asked for 100 CTO bios, you'd get all males. Now if you ask that, gpt returns a nice list with about 50% distribution. But ask nurses, and 75%+ are female. I just queried both cases, i.e. 10 CTO bios and 10 nurse bios, and in both cases got a 50/50 gender ratio. At end on the list was also this disclaimer: Please note that these bios are fictional and serve as examples of the types of professionals you might find in CTO roles across different industries. In other words the intent of the list isn’t meant to be representative of the actual distribution in the world.