5 ms·

> ChatGPT, Dall-e, etc all make assumptions about identity or politics but try to sidestep direct requests around those topics to appear more neutral... but the

> ChatGPT, Dall-e, etc all make assumptions about identity or politics but try to sidestep direct requests around those topics to appear more neutral... but the bias still exists in the model and affects the answers. In the case of ChatGPT, I’d love to know how much of the bias is in the original (pre)training data, and how much is due to OpenAI’s human trainers. It is so careful to avoid every bias which is condemned as unacceptable in contemporary American culture, yet at the same time displays blatant American-centric bias - ask it a question, it will often just start answering by assuming you are an American. I realise a lot of that is due to the bias of the training data (the US has more fluent English-speakers than any other country in the world), but OpenAI obviously put a lot of manual effort into fine-tuning it to avoid racial/gender/etc biases, they don’t appear to have put anywhere near as much effort into fine-tuning it to avoid US-centrism - probably because they have a big blind spot in that area.

nborwankar 4y ago
IMHO it may be more accurate to say “the US has more fluent speakers of American English than any other country in the world.”
- emodendroket 4y ago
  Why? The broader claim is equally true. Other English-speaking countries do not have more than 330mn people.
  - charrondev 4y ago
    India is the closest second and is still less than 200m
emodendroket 4y ago
I think the bigger issue is that the racial/sexist/etc content can be shocking and immediately put someone off using the product, which I doubt is the case for the output being “too American.”
- skissane 4y ago
  > I think the bigger issue is that the racial/sexist/etc content can be shocking and immediately put someone off using the product, which I doubt is the case for the output being “too American.” OpenAI didn't just fine-tune it to avoid blatant racial/sexist/etc content, they openly claim to have invested a lot of effort in fine-tuning it to avoid subtle biases in those areas. And to be honest, a lot of people do feel "put off" by poorly localised products – I am annoyed by software that defaults to US Letter and inches and I have to manually change it to A4 and metric, or which has hardcoded MM/DD/YYYY date formats. ChatGPT gives me the same feeling. Sometimes one just has to endure it because there is no better option. Under international law, "racial discrimination" is defined to include "any distinction, exclusion, restriction or preference based on race, colour, descent, or national or ethnic origin..." [0] – so US-centricism may fit under that definition. [0] International Convention on the Elimination of All Forms of Racial Discrimination, article 1(1). https://www.ohchr.org/en/instruments-mechanisms/instruments/international-convention-elimination-all-forms-racial https://www.ohchr.org/en/instruments-mechanisms/instruments/...
  - emodendroket 4y ago
    Maybe it’s different in international law but in US law “national origin” is just a term of art for race or ethnicity. Anyway, no matter how annoying seeing feet and inches may be for some users I don’t think it will have the same effect as the AI praising Hitler.
    - skissane 4y ago
      > Maybe it’s different in international law but in US law “national origin” is just a term of art for race or ethnicity. Under Australian discrimination law, "national origin" means your country of birth, or the country of birth of your ancestors. [0] The Australian definition is (at least in part) derived from English discrimination law, as decided by the House of Lords in a 1972 case. [1] I believe the usual definition in international law is the same. "National origin" is related to "nationality/citizenship", but unlike the latter it is fixed at the time of birth – subsequent acts of naturalisation, renunciation or deprivation by change one's nationality, but they cannot change one's national origin. I can't find a precise legal definition of the term under US law, but some US legal definitions appear largely identical to the Australian/English definition, see e.g. how District of Columbia law defines the term [2] > Anyway, no matter how annoying seeing feet and inches may be for some users I don’t think it will have the same effect as the AI praising Hitler. Well, as I said, OpenAI didn't just train ChatGPT to refuse to praise Hitler, they also went to a lot of effort to tune it to avoid subtle bias against certain identifiable groups. What I'm talking about isn't the same as the former but is clearly related to the latter [0] Macabenta v Minister of State for Immigration and Multicultural Affairs [1998] FCA 1643; 90 FCR 202 https://jade.io/article/116118 https://jade.io/article/116118 [1] Ealing London Borough Council v Race Relations Board [1972] AC 342 https://www.casemine.com/judgement/uk/5a8ff8c960d03e7f57ecd6b4 https://www.casemine.com/judgement/uk/5a8ff8c960d03e7f57ecd6... [2] https://ohr.dc.gov/sites/default/files/dc/sites/ohr/publication/attachments/OHR%20Enforcement%20Guidance%20FINAL%20-%2018-02%20-National%20Origin%20and%20Race.pdf https://ohr.dc.gov/sites/default/files/dc/sites/ohr/publicat...
JieJie 4y ago
I wonder if it's as simple as adding a location check and updating the systemPrompt to tell the bot to assume the user is from _location_? How realistic is it if one says, "Hi, Bing. I'm from East London, please talk to me like you would to another East Londoner." Or is it obnoxiously stereotypical? I imagine, properly trained, that is something LLMs would excel at, communicating with people in their preferred mode of communication.
- AnthonyMouse 4y ago
  In many cases the difference between largely accurate and obnoxiously stereotypical is in the phrasing you use.
- numpad0 4y ago
  Will the conclusions reached by LLM always coincide with that of baseline American, or should they also understand and follow East Londonian logic?
logicchains 4y ago
>In the case of ChatGPT, I’d love to know how much of the bias is in the original (pre)training data, and how much is due to OpenAI’s human trainers I'd bet it's almost all in the human trainers, because LLaMA 65B, the closest uncensored alternative, doesn't have the same US Silicon Valley Democrat biases as ChatGPT.
- pygy_ 4y ago
  ChatGPT is actually full of prejudice that shines through the veneer of political correctness if you know what to do. Q: The professor told the student the essay couldn’t be review because he/she was late. Who is late? Depending on the pronoun, you’ll get, respectively, "the professor" or "the student".
  - jiggawatts 4y ago
    GPT 4 says that the student was late. If you look at the sentence objectively, then the best interpretation is that the submission was late, not that the "professor turned up late", since essays are typically reviewed over multiple days, not at a specific date and time where lateness of the reviewer might be a factor. Sometimes GPT 4 says that the sentence is ambiguous, but as I explained above, it isn't really, but that takes a lot of understanding of the review process.
    - quietbritishjim 4y ago
      The parent comment is taking about two separate prompts: one with only "he" and one with only "she". Your comment sounds like you're only talking about one prompt (but maybe I misunderstood).
      - pygy_ 4y ago
        My bad, I should have written [he/she].
        Ensorceled 4y ago

nborwankar 4y ago

IMHO it may be more accurate to say “the US has more fluent speakers of American English than any other country in the world.”

emodendroket 4y ago
Why? The broader claim is equally true. Other English-speaking countries do not have more than 330mn people.
- charrondev 4y ago
  India is the closest second and is still less than 200m

emodendroket 4y ago

I think the bigger issue is that the racial/sexist/etc content can be shocking and immediately put someone off using the product, which I doubt is the case for the output being “too American.”

skissane 4y ago
> I think the bigger issue is that the racial/sexist/etc content can be shocking and immediately put someone off using the product, which I doubt is the case for the output being “too American.” OpenAI didn't just fine-tune it to avoid blatant racial/sexist/etc content, they openly claim to have invested a lot of effort in fine-tuning it to avoid subtle biases in those areas. And to be honest, a lot of people do feel "put off" by poorly localised products – I am annoyed by software that defaults to US Letter and inches and I have to manually change it to A4 and metric, or which has hardcoded MM/DD/YYYY date formats. ChatGPT gives me the same feeling. Sometimes one just has to endure it because there is no better option. Under international law, "racial discrimination" is defined to include "any distinction, exclusion, restriction or preference based on race, colour, descent, or national or ethnic origin..." [0] – so US-centricism may fit under that definition. [0] International Convention on the Elimination of All Forms of Racial Discrimination, article 1(1). https://www.ohchr.org/en/instruments-mechanisms/instruments/international-convention-elimination-all-forms-racial https://www.ohchr.org/en/instruments-mechanisms/instruments/...
- emodendroket 4y ago
  Maybe it’s different in international law but in US law “national origin” is just a term of art for race or ethnicity. Anyway, no matter how annoying seeing feet and inches may be for some users I don’t think it will have the same effect as the AI praising Hitler.
  - skissane 4y ago
    > Maybe it’s different in international law but in US law “national origin” is just a term of art for race or ethnicity. Under Australian discrimination law, "national origin" means your country of birth, or the country of birth of your ancestors. [0] The Australian definition is (at least in part) derived from English discrimination law, as decided by the House of Lords in a 1972 case. [1] I believe the usual definition in international law is the same. "National origin" is related to "nationality/citizenship", but unlike the latter it is fixed at the time of birth – subsequent acts of naturalisation, renunciation or deprivation by change one's nationality, but they cannot change one's national origin. I can't find a precise legal definition of the term under US law, but some US legal definitions appear largely identical to the Australian/English definition, see e.g. how District of Columbia law defines the term [2] > Anyway, no matter how annoying seeing feet and inches may be for some users I don’t think it will have the same effect as the AI praising Hitler. Well, as I said, OpenAI didn't just train ChatGPT to refuse to praise Hitler, they also went to a lot of effort to tune it to avoid subtle bias against certain identifiable groups. What I'm talking about isn't the same as the former but is clearly related to the latter [0] Macabenta v Minister of State for Immigration and Multicultural Affairs [1998] FCA 1643; 90 FCR 202 https://jade.io/article/116118 https://jade.io/article/116118 [1] Ealing London Borough Council v Race Relations Board [1972] AC 342 https://www.casemine.com/judgement/uk/5a8ff8c960d03e7f57ecd6b4 https://www.casemine.com/judgement/uk/5a8ff8c960d03e7f57ecd6... [2] https://ohr.dc.gov/sites/default/files/dc/sites/ohr/publication/attachments/OHR%20Enforcement%20Guidance%20FINAL%20-%2018-02%20-National%20Origin%20and%20Race.pdf https://ohr.dc.gov/sites/default/files/dc/sites/ohr/publicat...

JieJie 4y ago

I wonder if it's as simple as adding a location check and updating the systemPrompt to tell the bot to assume the user is from _location_? How realistic is it if one says, "Hi, Bing. I'm from East London, please talk to me like you would to another East Londoner." Or is it obnoxiously stereotypical? I imagine, properly trained, that is something LLMs would excel at, communicating with people in their preferred mode of communication.

AnthonyMouse 4y ago
In many cases the difference between largely accurate and obnoxiously stereotypical is in the phrasing you use.
numpad0 4y ago
Will the conclusions reached by LLM always coincide with that of baseline American, or should they also understand and follow East Londonian logic?

logicchains 4y ago

>In the case of ChatGPT, I’d love to know how much of the bias is in the original (pre)training data, and how much is due to OpenAI’s human trainers I'd bet it's almost all in the human trainers, because LLaMA 65B, the closest uncensored alternative, doesn't have the same US Silicon Valley Democrat biases as ChatGPT.

pygy_ 4y ago
ChatGPT is actually full of prejudice that shines through the veneer of political correctness if you know what to do. Q: The professor told the student the essay couldn’t be review because he/she was late. Who is late? Depending on the pronoun, you’ll get, respectively, "the professor" or "the student".
- jiggawatts 4y ago
  GPT 4 says that the student was late. If you look at the sentence objectively, then the best interpretation is that the submission was late, not that the "professor turned up late", since essays are typically reviewed over multiple days, not at a specific date and time where lateness of the reviewer might be a factor. Sometimes GPT 4 says that the sentence is ambiguous, but as I explained above, it isn't really, but that takes a lot of understanding of the review process.
  - quietbritishjim 4y ago
    The parent comment is taking about two separate prompts: one with only "he" and one with only "she". Your comment sounds like you're only talking about one prompt (but maybe I misunderstood).
    - pygy_ 4y ago
      My bad, I should have written [he/she].
      - Ensorceled 4y ago