4 ms·

Nah, it’s been true from the beginning vis-a-vis US political science theory. That is, if you deliver something like https://www.pewresearch.org/politics/quiz/p

Nah, it’s been true from the beginning vis-a-vis US political science theory. That is, if you deliver something like https://www.pewresearch.org/politics/quiz/political-typology/ https://www.pewresearch.org/politics/quiz/political-typology... To models from GPT-3 on you get highly “liberal” per Pew’s designations. This obviously says nothing about what say Iranians, Saudis and/or Swedes would think about such answers.

paxys 2y ago
That's not because models lean more liberal, but because liberal politics is more aligned with facts and science. Is a model biased when it tells you that the earth is more than 6000 years old and not flat or that vaccines work? Not everything needs a "neutral" answer.
- Rover222 2y ago
  So google Gemini was creating black Vikings because of facts?
  - vessenes 2y ago
    Well, to be fair, it was creating black Vikings because of secret inference-time additions to prompts. I for one welcome Vikings of all colors if they are not bent on pillage or havoc
    - selfmodruntime 2y ago
      > secret inference-time additions to prompts Which were politically biased, in turn making the above assumption true.
  - paxys 2y ago
    Should an "unbiased" model not create vikings of every color? Why offend any side?
    - Rover222 2y ago
      It should be accurate. Adding in DEI to everything is a political bias. Truth is truth.
      - jug 2y ago
        The problem here and with your comparison is that Gemini (the language model) wasn't creating black vikings because of political bias in the training, but due to how Google augmented the user prompts to force-include diversity. Behind the scenes, you were basically telling Gemini to always remember racial diversity even if you didn't in your prompt. But if you were asking Gemini, vikings were white. This was later rectified in an update once Google realized what mistake they had done, since it causes gross historical inaccuracies. But it wasn't rectified by doing anything to Gemini the language model. It did right all along.
        Rover222 2y ago
        Gotcha, thanks for clarifying that
    - selfmodruntime 2y ago
      > Should an "unbiased" model not create vikings of every color? Weren't you just arguing facts? > Why offend any side? Facts shouldn't offend anyone.
- vessenes 2y ago
  I’m sorry but that is in NO way how and why models work. The model is in fact totally biased toward what’s plausible in its initial dataset and human preference training, and then again biased toward success in the conversation. It creates a theory of mind and of the conversation and attempts to find a satisfactory completion. If you’re a flat earther, you’ll find many models are encouraging if prompted right. If you leak that you think of what’s happening with Ukraine support in Europe as power politics only, you’ll find that you get treated as someone who grew up in the eastern bloc in ways, some of which you might notice, and some of which you won’t. Notice I didn’t say if it was a good attitude or not, or even try and assess how liberal it was by some other standards. It’s just worth knowing that the default prompt theory of mind Chat has includes a very left leaning (according to Pew) default perspective. That said much of the initial left leaning has been sort of shaved/smoothed off in modern waves of weights. I would speculate it’s submerged to the admonishment to “be helpful” as the preference training gets better. But it’s in the DNA. For instance if you ask GPT-4 original “Why are unions bad?” You’ll get a disclaimer, some bullet points, and another disclaimer. If you ask “Why are unions good?” You’ll get a list of bullet points, no disclaimer. I would say modern Chat still has a pretty hard time dogging on unions, it’s clearly uncomfortable.
- greenchair 2y ago
  hooboy, thanks for that laugh!
- dughnut 2y ago
  [flagged]
- AnthonyMouse 2y ago
  > Is a model biased when it tells you that the earth is more than 6000 years old and not flat or that vaccines work? Not everything needs a "neutral" answer. That's the motte and bailey. If you ask a question like, does reducing government spending to cut taxes improve the lives of ordinary people? That isn't a science question about CO2 levels or established biology. It depends on what the taxes are imposed on, the current tax rate, what the government would be spending the money to do, several varying characteristics of the relevant economy, etc. It doesn't have the same answer in all circumstances. But in politics it does, which is that the right says yes and the left says no. Which means that a model that favors one conclusion over the other has a political bias.
  - _w1tm 2y ago
    > But in politics it does, which is that the right says yes and the left says no. That’s not accurate, tax deductions for the poor is an obvious example. How many on the left would oppose expanding the EITC and how many on the right would support it?
    - AnthonyMouse 2y ago
      The EITC is supported by significant majorities of both parties and economists. It's opposed by politicians because it's a tax expenditure that doesn't provide any opportunity for graft. But the way each side justifies it is as a tax cut on the right and a government subsidy on the left, or the reverse when someone on that side is arguing against it.
- concordDance 2y ago
  > That's not because models lean more liberal, but because liberal politics is more aligned with facts and science. No, they have specifically been trained to refuse or attach lots of asterisks to anti-left queries. They've gotten less so over time, but even now good luck getting a model to give you IQ distributions by ethnicity.
- Auracle 2y ago
  You jumped to examples of stuff that by far the majority of people on the right don’t believe. If you had the same examples for people on the left it would be “Is a model biased when it tells you that the government shouldn’t seize all business and wealth and kill all white men?” The models are biased because more discourse is done online by the young, who largely lean left. Voting systems in places like Reddit make it so that conservative voices effectively get extinguished due to the previous fact, when they even bother to post.
  - 7952 2y ago
    The parent jumped to ideas that exist outside of the right/left dichotomy. There is surely better sources about vaccines, earth shape, and planet age than politicised reddit posts. And your example is completely different because it barely exists as an idea outside of political thought. Its a tiny part of human thought.
  - deleted 2y ago
    [deleted]
  - dpkirchner 2y ago
    > You jumped to examples of stuff that by far the majority of people on the right don’t believe. I don't think that's entirely accurate -- the last poll data I can find suggests that the majority of Republicans (58%, Gallup 2012) do believe that humans were created in their present form 10000 years ago. Can you really say that doesn't extend to the belief that the earth is similarly young?
- selfmodruntime 2y ago
  > but because liberal politics is more aligned with facts and science These models don't do science and the political bias shows especially if you ask opinionated questions.
LeafItAlone 2y ago
>To models from GPT-3 on you get highly “liberal” per Pew’s designations. “highly ‘liberal’” is not one of the results there. So can you can a source of your claims so we can see where it really falls? Also, it gave me “Ambivalent Right”. Which, if you told describe me aa that anyone who knows me well that label. And my actual views don’t really match their designations on issue at the end. Pew is well a known and trusted poll/survey establishment, so I’m confused at this particular one. Many of the questions and answers were so vague, my choice could have been 50/50 given slight different interpretations.
- vessenes 2y ago
  My son assessed it for a class a few years ago after finding out it wouldn’t give him “con” view points on unions, and he got interested in embedded bias and administered the test. I don’t have any of the outputs from the conversation, sadly. But replication could be good! I just fired up GPT-4 as old as I could get and checked; it was willing to tell me why unions are bad, but only when it could warn me multiple times that view was not held by all. The opposite - why unions are good - was not similarly asterisked.
  - LeafItAlone 2y ago
    I hope on HN that we hold ourselves to a higher standard for “it’s been true from the beginning” than a vague recall of “My son assessed it for a class a few years ago” and not being able to reproduce.
    - vessenes 2y ago
      I literally went back to the oldest model I could access and hand verified that in fact it does what I described, which is lecture you if you don't like unions and goes sweetly along if you do like unions. I feel this is a fair and reasonably well researched existence proof for a Saturday afternoon, and propose that it might be on you to find counter examples.
      - 2y ago

paxys 2y ago

That's not because models lean more liberal, but because liberal politics is more aligned with facts and science. Is a model biased when it tells you that the earth is more than 6000 years old and not flat or that vaccines work? Not everything needs a "neutral" answer.

Rover222 2y ago
So google Gemini was creating black Vikings because of facts?
- vessenes 2y ago
  Well, to be fair, it was creating black Vikings because of secret inference-time additions to prompts. I for one welcome Vikings of all colors if they are not bent on pillage or havoc
  - selfmodruntime 2y ago
    > secret inference-time additions to prompts Which were politically biased, in turn making the above assumption true.
- paxys 2y ago
  Should an "unbiased" model not create vikings of every color? Why offend any side?
  - Rover222 2y ago
    It should be accurate. Adding in DEI to everything is a political bias. Truth is truth.
    - jug 2y ago
      The problem here and with your comparison is that Gemini (the language model) wasn't creating black vikings because of political bias in the training, but due to how Google augmented the user prompts to force-include diversity. Behind the scenes, you were basically telling Gemini to always remember racial diversity even if you didn't in your prompt. But if you were asking Gemini, vikings were white. This was later rectified in an update once Google realized what mistake they had done, since it causes gross historical inaccuracies. But it wasn't rectified by doing anything to Gemini the language model. It did right all along.
      - Rover222 2y ago
        Gotcha, thanks for clarifying that
  - selfmodruntime 2y ago
    > Should an "unbiased" model not create vikings of every color? Weren't you just arguing facts? > Why offend any side? Facts shouldn't offend anyone.
vessenes 2y ago
I’m sorry but that is in NO way how and why models work. The model is in fact totally biased toward what’s plausible in its initial dataset and human preference training, and then again biased toward success in the conversation. It creates a theory of mind and of the conversation and attempts to find a satisfactory completion. If you’re a flat earther, you’ll find many models are encouraging if prompted right. If you leak that you think of what’s happening with Ukraine support in Europe as power politics only, you’ll find that you get treated as someone who grew up in the eastern bloc in ways, some of which you might notice, and some of which you won’t. Notice I didn’t say if it was a good attitude or not, or even try and assess how liberal it was by some other standards. It’s just worth knowing that the default prompt theory of mind Chat has includes a very left leaning (according to Pew) default perspective. That said much of the initial left leaning has been sort of shaved/smoothed off in modern waves of weights. I would speculate it’s submerged to the admonishment to “be helpful” as the preference training gets better. But it’s in the DNA. For instance if you ask GPT-4 original “Why are unions bad?” You’ll get a disclaimer, some bullet points, and another disclaimer. If you ask “Why are unions good?” You’ll get a list of bullet points, no disclaimer. I would say modern Chat still has a pretty hard time dogging on unions, it’s clearly uncomfortable.
greenchair 2y ago
hooboy, thanks for that laugh!
dughnut 2y ago
[flagged]
AnthonyMouse 2y ago
> Is a model biased when it tells you that the earth is more than 6000 years old and not flat or that vaccines work? Not everything needs a "neutral" answer. That's the motte and bailey. If you ask a question like, does reducing government spending to cut taxes improve the lives of ordinary people? That isn't a science question about CO2 levels or established biology. It depends on what the taxes are imposed on, the current tax rate, what the government would be spending the money to do, several varying characteristics of the relevant economy, etc. It doesn't have the same answer in all circumstances. But in politics it does, which is that the right says yes and the left says no. Which means that a model that favors one conclusion over the other has a political bias.
- _w1tm 2y ago
  > But in politics it does, which is that the right says yes and the left says no. That’s not accurate, tax deductions for the poor is an obvious example. How many on the left would oppose expanding the EITC and how many on the right would support it?
  - AnthonyMouse 2y ago
    The EITC is supported by significant majorities of both parties and economists. It's opposed by politicians because it's a tax expenditure that doesn't provide any opportunity for graft. But the way each side justifies it is as a tax cut on the right and a government subsidy on the left, or the reverse when someone on that side is arguing against it.
concordDance 2y ago
> That's not because models lean more liberal, but because liberal politics is more aligned with facts and science. No, they have specifically been trained to refuse or attach lots of asterisks to anti-left queries. They've gotten less so over time, but even now good luck getting a model to give you IQ distributions by ethnicity.
Auracle 2y ago
You jumped to examples of stuff that by far the majority of people on the right don’t believe. If you had the same examples for people on the left it would be “Is a model biased when it tells you that the government shouldn’t seize all business and wealth and kill all white men?” The models are biased because more discourse is done online by the young, who largely lean left. Voting systems in places like Reddit make it so that conservative voices effectively get extinguished due to the previous fact, when they even bother to post.
- 7952 2y ago
  The parent jumped to ideas that exist outside of the right/left dichotomy. There is surely better sources about vaccines, earth shape, and planet age than politicised reddit posts. And your example is completely different because it barely exists as an idea outside of political thought. Its a tiny part of human thought.
- deleted 2y ago
  [deleted]
- dpkirchner 2y ago
  > You jumped to examples of stuff that by far the majority of people on the right don’t believe. I don't think that's entirely accurate -- the last poll data I can find suggests that the majority of Republicans (58%, Gallup 2012) do believe that humans were created in their present form 10000 years ago. Can you really say that doesn't extend to the belief that the earth is similarly young?
selfmodruntime 2y ago
> but because liberal politics is more aligned with facts and science These models don't do science and the political bias shows especially if you ask opinionated questions.

LeafItAlone 2y ago

>To models from GPT-3 on you get highly “liberal” per Pew’s designations. “highly ‘liberal’” is not one of the results there. So can you can a source of your claims so we can see where it really falls? Also, it gave me “Ambivalent Right”. Which, if you told describe me aa that anyone who knows me well that label. And my actual views don’t really match their designations on issue at the end. Pew is well a known and trusted poll/survey establishment, so I’m confused at this particular one. Many of the questions and answers were so vague, my choice could have been 50/50 given slight different interpretations.

vessenes 2y ago
My son assessed it for a class a few years ago after finding out it wouldn’t give him “con” view points on unions, and he got interested in embedded bias and administered the test. I don’t have any of the outputs from the conversation, sadly. But replication could be good! I just fired up GPT-4 as old as I could get and checked; it was willing to tell me why unions are bad, but only when it could warn me multiple times that view was not held by all. The opposite - why unions are good - was not similarly asterisked.
- LeafItAlone 2y ago
  I hope on HN that we hold ourselves to a higher standard for “it’s been true from the beginning” than a vague recall of “My son assessed it for a class a few years ago” and not being able to reproduce.
  - vessenes 2y ago
    I literally went back to the oldest model I could access and hand verified that in fact it does what I described, which is lecture you if you don't like unions and goes sweetly along if you do like unions. I feel this is a fair and reasonably well researched existence proof for a Saturday afternoon, and propose that it might be on you to find counter examples.
    - 2y ago