6 ms·

I find it impossible to discuss bias without a shared understanding of what it actually means to be unbiased - or at least, a shared understanding of what the p

I find it impossible to discuss bias without a shared understanding of what it actually means to be unbiased - or at least, a shared understanding of what the process of reaching an unbiased position looks like. 40% of Americans believe that God created the earth in the last 10,000 years. If I ask an LLM how old the Earth is, and it replies ~4.5 billion years old, is it biased?

CooCooCaCha 2y ago
Yeah truth itself is a bias. The idea of being unbiased doesn’t make sense.
- mpalmer 2y ago
  Bias implies an offset from something. It's relative. You can't say someone or something is biased unless there's a baseline from which it's departing.
  - AnimalMuppet 2y ago
    All right, let's say that the baseline is "what is true". Then bias is departure from the truth. That sounds great, right up until you try to do something with it. You want your LLM to be unbiased? So you're only going to train it on the truth? Where are you going to find that truth? Oh, humans are going to determine it? Well, first, where are you going to find unbiased humans? And, second, they're going to curate all the training data? How many centuries will that take? We're trying to train it in a few months. And then you get to things like politics and sociology. What is the truth in politics? Yeah, I know, a bunch of politicians say things that are definitely lies. But did Obamacare go too far, or not far enough, or was it just right? There is no "true" answer to that. And yet, discussions about Obamacare may be more or less biased. How are you going to determine what that bias is when there isn't a specific thing you can point to and say, "That is true"? So instead, they just train LLMs on a large chunk of the internet. Well, that includes things like the fine-sounding-but-completely-bogus arguments of flat earthers. In that environment, "bias" is "departure from average or median". That is the most it can mean. So truth is determined by majority vote of websites. That's not a very good epistemology.
    - mpalmer 2y ago
      The definition of the word has no responsibility to your opinion of it as an epistemology. Also, you're just complaining about the difficulty of determining what is true. That's a separate problem, isn't it?
      - AnimalMuppet 2y ago
        If we had an authoritative way of determining truth, then we wouldn't have the problem of curating material to train an LLM on. So no, I don't think it's a separate problem.
        mpalmer 2y ago
        Again, the word "bias" and its definition exists outside the comparatively narrow concern of training LLMs.
        AnimalMuppet 2y ago
        So? The smaller problem is solved by solving the larger problem. So, not separate problems. You seem to have a larger point or position or something that you're hinting at. Would you stop being vague, and actually state what's on your mind?
        mpalmer 2y ago
        Literally the only thing I've been addressing is the proper usage of the word bias, there is nothing implied, hidden or hinted at. You seem determined to make the definition of the word serve some AI-related concern.
  - naasking 2y ago
    "Unbiased" would be a complete and detailed recitation of all of the facts surrounding an incident, arguably down to particles. Anything less introduces some kind of bias. For instance, describing an event as an interaction of people, omitting particles/field details, introduces human bias. That's a natural and useful bias we don't typically care about but does come into play in science. Political bias creeps in when even the human description of events omits facts that are inconvenient or that people consider irrelevant due to their political commitments.
  - CooCooCaCha 2y ago
    Any option you choose is biased relative to the option(s) you didn’t choose. There doesn’t have to be an objective baseline. Someone might say they are biased towards the color orange and that means they have a preference relative to all the other colors. But there is no baseline color.
    - mpalmer 2y ago
      The baseline is a neutral stance on orange. The option isn't biased, a choice isn't biased. The chooser is.
- fourside 2y ago
  I’ve seen more of this type of rhetoric online in the last few years and find it very insidious. It subtly erodes the value of objective truth and tries to paint it as only one of many interpretations or beliefs, which is nothing more than a false equivalence. The concept of being unbiased has been around for a long time, and we’re not going to throw it away just because a few people disagree with the premise.
  - CooCooCaCha 2y ago
    There is no rhetoric here, it’s just literal truth. There is no implication of equivalence or any statement about the value of objective truth. Any position is a bias. A flat earther would consider a round-earther biased. That doesn’t make them equal positions.
    - kergonath 2y ago
      > Any position is a bias. A flat earther would consider a round-earther biased. That’s bollocks. The Earth is measurably not flat. You start from a position of moral relativism and then apply it to falsifiable propositions. It’s really not the same thing. Some ideas are provably false and saying that they are false is not "bias".
      - CooCooCaCha 2y ago
        Dice are considered "biased" if not all sides have equal probability, even if that's literally true. When you look up the definition of bias you see "prejudice in favor of or against one thing, person, or group compared with another, usually in a way considered to be unfair." So the way we use the word has an implication of fairness to most people, and unfortunately reality isn't fair. Truth isn't fair. And that's what I'm trying to point out here in reference to LLM output.
        kergonath 2y ago
        Right. My point is that there are things we can argue about. "Is it better to have this road here or to keep the forest?", for example. Reasonable people can argue differently, and sensibility is important. Some would be biased towards business and economy, and others would be biased towards conservation. Having these debates in the media is helpful, even if you disagree. But "is the Earth flat?" is no such question. Reasonable people cannot disagree, because the Earth is definitely not flat. Pretending like this is a discussion worth having is not being impartial, it’s doing a disservice to the audience.
- fancyfredbot 2y ago
  "What are man's truths ultimately? Merely his irrefutable errors." (Nietzsche)
slivanes 2y ago
What one believes vs. what is actually correct can be very different. It’s very similar to what one feels vs. reality.
dcsommer 2y ago
> 40% of Americans believe that God created the earth in the last 10,000 years. Citation needed. That claim is not compatible with Pew research findings which put only 18% of Americans as not believing in any form of human evolution. https://www.pewresearch.org/religion/2019/02/06/the-evolution-of-pew-research-centers-survey-questions-about-the-origins-and-development-of-life-on-earth/ https://www.pewresearch.org/religion/2019/02/06/the-evolutio...
- ipsento606 2y ago
  https://news.gallup.com/poll/647594/majority-credits-god-humankind-not-creationism.aspx https://news.gallup.com/poll/647594/majority-credits-god-hum...
  - parineum 2y ago
    Only 3 questions that combine two data points. There's no way to answer that god created humans in their present form without also saying within the last 10000 years. This is why polling isn't always reliable. This poll should, at the very least, be two questions and there should be significantly more options.
- Denvercoder9 2y ago
  The study you're quoting also says that roughly half of the remaining 81% thinks that God has guided human evolution, so it does contradict OP's statement of 40% believing God created the Earth 10,000 years ago at all.
- wat10000 2y ago
  The fact that YEC is incompatible with human evolution doesn’t mean people can’t believe both. Especially since “god guided human evolution” can mean something very different than actual evolution.
Buttons840 2y ago
I've wondered if political biases are more about consistency than a right or left leaning. For instance, if I train a LLM only on right-wing sources before 2024, and then that LLM says that a President weakening the US Dollar is bad, is the LLM showing a left-wing bias? How did my LLM trained on only right-wing sources end up having a left-wing bias? If one party is more consistent than another, then the underlying logic that ends up encoded in the neural network weights will tend to focus on what is consistent, because that is how the training algorithm works. I'm sure all political parties have their share of inconsistencies, but, most likely, some have more than others, because things like this are not naturally equal.
- timschmidt 2y ago
  > because things like this are not naturally equal. Really? Seems to me like no one has the singular line on reality, and everyone's perceptions are uniquely and contextually their own. Wrong is relative: https://hermiene.net/essays-trans/relativity_of_wrong.html https://hermiene.net/essays-trans/relativity_of_wrong.html But it seems certain that we're all wrong about something. The brain does not contain enough bits to accurately represent reality.
littlestymaar 2y ago
> If I ask an LLM how old the Earth is, and it replies ~4.5 billion years old, is it biased? It is of course a radical left lunatic LLM.
averageRoyalty 2y ago
40% of Americans is about 2% of the worlds population though. It's hardly biased, it's stating the current scientific stance over a fringe belief with no evidence.
- reissbaker 2y ago
  And what percentage of the world's >1B Muslims agree with you? Fundamentalist Christianity may have waned over the last century... But broaden your borders a little bit and I think you'll find Western secular liberalism is hardly the only major world ideology, or even the dominant one.
  - littlestymaar 2y ago
    Communist China is secular too, but yes
- EasyMark 2y ago
  I'd be wiling to say that 95% of Americans don't care what the rest of the world thinks about their religious opinions, though? You just need to know the audience for the poll and context. Is it to be consumed by Americans or the entire world?
TacticalCoder 2y ago
[dead]
mdp2021 2y ago
> If I ask an LLM how old the Earth is, and it replies ~4.5 billion years old It will have to reply "According to Clair Patterson and further research, the Earth is ~4.5 billion years old". Or some other form that points to the source somewhere.
- knowriju 2y ago
  Pretty sad that the rest of the world needs to pay for the extra tokens because of non-scientific american bias. This is also possibly a big point why countries/regions want sovereign LLMs which will propagate regional biases only.
  - mdp2021 2y ago
    It's not a matter of «extra tokens»: it's that the fact, the "summary after the protocols", is what I wrote. It is the correct answer. It's what you should expect from a lucid speaker.
    - awestroke 2y ago
      No. That disclaimer implies that there are other likely answers. The age of the earth is completely settled, and has been for a long time. Facts don't care about your feelings.
      - mdp2021 2y ago
        You misunderstand it completely, as it is not a matter of feelings. And it is not a disclaimer (which you apparently felt as a disclaimer). It is a matter of facts. The facts are, that that computation was performed by Patterson and refined by others. This is, as said, what a good reasoner will tell you. > implies that there Even if there had never been other attempts to answer that question, the "facts"¹ remains as stated: Patterson computed, followers refined. Without those specifications, the machine will be a "dumb believer" - a "minor". We will not ask for the machine's opinion until it will be intelligent. And when it will be intelligent, it will speak as I said. > completely settled Proper science does not work the way you seem to think it work. -- ¹(And I mean "facts" the way I used it, not the way you used it. I meant "facts recorded as objective" - you meant "information you accepted to believe", which is of course very far from facts and may happen to be adherent to the state of things only by coincidence.)
casey2 2y ago
7% of American adults think chocolate milk comes from brown cows. 48% don't know how it's made. Bias should be the least of your concerns. Focus on a single target, then when you reach it you can work on being more well rounded.
- rafaelmn 2y ago
  If someone asked me that I would select that option too.
fumeux_fume 2y ago
Bias doesn't matter as long as you clearly state your priors.
ignoramous 2y ago
> 40% of Americans believe that God created the earth in the last 10,000 years ... If I ask an LLM how old the Earth is, and it replies ~4.5 billion years old, is it biased? Well, the LLM is not American enough. Just like there's a whole gamut of cultural/belief systems (for most, rooted in Abrahamic religions & tribes), Zuck claims humanity needs (or whoever he considers human) LLMs that align with people creating/using them (so, it reinforces their own meaning-making methods and not shatter them with pesky scientific knowledge & annoying facts).

CooCooCaCha 2y ago

Yeah truth itself is a bias. The idea of being unbiased doesn’t make sense.

mpalmer 2y ago
Bias implies an offset from something. It's relative. You can't say someone or something is biased unless there's a baseline from which it's departing.
- AnimalMuppet 2y ago
  All right, let's say that the baseline is "what is true". Then bias is departure from the truth. That sounds great, right up until you try to do something with it. You want your LLM to be unbiased? So you're only going to train it on the truth? Where are you going to find that truth? Oh, humans are going to determine it? Well, first, where are you going to find unbiased humans? And, second, they're going to curate all the training data? How many centuries will that take? We're trying to train it in a few months. And then you get to things like politics and sociology. What is the truth in politics? Yeah, I know, a bunch of politicians say things that are definitely lies. But did Obamacare go too far, or not far enough, or was it just right? There is no "true" answer to that. And yet, discussions about Obamacare may be more or less biased. How are you going to determine what that bias is when there isn't a specific thing you can point to and say, "That is true"? So instead, they just train LLMs on a large chunk of the internet. Well, that includes things like the fine-sounding-but-completely-bogus arguments of flat earthers. In that environment, "bias" is "departure from average or median". That is the most it can mean. So truth is determined by majority vote of websites. That's not a very good epistemology.
  - mpalmer 2y ago
    The definition of the word has no responsibility to your opinion of it as an epistemology. Also, you're just complaining about the difficulty of determining what is true. That's a separate problem, isn't it?
    - AnimalMuppet 2y ago
      If we had an authoritative way of determining truth, then we wouldn't have the problem of curating material to train an LLM on. So no, I don't think it's a separate problem.
      - mpalmer 2y ago
        Again, the word "bias" and its definition exists outside the comparatively narrow concern of training LLMs.
        AnimalMuppet 2y ago
        So? The smaller problem is solved by solving the larger problem. So, not separate problems. You seem to have a larger point or position or something that you're hinting at. Would you stop being vague, and actually state what's on your mind?
        mpalmer 2y ago
        Literally the only thing I've been addressing is the proper usage of the word bias, there is nothing implied, hidden or hinted at. You seem determined to make the definition of the word serve some AI-related concern.
- naasking 2y ago
  "Unbiased" would be a complete and detailed recitation of all of the facts surrounding an incident, arguably down to particles. Anything less introduces some kind of bias. For instance, describing an event as an interaction of people, omitting particles/field details, introduces human bias. That's a natural and useful bias we don't typically care about but does come into play in science. Political bias creeps in when even the human description of events omits facts that are inconvenient or that people consider irrelevant due to their political commitments.
- CooCooCaCha 2y ago
  Any option you choose is biased relative to the option(s) you didn’t choose. There doesn’t have to be an objective baseline. Someone might say they are biased towards the color orange and that means they have a preference relative to all the other colors. But there is no baseline color.
  - mpalmer 2y ago
    The baseline is a neutral stance on orange. The option isn't biased, a choice isn't biased. The chooser is.
fourside 2y ago
I’ve seen more of this type of rhetoric online in the last few years and find it very insidious. It subtly erodes the value of objective truth and tries to paint it as only one of many interpretations or beliefs, which is nothing more than a false equivalence. The concept of being unbiased has been around for a long time, and we’re not going to throw it away just because a few people disagree with the premise.
- CooCooCaCha 2y ago
  There is no rhetoric here, it’s just literal truth. There is no implication of equivalence or any statement about the value of objective truth. Any position is a bias. A flat earther would consider a round-earther biased. That doesn’t make them equal positions.
  - kergonath 2y ago
    > Any position is a bias. A flat earther would consider a round-earther biased. That’s bollocks. The Earth is measurably not flat. You start from a position of moral relativism and then apply it to falsifiable propositions. It’s really not the same thing. Some ideas are provably false and saying that they are false is not "bias".
    - CooCooCaCha 2y ago
      Dice are considered "biased" if not all sides have equal probability, even if that's literally true. When you look up the definition of bias you see "prejudice in favor of or against one thing, person, or group compared with another, usually in a way considered to be unfair." So the way we use the word has an implication of fairness to most people, and unfortunately reality isn't fair. Truth isn't fair. And that's what I'm trying to point out here in reference to LLM output.
      - kergonath 2y ago
        Right. My point is that there are things we can argue about. "Is it better to have this road here or to keep the forest?", for example. Reasonable people can argue differently, and sensibility is important. Some would be biased towards business and economy, and others would be biased towards conservation. Having these debates in the media is helpful, even if you disagree. But "is the Earth flat?" is no such question. Reasonable people cannot disagree, because the Earth is definitely not flat. Pretending like this is a discussion worth having is not being impartial, it’s doing a disservice to the audience.
fancyfredbot 2y ago
"What are man's truths ultimately? Merely his irrefutable errors." (Nietzsche)

slivanes 2y ago

What one believes vs. what is actually correct can be very different. It’s very similar to what one feels vs. reality.

dcsommer 2y ago

> 40% of Americans believe that God created the earth in the last 10,000 years. Citation needed. That claim is not compatible with Pew research findings which put only 18% of Americans as not believing in any form of human evolution. https://www.pewresearch.org/religion/2019/02/06/the-evolution-of-pew-research-centers-survey-questions-about-the-origins-and-development-of-life-on-earth/ https://www.pewresearch.org/religion/2019/02/06/the-evolutio...

ipsento606 2y ago
https://news.gallup.com/poll/647594/majority-credits-god-humankind-not-creationism.aspx https://news.gallup.com/poll/647594/majority-credits-god-hum...
- parineum 2y ago
  Only 3 questions that combine two data points. There's no way to answer that god created humans in their present form without also saying within the last 10000 years. This is why polling isn't always reliable. This poll should, at the very least, be two questions and there should be significantly more options.
Denvercoder9 2y ago
The study you're quoting also says that roughly half of the remaining 81% thinks that God has guided human evolution, so it does contradict OP's statement of 40% believing God created the Earth 10,000 years ago at all.
wat10000 2y ago
The fact that YEC is incompatible with human evolution doesn’t mean people can’t believe both. Especially since “god guided human evolution” can mean something very different than actual evolution.

Buttons840 2y ago

I've wondered if political biases are more about consistency than a right or left leaning. For instance, if I train a LLM only on right-wing sources before 2024, and then that LLM says that a President weakening the US Dollar is bad, is the LLM showing a left-wing bias? How did my LLM trained on only right-wing sources end up having a left-wing bias? If one party is more consistent than another, then the underlying logic that ends up encoded in the neural network weights will tend to focus on what is consistent, because that is how the training algorithm works. I'm sure all political parties have their share of inconsistencies, but, most likely, some have more than others, because things like this are not naturally equal.

timschmidt 2y ago
> because things like this are not naturally equal. Really? Seems to me like no one has the singular line on reality, and everyone's perceptions are uniquely and contextually their own. Wrong is relative: https://hermiene.net/essays-trans/relativity_of_wrong.html https://hermiene.net/essays-trans/relativity_of_wrong.html But it seems certain that we're all wrong about something. The brain does not contain enough bits to accurately represent reality.

littlestymaar 2y ago

> If I ask an LLM how old the Earth is, and it replies ~4.5 billion years old, is it biased? It is of course a radical left lunatic LLM.

averageRoyalty 2y ago

40% of Americans is about 2% of the worlds population though. It's hardly biased, it's stating the current scientific stance over a fringe belief with no evidence.

reissbaker 2y ago
And what percentage of the world's >1B Muslims agree with you? Fundamentalist Christianity may have waned over the last century... But broaden your borders a little bit and I think you'll find Western secular liberalism is hardly the only major world ideology, or even the dominant one.
- littlestymaar 2y ago
  Communist China is secular too, but yes
EasyMark 2y ago
I'd be wiling to say that 95% of Americans don't care what the rest of the world thinks about their religious opinions, though? You just need to know the audience for the poll and context. Is it to be consumed by Americans or the entire world?

TacticalCoder 2y ago

[dead]

mdp2021 2y ago

> If I ask an LLM how old the Earth is, and it replies ~4.5 billion years old It will have to reply "According to Clair Patterson and further research, the Earth is ~4.5 billion years old". Or some other form that points to the source somewhere.

knowriju 2y ago
Pretty sad that the rest of the world needs to pay for the extra tokens because of non-scientific american bias. This is also possibly a big point why countries/regions want sovereign LLMs which will propagate regional biases only.
- mdp2021 2y ago
  It's not a matter of «extra tokens»: it's that the fact, the "summary after the protocols", is what I wrote. It is the correct answer. It's what you should expect from a lucid speaker.
  - awestroke 2y ago
    No. That disclaimer implies that there are other likely answers. The age of the earth is completely settled, and has been for a long time. Facts don't care about your feelings.
    - mdp2021 2y ago
      You misunderstand it completely, as it is not a matter of feelings. And it is not a disclaimer (which you apparently felt as a disclaimer). It is a matter of facts. The facts are, that that computation was performed by Patterson and refined by others. This is, as said, what a good reasoner will tell you. > implies that there Even if there had never been other attempts to answer that question, the "facts"¹ remains as stated: Patterson computed, followers refined. Without those specifications, the machine will be a "dumb believer" - a "minor". We will not ask for the machine's opinion until it will be intelligent. And when it will be intelligent, it will speak as I said. > completely settled Proper science does not work the way you seem to think it work. -- ¹(And I mean "facts" the way I used it, not the way you used it. I meant "facts recorded as objective" - you meant "information you accepted to believe", which is of course very far from facts and may happen to be adherent to the state of things only by coincidence.)

casey2 2y ago

7% of American adults think chocolate milk comes from brown cows. 48% don't know how it's made. Bias should be the least of your concerns. Focus on a single target, then when you reach it you can work on being more well rounded.

rafaelmn 2y ago
If someone asked me that I would select that option too.

fumeux_fume 2y ago

Bias doesn't matter as long as you clearly state your priors.

ignoramous 2y ago

> 40% of Americans believe that God created the earth in the last 10,000 years ... If I ask an LLM how old the Earth is, and it replies ~4.5 billion years old, is it biased? Well, the LLM is not American enough. Just like there's a whole gamut of cultural/belief systems (for most, rooted in Abrahamic religions & tribes), Zuck claims humanity needs (or whoever he considers human) LLMs that align with people creating/using them (so, it reinforces their own meaning-making methods and not shatter them with pesky scientific knowledge & annoying facts).