4 ms·

If it’s not trained to be biased towards Elon Musk is always right or whatever, I think it will be much less of a problem than humans. Humans are VERY politica

by kelipso 8mo ago

If it’s not trained to be biased towards Elon Musk is always right or whatever, I think it will be much less of a problem than humans. Humans are VERY political creatures. A hint that their side thinks X is true and humans will reorganize their entire philosophy and worldview retroactively to rationalize X. LLMs don’t have such instincts and can potentially be instructed to present or evaluate the primary, if opposing, arguments. So you architecturally predisposed argument, I don’t think is true.

Rebelgecko 8mo ago
There was a whole collection of posts where Grok says stuff like "Elon Musk is more athletic than LeBron James".
- kelipso 8mo ago
  Well yeah, probably because it was instructed to praise Musk. Doesn’t imply that there can exist no LLM that doesn’t do that…
  - kbenson 8mo ago
    Why would we assume an LLM, even one that doesn't appear to have a bias like that built in, doesn't have one? Just because we can't identify it immediately, does not mean it doesn't exist. Groups of people can and do have bias, but I also think it's much harder to control the outcome (for better or worse) when inputs are more diverse.
    - kelipso 8mo ago
      There very likely is existing research into evaluating political bias in LLMs, not too sure, but I do think it's very possible to have an evaluation framework that could test LLMs for political bias and other biases. Once we have such a test and an LLM that passes it, we can be certain (to some confidence, for some topics, for some biases, etc etc) that the LLM won't be biased. For humans, there is no such guarantee. The humans can lie, change their mind, etc. See Wikipedia, where they talk about how they are not biased, they have many processes that ensure no biases, blah blah blah, and it turns out they are massively biased, what a surprise. Of course, who evaluates the evaluators/evaluation frameworks comes into play but that's a much easier problem.
      - greggoB 8mo ago
        > See Wikipedia, where they talk about how they are not biased, they have many processes that ensure no biases, blah blah blah, and it turns out they are massively biased, what a surprise. It's clear you have some unfounded issue with Wikipedia. They are not "massively biased", that's a talking point propelled primarily by the right/far right because of a desire to rewrite history to match their ideological needs. Saying "there very likely is existing research into evaluating political bias in LLMs" essentially means very little because 1. By your own admission you can't even say for sure that such research is actually happening (it probably is, but you admit you don't actually know) 2. There is no guarantee such research will lead to anywhere anytime soon 3. Even if it does, how does a means of evaluating bias in LLMs provide a path to eliminating it?
        kelipso 8mo ago
        It’s not “unfounded”. Wikipedia is biased and saying that’s “propaganda” or a result of propaganda is a nonsense non-argument. > Saying "there very likely […] What’s with this nitpicky stuff. A simple google search shows there’s tons of research in LLM political bias evaluation. > There is no guarantee [..] path to eliminating it? It’s research. Sure there’s no guarantee but given progress in LLM, I would be optimistic rather than pessimistic.
        greggoB 8mo ago
        > It’s not “unfounded”. Wikipedia is biased and saying that’s “propaganda” or a result of propaganda is a nonsense non-argument. It specifically is unfounded if you have no credible sources to back it up. "Trust me bro" doesn't qualify. > What’s with this nitpicky stuff This is HN, you should be prepared to validate what you're saying, or accept you'll be challenged to do so. > It’s research. Sure there’s no guarantee but given progress in LLM, I would be optimistic rather than pessimistic. This is a really poor argument when advocating it (AI) as a viable replacement for the status quo.
        kelipso 8mo ago
woodruffw 8mo ago
> LLMs don’t have such instincts and can potentially be instructed to present or evaluate the primary, if opposing, arguments. It seems essentially wrong to anthropomorphize LLMs as having instincts or not. What they have is training, and there's currently no widely accepted test for determining whether a "fair" evaluation from an LLM stems from biases during training. (It should be clear that humans don't need to be unpolitical; what they need to be is accountable. Wikipedia appears to be at least passably competent at making its human editors accountable to each other.)
- kelipso 8mo ago
  I said LLM doesn’t have such instinct but yeah I agree there should be less anthropomorphizing and more evaluation based framing when talking about LLMs, but it’s not that easy in regular discussions. About Wikipedia, there is obvious bias and cliques there as has been discussed in this thread and HN for many years, not to mention the its bias is reason that Grokipedia came about in the first place.
  - greggoB 8mo ago
    > not to mention the its bias is reason that Grokipedia came about in the first place. No, the reason is Musk didn't like that the Wikipedia article on him added the factual record of him doing a Nazi salute [0] [0] https://www.lemonde.fr/en/united-states/article/2025/01/23/musk-wikipedia-founder-in-row-over-how-to-describe-nazi-salute_6737328_133.html https://www.lemonde.fr/en/united-states/article/2025/01/23/m...
  - croon 8mo ago
    > not to mention the its bias is reason that Grokipedia came about in the first place. claimed bias != bias It may have bias, or it may not, but the only reason Grokipedia exists is because Musk doesn't like the contents of Wikipedia.
  - rbanffy 8mo ago
    > bias is reason that Grokipedia came about in the first place. You are correct, but only in the sense that Musk was unable to impose his own biases upon Wikipedia, so he had to make one where he can tune bias to whatever is convenient at the moment.
Starman_Jones 8mo ago
Why do you think that an LLM wouldn't have biases?

by kelipso 8mo ago

Rebelgecko 8mo ago

There was a whole collection of posts where Grok says stuff like "Elon Musk is more athletic than LeBron James".

kelipso 8mo ago
Well yeah, probably because it was instructed to praise Musk. Doesn’t imply that there can exist no LLM that doesn’t do that…
- kbenson 8mo ago
  Why would we assume an LLM, even one that doesn't appear to have a bias like that built in, doesn't have one? Just because we can't identify it immediately, does not mean it doesn't exist. Groups of people can and do have bias, but I also think it's much harder to control the outcome (for better or worse) when inputs are more diverse.
  - kelipso 8mo ago
    There very likely is existing research into evaluating political bias in LLMs, not too sure, but I do think it's very possible to have an evaluation framework that could test LLMs for political bias and other biases. Once we have such a test and an LLM that passes it, we can be certain (to some confidence, for some topics, for some biases, etc etc) that the LLM won't be biased. For humans, there is no such guarantee. The humans can lie, change their mind, etc. See Wikipedia, where they talk about how they are not biased, they have many processes that ensure no biases, blah blah blah, and it turns out they are massively biased, what a surprise. Of course, who evaluates the evaluators/evaluation frameworks comes into play but that's a much easier problem.
    - greggoB 8mo ago
      > See Wikipedia, where they talk about how they are not biased, they have many processes that ensure no biases, blah blah blah, and it turns out they are massively biased, what a surprise. It's clear you have some unfounded issue with Wikipedia. They are not "massively biased", that's a talking point propelled primarily by the right/far right because of a desire to rewrite history to match their ideological needs. Saying "there very likely is existing research into evaluating political bias in LLMs" essentially means very little because 1. By your own admission you can't even say for sure that such research is actually happening (it probably is, but you admit you don't actually know) 2. There is no guarantee such research will lead to anywhere anytime soon 3. Even if it does, how does a means of evaluating bias in LLMs provide a path to eliminating it?
      - kelipso 8mo ago
        It’s not “unfounded”. Wikipedia is biased and saying that’s “propaganda” or a result of propaganda is a nonsense non-argument. > Saying "there very likely […] What’s with this nitpicky stuff. A simple google search shows there’s tons of research in LLM political bias evaluation. > There is no guarantee [..] path to eliminating it? It’s research. Sure there’s no guarantee but given progress in LLM, I would be optimistic rather than pessimistic.
        greggoB 8mo ago
        > It’s not “unfounded”. Wikipedia is biased and saying that’s “propaganda” or a result of propaganda is a nonsense non-argument. It specifically is unfounded if you have no credible sources to back it up. "Trust me bro" doesn't qualify. > What’s with this nitpicky stuff This is HN, you should be prepared to validate what you're saying, or accept you'll be challenged to do so. > It’s research. Sure there’s no guarantee but given progress in LLM, I would be optimistic rather than pessimistic. This is a really poor argument when advocating it (AI) as a viable replacement for the status quo.
        kelipso 8mo ago

woodruffw 8mo ago

> LLMs don’t have such instincts and can potentially be instructed to present or evaluate the primary, if opposing, arguments. It seems essentially wrong to anthropomorphize LLMs as having instincts or not. What they have is training, and there's currently no widely accepted test for determining whether a "fair" evaluation from an LLM stems from biases during training. (It should be clear that humans don't need to be unpolitical; what they need to be is accountable. Wikipedia appears to be at least passably competent at making its human editors accountable to each other.)

kelipso 8mo ago
I said LLM doesn’t have such instinct but yeah I agree there should be less anthropomorphizing and more evaluation based framing when talking about LLMs, but it’s not that easy in regular discussions. About Wikipedia, there is obvious bias and cliques there as has been discussed in this thread and HN for many years, not to mention the its bias is reason that Grokipedia came about in the first place.
- greggoB 8mo ago
  > not to mention the its bias is reason that Grokipedia came about in the first place. No, the reason is Musk didn't like that the Wikipedia article on him added the factual record of him doing a Nazi salute [0] [0] https://www.lemonde.fr/en/united-states/article/2025/01/23/musk-wikipedia-founder-in-row-over-how-to-describe-nazi-salute_6737328_133.html https://www.lemonde.fr/en/united-states/article/2025/01/23/m...
- croon 8mo ago
  > not to mention the its bias is reason that Grokipedia came about in the first place. claimed bias != bias It may have bias, or it may not, but the only reason Grokipedia exists is because Musk doesn't like the contents of Wikipedia.
- rbanffy 8mo ago
  > bias is reason that Grokipedia came about in the first place. You are correct, but only in the sense that Musk was unable to impose his own biases upon Wikipedia, so he had to make one where he can tune bias to whatever is convenient at the moment.

Starman_Jones 8mo ago

Why do you think that an LLM wouldn't have biases?