3 ms·

LLMs are absolutely biased They are biased by the training dataset, which probably also reflects the biases of the people who select the training dataset They

LLMs are absolutely biased They are biased by the training dataset, which probably also reflects the biases of the people who select the training dataset They are biased by the system prompts that are embedded into every request to keep them on the rails They are even biased by the prompt that you write into them, which can lead them to incorrect conclusions if you design the prompt to lead them to it I think it is a very careless mistake to think of LLMs as unbiased or neutral in any way

MarcoDewey 2y ago
You are correct that the notion of LLMs being completely unbiased or neutral does not make sense due to how they are trained. Perhaps my title is even misleading if taken at face value. When I talk about "unbiased oracles" I am speaking in the context of black box testing. I'm not suggesting they are free from all forms of bias. Instead, the key distinction I'm trying to draw is their lack of implementation-level bias towards the specific code they are testing.
gwern 2y ago
LLMs are also heavily biased after chatbot tuning leads to mode-collapse. That's why you see the same verbal tics coming out of them, like the em-dashes or the 'twist ending' in the more recent 4os. And if LLMs really were unbiased, you'd expect better scaling when you tried to bruteforce code correctness. Training a 'test LLM' will just wind up inheriting a lot of the shared blindspots. They aren't independent of the implementation at all (just like humans are not independent, even when they didn't write the original, and didn't see it either; and this is why you can't simply throw _n_ programmers at a piece of code and be certain you got all the bugs, and why fuzzers will continue to rampage through code).
- stuaxo 2y ago
  The code correctness part is very true. I don't mind LLMs as part of a journey on code, but it shouldn't be the end product. I see something submitted by a colleague that doesn't fit the problem we have + tech well, go and ask an LLM and it outputs very similar code. It's clear at that point that they submitted heavily LLMs produced code without giving it the work it needed.
Muromec 2y ago
This and state actors target ai crawlers specifically ti pouson llms with propaganda
ninetyninenine 2y ago
No this is just a very overly pedantic and technical way of looking at it. First of all you'll note that all people are also biased by the Exact same reasoning. You know this. Everyone knows that all people are biased. This isn't something you don't know. So if every single intelligence, human or not is biased. What is this article truly talking about? The article is basically saying LLMs are LESS biased then humans. Why are LLMs less biased then humans? Well maybe because the training set in an LLM is less biased then the training set given to a human. This makes sense right? A human will be made more biased by his individual experience and his parents biases while an LLM is literally inundated with as many sources of textual information as possible with no attempt at bias due to the sheer volume of knowledge they are trying to shove in there. The article is basically referring to this. But you will note interestingly that LLMs bias towards textual data more. They understand the world as if they have no eyes and ears and only text. So the way they think reflects this bias. But in terms of textual knowledge I think we can all agree, they are Less biased then humans. Evidence: an LLM is not an atheist or a theist or an agnostic. But you, reader, are at the very least one of those three things.

MarcoDewey 2y ago

You are correct that the notion of LLMs being completely unbiased or neutral does not make sense due to how they are trained. Perhaps my title is even misleading if taken at face value. When I talk about "unbiased oracles" I am speaking in the context of black box testing. I'm not suggesting they are free from all forms of bias. Instead, the key distinction I'm trying to draw is their lack of implementation-level bias towards the specific code they are testing.

gwern 2y ago

LLMs are also heavily biased after chatbot tuning leads to mode-collapse. That's why you see the same verbal tics coming out of them, like the em-dashes or the 'twist ending' in the more recent 4os. And if LLMs really were unbiased, you'd expect better scaling when you tried to bruteforce code correctness. Training a 'test LLM' will just wind up inheriting a lot of the shared blindspots. They aren't independent of the implementation at all (just like humans are not independent, even when they didn't write the original, and didn't see it either; and this is why you can't simply throw _n_ programmers at a piece of code and be certain you got all the bugs, and why fuzzers will continue to rampage through code).

stuaxo 2y ago
The code correctness part is very true. I don't mind LLMs as part of a journey on code, but it shouldn't be the end product. I see something submitted by a colleague that doesn't fit the problem we have + tech well, go and ask an LLM and it outputs very similar code. It's clear at that point that they submitted heavily LLMs produced code without giving it the work it needed.

Muromec 2y ago

This and state actors target ai crawlers specifically ti pouson llms with propaganda

ninetyninenine 2y ago

No this is just a very overly pedantic and technical way of looking at it. First of all you'll note that all people are also biased by the Exact same reasoning. You know this. Everyone knows that all people are biased. This isn't something you don't know. So if every single intelligence, human or not is biased. What is this article truly talking about? The article is basically saying LLMs are LESS biased then humans. Why are LLMs less biased then humans? Well maybe because the training set in an LLM is less biased then the training set given to a human. This makes sense right? A human will be made more biased by his individual experience and his parents biases while an LLM is literally inundated with as many sources of textual information as possible with no attempt at bias due to the sheer volume of knowledge they are trying to shove in there. The article is basically referring to this. But you will note interestingly that LLMs bias towards textual data more. They understand the world as if they have no eyes and ears and only text. So the way they think reflects this bias. But in terms of textual knowledge I think we can all agree, they are Less biased then humans. Evidence: an LLM is not an atheist or a theist or an agnostic. But you, reader, are at the very least one of those three things.