5 ms·
I don't think you can say an AI trained using RLHF - such as ChatGPT is - is really "simply reflecting the data it was trained on". ChatGPT was first trained on
by pcstl 4y ago
I don't think you can say an AI trained using RLHF - such as ChatGPT is - is really "simply reflecting the data it was trained on". ChatGPT was first trained on a load of data, then it was updated to act in specific ways based on feedback from humans who "nudged" it the way they wanted it to go.
- wizeman 4y agoAre those humans that nudged it representative of the population? Or were they mostly "woke" Silicon Valley employees? (not to dismiss woke Silicon Valley employees, I'm just saying their opinions are not representative of the entire population).
- dorchadas 4y agoThere's also bias in the data itself. That's the difficult thing to avoid. Even down to how we phrase a question, who we collect the data from, it all introduces a bias unless we're literally harvesting all data from every human being and using that for our models. There's no way to get rid of the bias, even if we take out the nudges.
- wizeman 4y agoHow about you select a representative (i.e. random and statistically significant) sample of the population and then ask them their opinions about certain (especially controversial) parts of your data, and then weigh your data according to these opinions? That's just an idea that occurred to me (in 30 seconds of thought) which could probably make the training data significantly more unbiased. But I'm sure there are research scientists who can come up with better methods for sampling data in a more unbiased fashion. Note that this is not an all or nothing approach. Your training data could presumably be 100% biased or 0% biased, but also any value in-between. The goal is to try to make it as close to 0% biased as feasible, given whatever effort you're comfortable expending.
- freejazz 4y agoI'm not sure why you are conflating bias with the "the statistical likelihood that a belief is held by someone". They have nothing to do with each other.
- BongoMcCat 4y agoOk, now I have read several of your replies in this thread that are basically all arguing the same thing, so, I'm basically replying to more than just this one post. When you say "the entire population", you mean the entire population of the country "USA", right? Because as someone from another continent, it seems like there is a very specific set of opinions that you want included. You use terms like "the other side" of the political discourse, which to me, reduces the set of opinion to two specific sets of opinions, namely the two sets represented by the two major parties in the american two party system. As someone from "the outside", this seems like a very narrow view of reality, even if you managed to get your "unbiased AI", that represents both major american political parties, it will still seem like a very narrow and biased AI to someone from the outside of that. Also, what exactly is the goal of a conversational AI? is it just to make a conversation with it seem like a conversation with an average american? If so, why would anyone want that? Wouldn't it be of more value to have an AI that could tell me what people with knowledge of a subject thinks of it, rather than what random people think?
- freejazz 4y agoAh, yes, someone with blue hair. Of course. They are often, if not always, the culprit!