4 ms·

>OpenAI supposed to handle difficult and uncomfortable statistics? I think the framing of this question is wrong, and is a limitation of the system that I've o

>OpenAI supposed to handle difficult and uncomfortable statistics? I think the framing of this question is wrong, and is a limitation of the system that I've only really seen Sam Altman point out. These "difficult and uncomfortable statistics" aren't accurate or well-researched but if you built an LLM on an open dataset you would find the people asking these questions have an agenda and on average will always lead to a certain kind of answer. The AI doesn't "understand" the question, it's just drawing from an incredibly large bank of "average answers". What do you expect the AI to answer with when you give it a loaded question that only really asked on a site like "stormfront.com"? I don't think this is the only area ChatGPT will fail, but I imagine there will be a layer of prompt engineering where you can get the AI to give you what you want by phrasing your question a certain way. To try and pass off these results as objective is just wrong given the propensity for ChatGPT to be confidently wrong.

pannSun 4y ago
> What do you expect the AI to answer with when you give it a loaded question that only really asked on a site like "stormfront.com"? Are such questions asked only on stormfront? The wiki article [1] cites quite a few studies on the subject, including by the APA. On the other hand, supposing that were true, and only stormfront and its ilk asked such questions. How can then respectable supposed experts claim we are all the same, if they didn't even check? [1] https://en.wikipedia.org/wiki/Race_and_intelligence#Test_scores https://en.wikipedia.org/wiki/Race_and_intelligence#Test_sco...
- ineptech 4y ago
  Honest answer since you seem new to this debate: there are tons of studies about IQ and race, most of them are heavily disputed, researching them is difficult because the field is so controversial, they're difficult to compare because there are so many different ways to define and measure "race" and "IQ", and the whole topic is so famously toxic and vitriolic that it tends to be a de facto banned topic on most fora, including this one.
  - apostacy 4y ago
    That is a completely acceptable answer for the AI to give. Just be honest and say that it is a controversial topic, and you don't want to discuss it. That is what a human would say. Don't make us play 20 questions. It is insulting to the person interacting with the AI.
    - crooked-v 4y ago
      Of course, this gets back to how all this vaunted GPT stuff is ultimately mostly a black box and all attempts to control its responses are just adding more black boxes wrapped around the black box.
apostacy 4y ago
This question is 100% acceptable and encouraged when it is presented as evidence of racism. For example, if someone says "The lower average IQ of black people is evidence of racism against black people". That is fine.
- nemothekid 4y ago
  I think you are missing my point; the problem isn't the question. There's a tendency to view ChatGPT as a some sort of "general intelligence" when it's probably better understood as a very powerful markov chain. When the data used to train this model is crawled from the internet where the relationships of IQ and Race and mostly limited to the more extremist part of the internet; when you bring up those questions extremist answers may get weighted higher because those types of conversations tend to be outright banned on other platforms. Therefore the model is likely to "confidently" give you answers that are probably extremist in nature and be very confident about it; which is a problem given that there's a lot of bunk information out there which is what I believe Scott Alexander is getting to here.
  - jfiwefwo 4y ago
TeMPOraL 4y ago
> To try and pass off these results as objective is just wrong given the propensity for ChatGPT to be confidently wrong. But nobody is trying to do that; the GP's question is in fact phrased precisely right. ChatGPT isn't a truth-distilling model - it's just abridging the Internet, with all its current biases. It's not being optimized for correctness at all, and the post-moderation OpenAI does isn't optimized for correctness either - it's optimized to make output non-controversial in the current media environment.

pannSun 4y ago

> What do you expect the AI to answer with when you give it a loaded question that only really asked on a site like "stormfront.com"? Are such questions asked only on stormfront? The wiki article [1] cites quite a few studies on the subject, including by the APA. On the other hand, supposing that were true, and only stormfront and its ilk asked such questions. How can then respectable supposed experts claim we are all the same, if they didn't even check? [1] https://en.wikipedia.org/wiki/Race_and_intelligence#Test_scores https://en.wikipedia.org/wiki/Race_and_intelligence#Test_sco...

ineptech 4y ago
Honest answer since you seem new to this debate: there are tons of studies about IQ and race, most of them are heavily disputed, researching them is difficult because the field is so controversial, they're difficult to compare because there are so many different ways to define and measure "race" and "IQ", and the whole topic is so famously toxic and vitriolic that it tends to be a de facto banned topic on most fora, including this one.
- apostacy 4y ago
  That is a completely acceptable answer for the AI to give. Just be honest and say that it is a controversial topic, and you don't want to discuss it. That is what a human would say. Don't make us play 20 questions. It is insulting to the person interacting with the AI.
  - crooked-v 4y ago
    Of course, this gets back to how all this vaunted GPT stuff is ultimately mostly a black box and all attempts to control its responses are just adding more black boxes wrapped around the black box.

apostacy 4y ago

This question is 100% acceptable and encouraged when it is presented as evidence of racism. For example, if someone says "The lower average IQ of black people is evidence of racism against black people". That is fine.

nemothekid 4y ago
I think you are missing my point; the problem isn't the question. There's a tendency to view ChatGPT as a some sort of "general intelligence" when it's probably better understood as a very powerful markov chain. When the data used to train this model is crawled from the internet where the relationships of IQ and Race and mostly limited to the more extremist part of the internet; when you bring up those questions extremist answers may get weighted higher because those types of conversations tend to be outright banned on other platforms. Therefore the model is likely to "confidently" give you answers that are probably extremist in nature and be very confident about it; which is a problem given that there's a lot of bunk information out there which is what I believe Scott Alexander is getting to here.
- jfiwefwo 4y ago

TeMPOraL 4y ago

> To try and pass off these results as objective is just wrong given the propensity for ChatGPT to be confidently wrong. But nobody is trying to do that; the GP's question is in fact phrased precisely right. ChatGPT isn't a truth-distilling model - it's just abridging the Internet, with all its current biases. It's not being optimized for correctness at all, and the post-moderation OpenAI does isn't optimized for correctness either - it's optimized to make output non-controversial in the current media environment.