5 ms·

Very fascinating. If we train on every piece of knowledge, then statistically that should average around 100IQ right? It's not perfect analogy, but the model is

Very fascinating. If we train on every piece of knowledge, then statistically that should average around 100IQ right? It's not perfect analogy, but the model is shooting more for consensus, a bit hive mind. It feels like to improve the quality of the model, we need to be more selective of training data.

dTal 3y ago
It's not entirely clear what precisely IQ is measuring. It's not a stretch that averaging the output of a bunch of 100 IQ people yields something better than what any one of them could have produced. Perhaps "there's only one way to be right, but many ways to be wrong".
- prettyStandard 3y ago
  We know what IQ is. It's the ability to think abstractly. Even more precisely it's the ability to do IQ tests.
  - QuiDortDine 3y ago
    This is a common belief that used to be true, but no longer is. IQ tests are reasonably good at measuring general intelligence, ie the g factor : https://en.wikipedia.org/wiki/G_factor_(psychometrics) https://en.wikipedia.org/wiki/G_factor_(psychometrics)
    - prettyStandard 3y ago
      I'm glad you posted this. Sometimes I try to describe this gfactor to people, saying "the best indicator of intelligence is any other intelligence". Putting a name on things helps people understand I am not a total bullshiter. Why do you say that this is a common belief that used to be true, but is now no longer true?
PartiallyTyped 3y ago
IQ is not measurable by knowledge, but by speed of execution and understanding through limited information.
- fnordpiglet 3y ago
  It’s not just that its the ability to reason in different modalities without specific knowledge, as well as others. Speed is certainly a part, but not the only part, of the IQ text.
- eloff 3y ago
  It’s both really if you look at IQ tests. It’s part of the reason immigrants will often score lower, they lack the culture or language specific knowledge being tested. IQ is a proxy for intelligence. People often use them interchangeably, but they’re not the same thing. Your statement is more valid for intelligence than IQ.
  - PartiallyTyped 3y ago
    Usually in such cases the person administering the test and the system takes it into account. A person with say math degrees will not take the same test as a person with a highschool diploma.
golol 3y ago
No. The prompt can condition the probabilities to correspond to a higher IQ output.
czbond 3y ago
> If we train on every piece of knowledge, then statistically that should average around 100IQ right? Interesting point - I'd never considered that.
jerrygenser 3y ago
Probably lower since the models have actually been lobotomized to a degree for safety purposes
luckylion 3y ago
Written knowledge will probably skew towards higher than average IQ. For starters, you've removed everyone who can't write, and everyone who isn't comfortable writing will be underrepresented.
kurthr 3y ago
I agree there is a lot of "averaging" going on, but here are two things that I think make it somewhat better than "100IQ". First, it seems like the people who talk the most about many narrow subjects actually have more interest and perhaps capability in them than the median person, and it will train mostly on the people who talk the most about particular token sets. Second, there is the unreasonable "wisdom of crowds" which works for some set of things (though it may no apply to LLMs). So it's weird, but I expect LLMs to give me better answers in narrow well discussed fields than broad but highly argued fields. I guess it's still really important to know how to ask the right question.
swid 3y ago
Or, it’s a simulator capable of producing output that is plausible for any level of intelligence up to the capabilities of the simulator. It could be capable of producing text similar to anyone from 0-140 IQ depending on the prompt. It will act “smarter” if the prompt indicates a smart person wrote the text.
tel 3y ago
I don't think this is true at all and it suggests a view on what GPT is that I think is very incorrect. Not to rag, it's just a common POV. GPT does not attempt to learn the average "text-generating system". It attempts to learn all of them. In superposition. Just imaging how bad it would be at predicting a scientific paper if it spoke (and reasoned?) like the average person. Such a model would not survive training. Training GPT does not make it behave more like the average, but instead widens and diversifies its probability distribution over all possible follow-ups. To make this clear, ask it to take on a persona. It can pretend to be all sorts of people, invented or sufficiently catalogued. I've had fun having it play multiple roles from podcasts I enjoy. Or it can pretend to be a fictional FTP server at Disney where poor authors have stored their unpublished screenplays. You can ask GPT to run dialogues with itself, playing both sides of the argument. There likely is some level of global persona at play, either through fine tuning or there being a general attractor basin of "helpful assistant" that we're all reinforcing. But there's no reason to believe that this looks anything like consensus or averaging.

dTal 3y ago

It's not entirely clear what precisely IQ is measuring. It's not a stretch that averaging the output of a bunch of 100 IQ people yields something better than what any one of them could have produced. Perhaps "there's only one way to be right, but many ways to be wrong".

prettyStandard 3y ago
We know what IQ is. It's the ability to think abstractly. Even more precisely it's the ability to do IQ tests.
- QuiDortDine 3y ago
  This is a common belief that used to be true, but no longer is. IQ tests are reasonably good at measuring general intelligence, ie the g factor : https://en.wikipedia.org/wiki/G_factor_(psychometrics) https://en.wikipedia.org/wiki/G_factor_(psychometrics)
  - prettyStandard 3y ago
    I'm glad you posted this. Sometimes I try to describe this gfactor to people, saying "the best indicator of intelligence is any other intelligence". Putting a name on things helps people understand I am not a total bullshiter. Why do you say that this is a common belief that used to be true, but is now no longer true?

PartiallyTyped 3y ago

IQ is not measurable by knowledge, but by speed of execution and understanding through limited information.

fnordpiglet 3y ago
It’s not just that its the ability to reason in different modalities without specific knowledge, as well as others. Speed is certainly a part, but not the only part, of the IQ text.
eloff 3y ago
It’s both really if you look at IQ tests. It’s part of the reason immigrants will often score lower, they lack the culture or language specific knowledge being tested. IQ is a proxy for intelligence. People often use them interchangeably, but they’re not the same thing. Your statement is more valid for intelligence than IQ.
- PartiallyTyped 3y ago
  Usually in such cases the person administering the test and the system takes it into account. A person with say math degrees will not take the same test as a person with a highschool diploma.

golol 3y ago

No. The prompt can condition the probabilities to correspond to a higher IQ output.

czbond 3y ago

> If we train on every piece of knowledge, then statistically that should average around 100IQ right? Interesting point - I'd never considered that.

jerrygenser 3y ago

Probably lower since the models have actually been lobotomized to a degree for safety purposes

luckylion 3y ago

Written knowledge will probably skew towards higher than average IQ. For starters, you've removed everyone who can't write, and everyone who isn't comfortable writing will be underrepresented.

kurthr 3y ago

I agree there is a lot of "averaging" going on, but here are two things that I think make it somewhat better than "100IQ". First, it seems like the people who talk the most about many narrow subjects actually have more interest and perhaps capability in them than the median person, and it will train mostly on the people who talk the most about particular token sets. Second, there is the unreasonable "wisdom of crowds" which works for some set of things (though it may no apply to LLMs). So it's weird, but I expect LLMs to give me better answers in narrow well discussed fields than broad but highly argued fields. I guess it's still really important to know how to ask the right question.

swid 3y ago

Or, it’s a simulator capable of producing output that is plausible for any level of intelligence up to the capabilities of the simulator. It could be capable of producing text similar to anyone from 0-140 IQ depending on the prompt. It will act “smarter” if the prompt indicates a smart person wrote the text.

tel 3y ago

I don't think this is true at all and it suggests a view on what GPT is that I think is very incorrect. Not to rag, it's just a common POV. GPT does not attempt to learn the average "text-generating system". It attempts to learn all of them. In superposition. Just imaging how bad it would be at predicting a scientific paper if it spoke (and reasoned?) like the average person. Such a model would not survive training. Training GPT does not make it behave more like the average, but instead widens and diversifies its probability distribution over all possible follow-ups. To make this clear, ask it to take on a persona. It can pretend to be all sorts of people, invented or sufficiently catalogued. I've had fun having it play multiple roles from podcasts I enjoy. Or it can pretend to be a fictional FTP server at Disney where poor authors have stored their unpublished screenplays. You can ask GPT to run dialogues with itself, playing both sides of the argument. There likely is some level of global persona at play, either through fine tuning or there being a general attractor basin of "helpful assistant" that we're all reinforcing. But there's no reason to believe that this looks anything like consensus or averaging.