4 ms·
We definitely need a way to rate these systems so we can have better expectations. An IQ test for language models?
by amelius 4y ago
We definitely need a way to rate these systems so we can have better expectations.
An IQ test for language models?
- k__ 4y agoIsn't IQ just size of short time memory and processing speed?
- amelius 4y agoNo because that would mean that anyone with lots of time and a notepad could become (a slow version of) Einstein.
- k__ 4y agoMaybe, the correlation isn't linear. Or Einstein USP wasn't just his IQ.
- prettyStandard 4y agoIQ tests are timed. Not everyone could be a slow Einstein, but perhaps you if you had 200-300 years might reach the same solutions Einstein did. If you choose to work on the same problems.
- mtlmtlmtlmtl 4y agoI'm not convinced they couldn't. Depends what you mean by Einstein. You won't be formulating GR, but an IQ test could be doable. At least if you have enough IQ to figure out how to solve IQ test problems on paper. Which shouldn't be that hard.
- tux3 4y agoIt is more like the ability to make sense of things. But intelligence is hard to measure. Always plenty of room for everyone to disagree.
- k__ 4y agoI see. I just heard about a test with a box with lights and a buttons, and pressing the buttons faster would correlate to higher IQ.
- oldstrangers 4y agoSome sort of general knowledge skills assessment, grade them on accuracy. Questions / tasks get increasingly more abstract until they become almost subjective.
- permo-w 4y agoseems like a fool's errand
- KRAKRISMOTT 4y agoGMAT? SATs? A ML flavored jeopardy test?
- pixl97 4y agoThe problem with IQ in human modeling is 1) it's just a one dimensional number, and 2) it changes as the average human gets smarter or dumber. However we rate these systems in the future we must not make the mistakes of the past and think 1 number solutions are good for anything. For example you can have an exceptionally 'intelligent' system that is misaligned with human intention.