4 ms·
Nowhere do they discuss an actual IQ test! Last I checked, even the largest models fail hard on IQ tests, both visual and verbal. Of course they do find the u
by Straw 4y ago
Nowhere do they discuss an actual IQ test!
Last I checked, even the largest models fail hard on IQ tests, both visual and verbal.
Of course they do find the underlying issue that benchmarks often don't test as much as we think they do. I've been told that people used to think that a chess AI would have to match human intelligence!
- marcodiego 4y agoThe more we know about intelligence, the more we can see what we still don't know. Nevertheless, the progress has been really impressive. Couple a few GPT-like models with things like wolfram-alpha and you've got someone you can talk to and looks super-smart.
- mdp2021 4y ago> The more we know about intelligence, the more we can see what we still don't know Really? Surely not to the level of the faults in current ANN based AI. Honestly, I am sometimes feeling more "at home" with "expert systems" and "case based reasoning" - "pretense" was lower and the concept more promising, both criteria read in terms of honesty. > Couple a few GPT-like models with things like wolfram-alpha ... looks Still "looks" though, Marco, just looks, and toys should not be all we spend our resources on. At the point that the article depicts, it is "time to stop faking it and start implementing it".
- ravenstine 4y agoEven if they did, there's a possibility that AI being able to score high on IQ tests would demonstrate that IQ tests are of low validity. Research already shows that people with conditions like ADHD frequently score lower on IQ tests but show normal intelligence when given tests that aren't so adversarial to working memory. A standard IQ test (WAIS, for instance) can make an ADHD brain seem a whole standard deviation lower than what is likely their actual intelligence. In the opposite direction, an AI might be able to pass visual and verbal tests with flying colors but totally fail to comprehend the world and adapt to new problems, or actually understand anything. Visual and verbal skills can to an extent measure intelligence but neither of those factors are strictly correlated to intelligence. The more that a processor becomes specialized, whether it's a human brain or ML on computer hardware, the more the test will measure their specialization and less their fluid intelligence. Don't get me wrong, I think this is all interesting, but AI may soon make it apparent the flaw of using IQ as anything other than a measure of mental fitness.
- Straw 4y agoIQ in humans strongly correlates with almost any measure of performance you pick, so I think its definitely telling us some measure of capability even if not the whole story. I imagine people with ADHD do have trouble with some types of tasks (other than IQ tests), or we wouldn't think of it as a disorder? Of course, we can't necessarily expect this trend to continue for machines taking IQ tests! IQ tests may not test world understanding to a large degree, but definitely do test ability to adapt to new problems. Its hard to define understanding, so I prefer to stick with quantifiable measures of performance.
- mountainriver 4y agoActually modern models mostly outperform humans on IQ tests
- Straw 4y agoSource?
- mountainriver 4y agohttps://www.science.org/content/article/computers-ace-iq-tests-still-make-dumb-mistakes-can-different-tests-help https://www.science.org/content/article/computers-ace-iq-tes...
- Straw 4y agoIsn't this the original article? Which doesn't actually look at IQ tests.