5 ms·

famous cognitive psychology experiments that do replicate: IQ tests http://www.psychpage.com/learning/library/intell/mainstream.html http://www.psychpage.com/l

famous cognitive psychology experiments that do replicate: IQ tests http://www.psychpage.com/learning/library/intell/mainstream.html http://www.psychpage.com/learning/library/intell/mainstream.... in fact, the foundational statistical models considered the gold standard for statistics today were developed for this testing.

alphazard 1y ago
> in fact, the foundational statistical models considered the gold standard for statistics today were developed for this testing. The normal distribution predates the general factor model of IQ by hundreds of years.[0] You can try other distributions yourself, it's going to be hard to find one that better fits the existing IQ data than the normal (bell curve) distribution. [0] https://en.wikipedia.org/wiki/Normal_distribution#History https://en.wikipedia.org/wiki/Normal_distribution#History
- fsckboy 1y ago
  Darwin's cousin, Francis Galton, for whom the log-normal distribution is often called the Galton distribution, was among the first to investigate psychometrics. not realizing he was hundreds of years late to the game, he still went ahead and coined the term "median" more tidbits here https://en.wikipedia.org/wiki/Francis_Galton#Statistical_innovation_and_psychological_theory https://en.wikipedia.org/wiki/Francis_Galton#Statistical_inn...
astrange 1y ago
Survivorship bias. You can easily make someone's IQ test not replicate. (Hit them on the head really hard.)
dlcarrier 1y ago
I took an IQ test as a high school student, and one of the subtests involved placing a stack of shuffled pictures in chronological order. I had one series in the incorrect order, because I had no understanding of the typical behavior of snowfall. The test proctor said almost everyone she tested mixed that one up, because it doesn't snow in the area where I live. I have no doubt that IQ tests reproducibly measure the test takers ability to pass tests, as well as to perform in a society that the tests are based on. I think it's disingenuous to attribute IQ to intelligence as a whole though, and it is better understood as an indicator of cultural intelligence. I would expect that, for cultures who's members score below average on IQ tests from the US, an equivalent IQ test created within that culture would show average members of that culture scoring higher than average members of US culture.
- 3cKU 1y ago
  Raven's Progressive Matrices is often administered. Is that test culturally biased? Does that test measure only ability to take that test and nothing else?
  - teamonkey 1y ago
    Yes, it’s almost certainly linked to quality of schooling and exposure to those types of problems, amongst other things, see the Flynn Effect. https://en.wikipedia.org/wiki/Flynn_effect https://en.wikipedia.org/wiki/Flynn_effect
    - 3cKU 1y ago
      Access to schooling etc can't be the whole story: "black students from prosperous families tend to score higher in IQ than blacks from poor families, but they score no higher, on average, than whites from poor families".
      - tptacek 1y ago
        So IQ is malleable and SES-dependent and GxE interactions are real.
        3cKU 1y ago
        No. The first part of that quote is consistent with any hypothesis (G only, E only, G&E), i.e. cannot distinguish between them.
  - dlcarrier 1y ago
    Puzzle tests have their own problems. They're only effective at measuring puzzles solving abilities when they are novel, so retaking the test would lead to higher scores, and practicing even more so. They also only measure puzzle solving abilities which are necessary in some but not all applied intelligence tasks.
  - NalNezumi 1y ago
pessimizer 1y ago
What exactly are they meant to replicate other than other IQ tests? They don't make a statement about anything that is falsifiable, other than that if you give somebody who scores high on a test carefully designed and tested to match the results of previously given IQ tests when given to the same people, they'll tend to match the results that those people will get on other tests that were calibrated in the exact same way. If you're trying to say they replicate over the lifetime of the same person, I've had a 15 point swing between tests, out of the few I've taken. What did stay constant for me from age 10 to age 40 was my Myers-Briggs test (my dad was a metrics obsessive), and that's obvious horseshit. Consistency doesn't mean you're measuring what you claim to be measuring. edit: if it matters, scores were between 137 and 152, so exactly an entire standard deviation. That's like the difference between sub-Saharan and European that racists are always crowing about, but in the same person. IQ doesn't even personally replicate for me.
- teamonkey 1y ago
  You can prepare for IQ tests, just as you can for any other test, and you can get better at some of the problems in these tests the more you practice them, just as you get better at Sudoku puzzles the more you do them. Related: that brain is plastic and can adapt to challenges in different ways. https://www.scientificamerican.com/article/london-taxi-memory/ https://www.scientificamerican.com/article/london-taxi-memor...
- fsckboy 1y ago
  >What exactly are they meant to replicate other than other IQ tests? if a variety of different IQ tests sort the same people the same way, even though every question on the tests is different from the other tests, you have shown that the test is showing something about the subjects, not something about the tests. and that is replicable, and falsifiable. if you follow the same people over time and provide them with new tests, and they continue to sort in the same relative fashion, you have increased confidence that you are measuring something relatively fixed, not variable. For statistical significance (look it up) you don't draw conclusions on the basis of one person (or one Dad) but on population samples tested under standard conditions. this is like all study results published here, a thousand nerds who've never studied intelligence come up with a hundred objections to what was tested, assuming with arrogance that the people who specialized and did the work aren't considering what comes off the top of this nerd's head. Better qualified nerds did this work. >Myers-Briggs...'s obvious horseshit Myers Briggs is not complete horseshit, correlates closely to, but not as good a fit as, the generally accepted Big Five Factor system, the gold standard of personality tests: you should educate yourself a bit more. Myers-Briggs essentially tries to phrase everything in a postive way, where the Big Five separates and includes Neuroticism which is a more negative (for the person) trait. All these traits should be considered adaptive till proven otherwise, so resist the urge to judge.

alphazard 1y ago

> in fact, the foundational statistical models considered the gold standard for statistics today were developed for this testing. The normal distribution predates the general factor model of IQ by hundreds of years.[0] You can try other distributions yourself, it's going to be hard to find one that better fits the existing IQ data than the normal (bell curve) distribution. [0] https://en.wikipedia.org/wiki/Normal_distribution#History https://en.wikipedia.org/wiki/Normal_distribution#History

fsckboy 1y ago
Darwin's cousin, Francis Galton, for whom the log-normal distribution is often called the Galton distribution, was among the first to investigate psychometrics. not realizing he was hundreds of years late to the game, he still went ahead and coined the term "median" more tidbits here https://en.wikipedia.org/wiki/Francis_Galton#Statistical_innovation_and_psychological_theory https://en.wikipedia.org/wiki/Francis_Galton#Statistical_inn...

astrange 1y ago

Survivorship bias. You can easily make someone's IQ test not replicate. (Hit them on the head really hard.)

dlcarrier 1y ago

I took an IQ test as a high school student, and one of the subtests involved placing a stack of shuffled pictures in chronological order. I had one series in the incorrect order, because I had no understanding of the typical behavior of snowfall. The test proctor said almost everyone she tested mixed that one up, because it doesn't snow in the area where I live. I have no doubt that IQ tests reproducibly measure the test takers ability to pass tests, as well as to perform in a society that the tests are based on. I think it's disingenuous to attribute IQ to intelligence as a whole though, and it is better understood as an indicator of cultural intelligence. I would expect that, for cultures who's members score below average on IQ tests from the US, an equivalent IQ test created within that culture would show average members of that culture scoring higher than average members of US culture.

3cKU 1y ago
Raven's Progressive Matrices is often administered. Is that test culturally biased? Does that test measure only ability to take that test and nothing else?
- teamonkey 1y ago
  Yes, it’s almost certainly linked to quality of schooling and exposure to those types of problems, amongst other things, see the Flynn Effect. https://en.wikipedia.org/wiki/Flynn_effect https://en.wikipedia.org/wiki/Flynn_effect
  - 3cKU 1y ago
    Access to schooling etc can't be the whole story: "black students from prosperous families tend to score higher in IQ than blacks from poor families, but they score no higher, on average, than whites from poor families".
    - tptacek 1y ago
      So IQ is malleable and SES-dependent and GxE interactions are real.
      - 3cKU 1y ago
        No. The first part of that quote is consistent with any hypothesis (G only, E only, G&E), i.e. cannot distinguish between them.
- dlcarrier 1y ago
  Puzzle tests have their own problems. They're only effective at measuring puzzles solving abilities when they are novel, so retaking the test would lead to higher scores, and practicing even more so. They also only measure puzzle solving abilities which are necessary in some but not all applied intelligence tasks.
- NalNezumi 1y ago

pessimizer 1y ago

What exactly are they meant to replicate other than other IQ tests? They don't make a statement about anything that is falsifiable, other than that if you give somebody who scores high on a test carefully designed and tested to match the results of previously given IQ tests when given to the same people, they'll tend to match the results that those people will get on other tests that were calibrated in the exact same way. If you're trying to say they replicate over the lifetime of the same person, I've had a 15 point swing between tests, out of the few I've taken. What did stay constant for me from age 10 to age 40 was my Myers-Briggs test (my dad was a metrics obsessive), and that's obvious horseshit. Consistency doesn't mean you're measuring what you claim to be measuring. edit: if it matters, scores were between 137 and 152, so exactly an entire standard deviation. That's like the difference between sub-Saharan and European that racists are always crowing about, but in the same person. IQ doesn't even personally replicate for me.

teamonkey 1y ago
You can prepare for IQ tests, just as you can for any other test, and you can get better at some of the problems in these tests the more you practice them, just as you get better at Sudoku puzzles the more you do them. Related: that brain is plastic and can adapt to challenges in different ways. https://www.scientificamerican.com/article/london-taxi-memory/ https://www.scientificamerican.com/article/london-taxi-memor...
fsckboy 1y ago
>What exactly are they meant to replicate other than other IQ tests? if a variety of different IQ tests sort the same people the same way, even though every question on the tests is different from the other tests, you have shown that the test is showing something about the subjects, not something about the tests. and that is replicable, and falsifiable. if you follow the same people over time and provide them with new tests, and they continue to sort in the same relative fashion, you have increased confidence that you are measuring something relatively fixed, not variable. For statistical significance (look it up) you don't draw conclusions on the basis of one person (or one Dad) but on population samples tested under standard conditions. this is like all study results published here, a thousand nerds who've never studied intelligence come up with a hundred objections to what was tested, assuming with arrogance that the people who specialized and did the work aren't considering what comes off the top of this nerd's head. Better qualified nerds did this work. >Myers-Briggs...'s obvious horseshit Myers Briggs is not complete horseshit, correlates closely to, but not as good a fit as, the generally accepted Big Five Factor system, the gold standard of personality tests: you should educate yourself a bit more. Myers-Briggs essentially tries to phrase everything in a postive way, where the Big Five separates and includes Neuroticism which is a more negative (for the person) trait. All these traits should be considered adaptive till proven otherwise, so resist the urge to judge.