6 ms·

Ugh, this study selected its "superior iq" group by surveying Mensa members. Study (https://www.sciencedirect.com/science/article/pii/S0160289616303324#! https

Ugh, this study selected its "superior iq" group by surveying Mensa members. Study (https://www.sciencedirect.com/science/article/pii/S0160289616303324#! https://www.sciencedirect.com/science/article/pii/S016028961...) makes no mention of this selection bias, though even the article does.

deleted 7y ago
[deleted]
deleted 7y ago
[deleted]
anewguy9000 7y ago
any sampling method is 'biased', thats why its a 'sample'
- bentona 7y ago
  This study not only selected for people who go out of their way to join a society based on high IQ, it also selected for people who open & complete email surveys. Yes, many practical sampling methods are biased, but it is not difficult for me to imagine that any result here is caused entirely by these biases.
  - anewguy9000 7y ago
    i agree with your criticism in general -- actually it applies to virtually all psychological and medical studies. but if you think the physical disabilities are better explained by something else to do with this population sample than their iq, you could very clearly publish this theory
- scarmig 7y ago
  Mensa members are... Not representative of people with high IQs.
- mindblock 7y ago
  Yeah I don’t think smart people pay a membership fee to an organization to be certified as smart.
- mlyle 7y ago
  This is what my 4th grader just learned is called a "convenience sample" and all the ways it can be biased... :P If he can understand the shortcomings, then so can the rest of us.
- richdougherty 7y ago
  If you take a look at the definition and you'll see it's possible to have an unbiased sample. https://en.m.wikipedia.org/wiki/Sample_(statistics) https://en.m.wikipedia.org/wiki/Sample_(statistics)
  - anewguy9000 7y ago
    how do you randomly sample high iq? by definition the population would have had to have taken a test. so already, even if the study looked at every known high iq individual in the world, it would still be biased -- for those with academic backgrounds, psychological histories, criminal backgrounds, and whatever else leads one to take an iq test. nb i am not advocating for this particular study nor their methodology.
    - zingermc 7y ago
      You would have to sample a lot of people (yes, there is a selection bias in the people that participate) and give all of them IQ tests.
    - dahart 7y ago
      What you meant to say is that IQ tests are biased, not that “any sampling method is biased”. IQ tests are biased, but statistical sampling methods can be unbiased.
    - Mountain_Skies 7y ago
      Are there any widespread test given in schools that are proxy IQ tests? Mensa accepts older ACT and SAT scores as they consider the current versions to be knowledge tests but older versions to be intelligence tests. Not sure if this is still the case but at one time all high school students in the State of New York were required to take the SAT prior to graduation. While not representative of the entire population, it would have been a good representation of the high school student slice of the population across the entire state. Perhaps that would have been good enough for some uses.
btilly 7y ago
I have long joked that Mensa membership indicates that you passed the IQ test and failed the intelligence test. More seriously, Mensa membership selects for people whose main accomplishment in life is doing well on an IQ test. So they are at least somewhat smart. But if you look for people who have actual accomplishments, you'll find people who tend to also be smart but are more interesting to be around. Note, this is not sour grapes. My test scores mean that I'm qualified for Mensa, but I've never been impressed enough by any Mensa member that I met to be interested.
- kerkeslager 7y ago
  This was sort of my experience with Mensa. Their three purposes supposedly are: 1. to identify and to foster human intelligence for the benefit of humanity; 2. to encourage research into the nature, characteristics, and uses of intelligence; 3. and to provide a stimulating intellectual and social environment for its members. It's not entirely clear to me how anything they do benefits humanity (1), the "nature, characteristics, and uses of intelligence" seem to revolve around games for them which is a pretty meaningless measure of any form of intelligence I care about (2) and while I'm not a member so I can't say whether it's stimulating to them, I can certainly say that all the materials I've come across from Mensa have soon bored me(3). So from where I sit, Mensa fails on two of their purposes, and only succeeds in their third because of a technicality in their wording.
- mcv 7y ago
  Mensa seems to have a pretty bad reputation in the US. I never had much opinion about and didn't know any Mensa members until my sister joined. She meets loads of interesting people there. They have game weekends where they play all sorts of interesting board games, but also in other activities it's nice to be around people who think a bit like you do. I've never felt the need to join Mensa because I studied CS and AI, and all my friends from there are probably about as smart as I am, so I already have plenty of people to relate to, but she was always very interested in sport, which probably attracts a different crowd (though she's working in IT now).
- cs02rm0 7y ago
  My test scores meant I was qualified for Mensa too, I could've taken up the membership when offered as a kid but declined at the time. Eventually I took the quick test which suggested I should take the longer test as a young adult looking to bolster my CV (résumé). Narrowly missed out and before I was able to submit again I had enough on there not to give it much thought again for me. An IQ test was part of the qualification process for the first job I took out of university, something I've not seen since. I think my eldest son's probably at least as bright as I am, so this sort of thing has been on my mind again. We try and give him a breadth of experience - he does a lot of rugby (completely non-contact at his age), running and other sports, some child adventure centres, music lessons, scouts soon, etc. But I don't see too much wrong with adding Mensa as a small part in the context of a larger picture.
MagnumPIG 7y ago
Yeahhh this study is near worthless. Still an interesting research subject if you can do away with the sampling bias though.
buboard 7y ago
ah yes the lazy way to find smart people: go to mensa. Everyone in there is guaranteed to be well balanced, and surely will not exhibit any narcissism (did they check for narcissism btw?)
cirgue 7y ago
Literally, using Mensa membership as a proxy for IQ was an example of selection bias in one of my undergrad research methods classes.
gwern 7y ago
Indeed. This study is complete bullshit: https://www.gwern.net/SMPY#fn1 https://www.gwern.net/SMPY#fn1 It contradicts every (non-self-selected) longitudinal or cross-sectional study, including far more elite samples (which if their theory was correct would show vastly more dysfunctionality, except of course it isn't and those samples don't), and every genetic study as well, and the reported (self-diagnosis) rates are often orders of magnitude larger than any risk factor ever confirmed and literally unbelievable. Incidentally, Hambrick's writeup here is also quite bad. Aside from failing to mention all of those reasons why it's BS, he doesn't describe accurately the research he does mention. Consider this paragraph from the end: > All the same, Karpinski and her colleagues’ findings set the stage for research that promises to shed new light on the link between intelligence and health. One possibility is that associations between intelligence and health outcomes reflect pleiotropy, which occurs when a gene influences seemingly unrelated traits. There is already some evidence to suggest that this is the case. In a 2015 study, Rosalind Arden and her colleagues concluded that the association between IQ and longevity is mostly explained by genetic factors. How did he miss the fact that the Arden study in question shows, as all such studies show, that intelligence correlates with greater lifespan/longevity when he's written an entire credulous column about how intelligence is bad for you and makes people crazy and increases the rate of mental disorders (some of which, like schizophrenia, reduce your life expectancy by decades)? It's right there in the abstract, it's not hidden away. One would think that would at least merit a brief pause to puzzle over the discrepancy...
anbotero 7y ago
Actually, I think it's kinda funny and ironic that in Spanish we use "menso" as an insult for someone that is stupid. Formal (kinda) definition on how we use it in the Spanish section here: https://en.wiktionary.org/wiki/menso https://en.wiktionary.org/wiki/menso

deleted 7y ago

[deleted]

deleted 7y ago

[deleted]

anewguy9000 7y ago

any sampling method is 'biased', thats why its a 'sample'

bentona 7y ago
This study not only selected for people who go out of their way to join a society based on high IQ, it also selected for people who open & complete email surveys. Yes, many practical sampling methods are biased, but it is not difficult for me to imagine that any result here is caused entirely by these biases.
- anewguy9000 7y ago
  i agree with your criticism in general -- actually it applies to virtually all psychological and medical studies. but if you think the physical disabilities are better explained by something else to do with this population sample than their iq, you could very clearly publish this theory
scarmig 7y ago
Mensa members are... Not representative of people with high IQs.
mindblock 7y ago
Yeah I don’t think smart people pay a membership fee to an organization to be certified as smart.
mlyle 7y ago
This is what my 4th grader just learned is called a "convenience sample" and all the ways it can be biased... :P If he can understand the shortcomings, then so can the rest of us.
richdougherty 7y ago
If you take a look at the definition and you'll see it's possible to have an unbiased sample. https://en.m.wikipedia.org/wiki/Sample_(statistics) https://en.m.wikipedia.org/wiki/Sample_(statistics)
- anewguy9000 7y ago
  how do you randomly sample high iq? by definition the population would have had to have taken a test. so already, even if the study looked at every known high iq individual in the world, it would still be biased -- for those with academic backgrounds, psychological histories, criminal backgrounds, and whatever else leads one to take an iq test. nb i am not advocating for this particular study nor their methodology.
  - zingermc 7y ago
    You would have to sample a lot of people (yes, there is a selection bias in the people that participate) and give all of them IQ tests.
  - dahart 7y ago
    What you meant to say is that IQ tests are biased, not that “any sampling method is biased”. IQ tests are biased, but statistical sampling methods can be unbiased.
  - Mountain_Skies 7y ago
    Are there any widespread test given in schools that are proxy IQ tests? Mensa accepts older ACT and SAT scores as they consider the current versions to be knowledge tests but older versions to be intelligence tests. Not sure if this is still the case but at one time all high school students in the State of New York were required to take the SAT prior to graduation. While not representative of the entire population, it would have been a good representation of the high school student slice of the population across the entire state. Perhaps that would have been good enough for some uses.

btilly 7y ago

I have long joked that Mensa membership indicates that you passed the IQ test and failed the intelligence test. More seriously, Mensa membership selects for people whose main accomplishment in life is doing well on an IQ test. So they are at least somewhat smart. But if you look for people who have actual accomplishments, you'll find people who tend to also be smart but are more interesting to be around. Note, this is not sour grapes. My test scores mean that I'm qualified for Mensa, but I've never been impressed enough by any Mensa member that I met to be interested.

kerkeslager 7y ago
This was sort of my experience with Mensa. Their three purposes supposedly are: 1. to identify and to foster human intelligence for the benefit of humanity; 2. to encourage research into the nature, characteristics, and uses of intelligence; 3. and to provide a stimulating intellectual and social environment for its members. It's not entirely clear to me how anything they do benefits humanity (1), the "nature, characteristics, and uses of intelligence" seem to revolve around games for them which is a pretty meaningless measure of any form of intelligence I care about (2) and while I'm not a member so I can't say whether it's stimulating to them, I can certainly say that all the materials I've come across from Mensa have soon bored me(3). So from where I sit, Mensa fails on two of their purposes, and only succeeds in their third because of a technicality in their wording.
mcv 7y ago
Mensa seems to have a pretty bad reputation in the US. I never had much opinion about and didn't know any Mensa members until my sister joined. She meets loads of interesting people there. They have game weekends where they play all sorts of interesting board games, but also in other activities it's nice to be around people who think a bit like you do. I've never felt the need to join Mensa because I studied CS and AI, and all my friends from there are probably about as smart as I am, so I already have plenty of people to relate to, but she was always very interested in sport, which probably attracts a different crowd (though she's working in IT now).
cs02rm0 7y ago
My test scores meant I was qualified for Mensa too, I could've taken up the membership when offered as a kid but declined at the time. Eventually I took the quick test which suggested I should take the longer test as a young adult looking to bolster my CV (résumé). Narrowly missed out and before I was able to submit again I had enough on there not to give it much thought again for me. An IQ test was part of the qualification process for the first job I took out of university, something I've not seen since. I think my eldest son's probably at least as bright as I am, so this sort of thing has been on my mind again. We try and give him a breadth of experience - he does a lot of rugby (completely non-contact at his age), running and other sports, some child adventure centres, music lessons, scouts soon, etc. But I don't see too much wrong with adding Mensa as a small part in the context of a larger picture.

MagnumPIG 7y ago

Yeahhh this study is near worthless. Still an interesting research subject if you can do away with the sampling bias though.

buboard 7y ago

ah yes the lazy way to find smart people: go to mensa. Everyone in there is guaranteed to be well balanced, and surely will not exhibit any narcissism (did they check for narcissism btw?)

cirgue 7y ago

Literally, using Mensa membership as a proxy for IQ was an example of selection bias in one of my undergrad research methods classes.

gwern 7y ago

Indeed. This study is complete bullshit: https://www.gwern.net/SMPY#fn1 https://www.gwern.net/SMPY#fn1 It contradicts every (non-self-selected) longitudinal or cross-sectional study, including far more elite samples (which if their theory was correct would show vastly more dysfunctionality, except of course it isn't and those samples don't), and every genetic study as well, and the reported (self-diagnosis) rates are often orders of magnitude larger than any risk factor ever confirmed and literally unbelievable. Incidentally, Hambrick's writeup here is also quite bad. Aside from failing to mention all of those reasons why it's BS, he doesn't describe accurately the research he does mention. Consider this paragraph from the end: > All the same, Karpinski and her colleagues’ findings set the stage for research that promises to shed new light on the link between intelligence and health. One possibility is that associations between intelligence and health outcomes reflect pleiotropy, which occurs when a gene influences seemingly unrelated traits. There is already some evidence to suggest that this is the case. In a 2015 study, Rosalind Arden and her colleagues concluded that the association between IQ and longevity is mostly explained by genetic factors. How did he miss the fact that the Arden study in question shows, as all such studies show, that intelligence correlates with greater lifespan/longevity when he's written an entire credulous column about how intelligence is bad for you and makes people crazy and increases the rate of mental disorders (some of which, like schizophrenia, reduce your life expectancy by decades)? It's right there in the abstract, it's not hidden away. One would think that would at least merit a brief pause to puzzle over the discrepancy...

anbotero 7y ago

Actually, I think it's kinda funny and ironic that in Spanish we use "menso" as an insult for someone that is stupid. Formal (kinda) definition on how we use it in the Spanish section here: https://en.wiktionary.org/wiki/menso https://en.wiktionary.org/wiki/menso