3 ms·

Do you have any serious evidence that shows that it doesn't have external validity? If say some person knows two people and that one tested as INTP and the oth

Do you have any serious evidence that shows that it doesn't have external validity? If say some person knows two people and that one tested as INTP and the other as ESFJ and this person doesn't know for a fact who had which test results. Are you really sure they won't have better than even odds at figuring out which is which based on their interactions with the people that don't involve asking for their MBTI test results? If so, based on what study? It's clear I wouldn't be able to tell them apart based on their birthday (unless they were super young or something), but it's not at all clear that it's the case for MBTI.

ivraatiems 10y ago
That's not how science works. The burden is not on me to prove MBTI does not have external validity, the burden is on the MBTI to prove it does have it. You have the chain of cause and effect backwards. If you say "the sky is green," it's not my job to prove to you it's not. It's your job to prove it is. You're the one making the claim. But here is a (more politely worded than I) paper on the MBTI's ineffectiveness for your perusal: http://epublications.bond.edu.au/cgi/viewcontent.cgi?article=1026&context=hss_pubs&sei-redir=1 http://epublications.bond.edu.au/cgi/viewcontent.cgi?article... - simply googling "mbti external validity" will also give you plenty of explanation. I also recommend looking at the many studies that review cites, since its whole purpose is amassing information on whether the MBTI is effective.
- 0xfffafaCrash 10y ago
  I'm a scientist, so I kind of know how science works. There's a difference between saying the extent of something's effectiveness hasn't been established by existing research and saying that it is ineffective. I've read the publication you are linking to before and it is saying the latter without the evidence necessary to back up the claim.
  - ivraatiems 10y ago
    I'm sorry, but I don't believe you. You are lacking a basic understanding of how the research process works in social sciences. I don't want to be "credentials guy," but I have a degree in cognitive science and I've conducted human subjects research. What you are saying is simply wrong. That publication cites dozens of studies to back up its claims. Are you suggesting they are all invalid? If so, why?
    - rlucas 10y ago
      Did you read the same linked article I did? There were not cites to "dozens of studies;" rather, there were quotes from the Guardian and something called Skeptoid, plus a quoted but uncited "researcher." Combined with the fact that this is basically a listicle ("quoticle?") clickbait from a mid/low-brow popsci website, with no analysis or synthesis, and you can pretty much throw TFA away, regardless of your side of what's shaping up to be a religious war here on HN.
      - ivraatiems 10y ago
        I'm talking about the study I just linked, not the article that started this discussion.
        rlucas 10y ago
        Apologies, I missed that. The review paper you linked was indeed deeply researched and had a good synthesis of what it discussed. Decided not a pop Sci listicle :)
ivan_gammel 10y ago
Science works another way. You have to prove that the theory works, not that it does not work (no sense in asking for evidence of that). Your personal observation does not matter if it's not confirmed by others, so for this theory to be confirmed, a) it shall be falsifiable, b) there shall be a reliable proof, c) there shall be independent confirmation, d) other possible explanations of observed results must be considered and rejected
Bartweiss 10y ago
> Do you have any serious evidence that shows that it doesn't have external validity? I can't prove a negative, all that a study can do is say "no results observed", but there's plenty of that. From the researcher cited in the article: "Many very specific predictions about the MBTI have not been confirmed or have been proved wrong." "Finally, there is no evidence that the MBTI measures anything of value." This aligns with all of the other at-all-convincing research I've seen on MBTI: it assigned binary distributions to normally distributed data, and has very low predictive power. The majority of people will be near the middle of those distributions, where MBTI is at its least useful. Further, I think your example is unfair: MBTI is a one-way function. I can meet people and anticipate their scores to with 1-2 errors (a huge margin, but we talk like it's 'close'!) I can't go from an MBTI score to useful predictions about the person though, because it's so massively information-destroying. Even on the things it purports to measure, the normal-to-binary problem means that it has awful predictive power. MBTI defines a somewhat-consistent function from people to letters, but since it's an arbitrary clustering system it doesn't actually allow new predictions or efficiently compress information about people. So yes, I could probably tell some people apart by MBTI, but you could also assemble 10 different ESFJ's who are more dissimilar than the ESFJ and the INTP. It's like a compression algorithm that can't be reversed, and about as useful.
- 0xfffafaCrash 10y ago
  I agree with a lot of this. I don't disagree with the fact that it's not necessarily super effective because the distributions of personalities of people are not bimodal, as would be ideal for this simplistic sort of classification. It's also unlikely to be supremely effective at clustering in that way because the design of the metric wasn't really data driven in any systemic way. It's not even like someone used principal component analysis to really establish that the four classifiers are reasonably orthogonal. It's probably not super obvious/trivial to find human language-friendly orthogonal aspects of personality that are each bimodal, but I imagine the surveys could be improved informed by better data science. It's certainly a compression algorithm, as are all classification based metrics based on answers to multiple questions, and I'm not arguing it's the most effective one around. However I think before making claims like it has no "external validity" the degree to which it might have it should actually be tested with reasonable methodology and I'm offering something that can absolutely be tested scientifically. There's nothing impossible about having some people take the test and having others who know them try to classify them without the results and see how well the two coincide. It's also possible to evaluate the degree to which people can decompress the data. You can set it up where people have equal time spent with two people of each MBTI and see how well they can guess their answers to the original survey questions with additional knowledge of the MBTI as opposed to without it. It won't establish an upperbound for how useful an MBTI is but it will help gauge a lower bound and I'm pretty confident the lower bound will be higher than that of a zodiac. I imagine we can design a better classification, probably with even the same survey questions, but just because a metric wasn't well-designed doesn't mean the information it provides is meaningless.

ivraatiems 10y ago

That's not how science works. The burden is not on me to prove MBTI does not have external validity, the burden is on the MBTI to prove it does have it. You have the chain of cause and effect backwards. If you say "the sky is green," it's not my job to prove to you it's not. It's your job to prove it is. You're the one making the claim. But here is a (more politely worded than I) paper on the MBTI's ineffectiveness for your perusal: http://epublications.bond.edu.au/cgi/viewcontent.cgi?article=1026&context=hss_pubs&sei-redir=1 http://epublications.bond.edu.au/cgi/viewcontent.cgi?article... - simply googling "mbti external validity" will also give you plenty of explanation. I also recommend looking at the many studies that review cites, since its whole purpose is amassing information on whether the MBTI is effective.

0xfffafaCrash 10y ago
I'm a scientist, so I kind of know how science works. There's a difference between saying the extent of something's effectiveness hasn't been established by existing research and saying that it is ineffective. I've read the publication you are linking to before and it is saying the latter without the evidence necessary to back up the claim.
- ivraatiems 10y ago
  I'm sorry, but I don't believe you. You are lacking a basic understanding of how the research process works in social sciences. I don't want to be "credentials guy," but I have a degree in cognitive science and I've conducted human subjects research. What you are saying is simply wrong. That publication cites dozens of studies to back up its claims. Are you suggesting they are all invalid? If so, why?
  - rlucas 10y ago
    Did you read the same linked article I did? There were not cites to "dozens of studies;" rather, there were quotes from the Guardian and something called Skeptoid, plus a quoted but uncited "researcher." Combined with the fact that this is basically a listicle ("quoticle?") clickbait from a mid/low-brow popsci website, with no analysis or synthesis, and you can pretty much throw TFA away, regardless of your side of what's shaping up to be a religious war here on HN.
    - ivraatiems 10y ago
      I'm talking about the study I just linked, not the article that started this discussion.
      - rlucas 10y ago
        Apologies, I missed that. The review paper you linked was indeed deeply researched and had a good synthesis of what it discussed. Decided not a pop Sci listicle :)

ivan_gammel 10y ago

Science works another way. You have to prove that the theory works, not that it does not work (no sense in asking for evidence of that). Your personal observation does not matter if it's not confirmed by others, so for this theory to be confirmed, a) it shall be falsifiable, b) there shall be a reliable proof, c) there shall be independent confirmation, d) other possible explanations of observed results must be considered and rejected

Bartweiss 10y ago

> Do you have any serious evidence that shows that it doesn't have external validity? I can't prove a negative, all that a study can do is say "no results observed", but there's plenty of that. From the researcher cited in the article: "Many very specific predictions about the MBTI have not been confirmed or have been proved wrong." "Finally, there is no evidence that the MBTI measures anything of value." This aligns with all of the other at-all-convincing research I've seen on MBTI: it assigned binary distributions to normally distributed data, and has very low predictive power. The majority of people will be near the middle of those distributions, where MBTI is at its least useful. Further, I think your example is unfair: MBTI is a one-way function. I can meet people and anticipate their scores to with 1-2 errors (a huge margin, but we talk like it's 'close'!) I can't go from an MBTI score to useful predictions about the person though, because it's so massively information-destroying. Even on the things it purports to measure, the normal-to-binary problem means that it has awful predictive power. MBTI defines a somewhat-consistent function from people to letters, but since it's an arbitrary clustering system it doesn't actually allow new predictions or efficiently compress information about people. So yes, I could probably tell some people apart by MBTI, but you could also assemble 10 different ESFJ's who are more dissimilar than the ESFJ and the INTP. It's like a compression algorithm that can't be reversed, and about as useful.

0xfffafaCrash 10y ago
I agree with a lot of this. I don't disagree with the fact that it's not necessarily super effective because the distributions of personalities of people are not bimodal, as would be ideal for this simplistic sort of classification. It's also unlikely to be supremely effective at clustering in that way because the design of the metric wasn't really data driven in any systemic way. It's not even like someone used principal component analysis to really establish that the four classifiers are reasonably orthogonal. It's probably not super obvious/trivial to find human language-friendly orthogonal aspects of personality that are each bimodal, but I imagine the surveys could be improved informed by better data science. It's certainly a compression algorithm, as are all classification based metrics based on answers to multiple questions, and I'm not arguing it's the most effective one around. However I think before making claims like it has no "external validity" the degree to which it might have it should actually be tested with reasonable methodology and I'm offering something that can absolutely be tested scientifically. There's nothing impossible about having some people take the test and having others who know them try to classify them without the results and see how well the two coincide. It's also possible to evaluate the degree to which people can decompress the data. You can set it up where people have equal time spent with two people of each MBTI and see how well they can guess their answers to the original survey questions with additional knowledge of the MBTI as opposed to without it. It won't establish an upperbound for how useful an MBTI is but it will help gauge a lower bound and I'm pretty confident the lower bound will be higher than that of a zodiac. I imagine we can design a better classification, probably with even the same survey questions, but just because a metric wasn't well-designed doesn't mean the information it provides is meaningless.