3 ms·

> Do you have any serious evidence that shows that it doesn't have external validity? I can't prove a negative, all that a study can do is say "no results obse

> Do you have any serious evidence that shows that it doesn't have external validity? I can't prove a negative, all that a study can do is say "no results observed", but there's plenty of that. From the researcher cited in the article: "Many very specific predictions about the MBTI have not been confirmed or have been proved wrong." "Finally, there is no evidence that the MBTI measures anything of value." This aligns with all of the other at-all-convincing research I've seen on MBTI: it assigned binary distributions to normally distributed data, and has very low predictive power. The majority of people will be near the middle of those distributions, where MBTI is at its least useful. Further, I think your example is unfair: MBTI is a one-way function. I can meet people and anticipate their scores to with 1-2 errors (a huge margin, but we talk like it's 'close'!) I can't go from an MBTI score to useful predictions about the person though, because it's so massively information-destroying. Even on the things it purports to measure, the normal-to-binary problem means that it has awful predictive power. MBTI defines a somewhat-consistent function from people to letters, but since it's an arbitrary clustering system it doesn't actually allow new predictions or efficiently compress information about people. So yes, I could probably tell some people apart by MBTI, but you could also assemble 10 different ESFJ's who are more dissimilar than the ESFJ and the INTP. It's like a compression algorithm that can't be reversed, and about as useful.

0xfffafaCrash 10y ago
I agree with a lot of this. I don't disagree with the fact that it's not necessarily super effective because the distributions of personalities of people are not bimodal, as would be ideal for this simplistic sort of classification. It's also unlikely to be supremely effective at clustering in that way because the design of the metric wasn't really data driven in any systemic way. It's not even like someone used principal component analysis to really establish that the four classifiers are reasonably orthogonal. It's probably not super obvious/trivial to find human language-friendly orthogonal aspects of personality that are each bimodal, but I imagine the surveys could be improved informed by better data science. It's certainly a compression algorithm, as are all classification based metrics based on answers to multiple questions, and I'm not arguing it's the most effective one around. However I think before making claims like it has no "external validity" the degree to which it might have it should actually be tested with reasonable methodology and I'm offering something that can absolutely be tested scientifically. There's nothing impossible about having some people take the test and having others who know them try to classify them without the results and see how well the two coincide. It's also possible to evaluate the degree to which people can decompress the data. You can set it up where people have equal time spent with two people of each MBTI and see how well they can guess their answers to the original survey questions with additional knowledge of the MBTI as opposed to without it. It won't establish an upperbound for how useful an MBTI is but it will help gauge a lower bound and I'm pretty confident the lower bound will be higher than that of a zodiac. I imagine we can design a better classification, probably with even the same survey questions, but just because a metric wasn't well-designed doesn't mean the information it provides is meaningless.
triplesec 10y ago
This is by far the clearest methodological explanation of the problem so far from someone who understands statistics and the problems of empirical research. Thank you. It's amazing how many people are too wedded to the truthiness of it, and are unable to let go of something they did once and believed because it was plausible.

0xfffafaCrash 10y ago

I agree with a lot of this. I don't disagree with the fact that it's not necessarily super effective because the distributions of personalities of people are not bimodal, as would be ideal for this simplistic sort of classification. It's also unlikely to be supremely effective at clustering in that way because the design of the metric wasn't really data driven in any systemic way. It's not even like someone used principal component analysis to really establish that the four classifiers are reasonably orthogonal. It's probably not super obvious/trivial to find human language-friendly orthogonal aspects of personality that are each bimodal, but I imagine the surveys could be improved informed by better data science. It's certainly a compression algorithm, as are all classification based metrics based on answers to multiple questions, and I'm not arguing it's the most effective one around. However I think before making claims like it has no "external validity" the degree to which it might have it should actually be tested with reasonable methodology and I'm offering something that can absolutely be tested scientifically. There's nothing impossible about having some people take the test and having others who know them try to classify them without the results and see how well the two coincide. It's also possible to evaluate the degree to which people can decompress the data. You can set it up where people have equal time spent with two people of each MBTI and see how well they can guess their answers to the original survey questions with additional knowledge of the MBTI as opposed to without it. It won't establish an upperbound for how useful an MBTI is but it will help gauge a lower bound and I'm pretty confident the lower bound will be higher than that of a zodiac. I imagine we can design a better classification, probably with even the same survey questions, but just because a metric wasn't well-designed doesn't mean the information it provides is meaningless.

triplesec 10y ago

This is by far the clearest methodological explanation of the problem so far from someone who understands statistics and the problems of empirical research. Thank you. It's amazing how many people are too wedded to the truthiness of it, and are unable to let go of something they did once and believed because it was plausible.