4 ms·

Your physician was identified for medical school, residency, and fellowship based in large part on standardized test scores. And the schools and residency progr

Your physician was identified for medical school, residency, and fellowship based in large part on standardized test scores. And the schools and residency programs are tyrannized by their students and residents scores. I gotta say, it doesn't take long to identify the docs who did well on those tests: they're generally the best docs. They're the most articulate, ask the most insightful questions during their patient interviews, develop the best diagnostic and management plans, and accomplish the most difficult research. I suspect the same goes for lawyers, law programs, and their bar exams, engineers, engineering programs, and their EIT exams, etc. Why shouldn't kids learn to play by the same rules their parents do? Indeed, isn't that what kids want to do?

andrewce 16y ago
Except that the MCAT measures at least some of the core competencies necessary for being a doctor, and includes a lot of vital information which, if you don't have, make it pretty tough to be a physician. Even then, there are plenty of people whose MCAT scores were high enough to get them into medical school who ended up being terrible doctors, just as there are plenty of people who barely got into med school who ended up being terrific doctors. I can't cite any studies, but I imagine that just like anything else the variability within a score band is just as much as the variability between score bands. (Similarly: plenty of bad lawyers have passed the bar, just as plenty of good lawyers failed it before they passed. There's enough variability between the Bar exams of different states to make this possible. I suspect the same is true about engineering programs as well). It may just be that the MCAT is a good indicator of one's ability to become a doctor. Med school admissions are so competitive right now that these schools are able to be pretty selective about who they admit. It may be that these tests were designed at least in part by the professionals in those respective fields to measure knowledge and competencies that directly relate to the domains in question. More to the point, this is a specious comparison because the exams you cite are professional vocational exams which correspond to a specific career path/domain. There's no such vocational test that will adequately cover every child, nor do we even have philosophical agreement about the skills necessary to live a good life, much less the relative importance of such skills, and even if we did, there are so many different ways that things like literacy and numeracy manifest that it'd be downright impossible to construct a test that can even come close to measuring it. The tests you cite are tests that limit who can practice in a given profession, and while they're not perfect, I'll concede their usefulness. More importantly, they're on an individual basis. There'll definitely be some false-positives (people who are good at taking tests but poor at performing the basic skills necessary for competency) and false-negatives, but those can be corrected. It's a whole different ball-game when you're talking about a non-vocational test, particularly one aimed at people who are not yet adults and who have not yet chosen a particular career path.
- djacobs 16y ago
  The MCAT actually doesn't measure much in the way of knowledge, just of speed. The actual questions are easy, and your score (after a certain knowledge/intelligence threshold) is solely determined by how many practice questions you've waded through.
- niels_olson 16y ago
  >MCAT arguement The USMLE exams (Step I: basic science, Step IICK: clinical knowledge, Step IICS: clinical skills, and Step III: clinical problem solving) are more like the bar exams. > this is a specious comparison because the exams you cite are professional vocational exams which correspond to a specific career path/domain. Well, you're throwing in the MCAT, which is an aptitude test based on Bloom's Taxonomy of Educational Objectives, more like the GRE than the USMLE or bar exams. > there are so many different ways that things like literacy and numeracy manifest Literacy and numeracy can't be measured? You sure you want to hang your hat on that? Look, to quote Allah, I mean, pg, himself, (http://www.paulgraham.com/wealth.html http://www.paulgraham.com/wealth.html) > To get rich you need to get yourself in a situation with two things, measurement and leverage. You need to be in a position where your performance can be measured, or there is no way to get paid more by doing more. And you have to have leverage, in the sense that the decisions you make have a big effect. We can easily modify this paragraph to most if not all domains and industries. I leave as an exercise to the reader to modify it for "improving education".
jbooth 16y ago
Well, teaching's different from doctoring/lawyering in a few ways. If you judge, say, a couple defense lawyers strictly on their success rate, put one on OJ Simpson's team and give another a long backlog of public defender cases, then Johnny Cochran's going to be the best layer ever. Does that mean he would've done better with the cases where you're set up to fail? Not to say that metrics are useless, but they can definitely be flawed.
MichaelSalib 16y ago
I gotta say, it doesn't take long to identify the docs who did well on those tests: they're generally the best docs. Do you have any evidence to support this notion at all? Or is thus just some religious notion that we're supposed to take on faith?
- niels_olson 16y ago
  Yes, I'm a doctor. I work with other doctors.
  - MichaelSalib 16y ago
    Wait, you mean you have anecdotes? Why didn't you say that? I mean, now that I know you've got some anecdotes, I'm totally convinced.

andrewce 16y ago

Except that the MCAT measures at least some of the core competencies necessary for being a doctor, and includes a lot of vital information which, if you don't have, make it pretty tough to be a physician. Even then, there are plenty of people whose MCAT scores were high enough to get them into medical school who ended up being terrible doctors, just as there are plenty of people who barely got into med school who ended up being terrific doctors. I can't cite any studies, but I imagine that just like anything else the variability within a score band is just as much as the variability between score bands. (Similarly: plenty of bad lawyers have passed the bar, just as plenty of good lawyers failed it before they passed. There's enough variability between the Bar exams of different states to make this possible. I suspect the same is true about engineering programs as well). It may just be that the MCAT is a good indicator of one's ability to become a doctor. Med school admissions are so competitive right now that these schools are able to be pretty selective about who they admit. It may be that these tests were designed at least in part by the professionals in those respective fields to measure knowledge and competencies that directly relate to the domains in question. More to the point, this is a specious comparison because the exams you cite are professional vocational exams which correspond to a specific career path/domain. There's no such vocational test that will adequately cover every child, nor do we even have philosophical agreement about the skills necessary to live a good life, much less the relative importance of such skills, and even if we did, there are so many different ways that things like literacy and numeracy manifest that it'd be downright impossible to construct a test that can even come close to measuring it. The tests you cite are tests that limit who can practice in a given profession, and while they're not perfect, I'll concede their usefulness. More importantly, they're on an individual basis. There'll definitely be some false-positives (people who are good at taking tests but poor at performing the basic skills necessary for competency) and false-negatives, but those can be corrected. It's a whole different ball-game when you're talking about a non-vocational test, particularly one aimed at people who are not yet adults and who have not yet chosen a particular career path.

djacobs 16y ago
The MCAT actually doesn't measure much in the way of knowledge, just of speed. The actual questions are easy, and your score (after a certain knowledge/intelligence threshold) is solely determined by how many practice questions you've waded through.
niels_olson 16y ago
>MCAT arguement The USMLE exams (Step I: basic science, Step IICK: clinical knowledge, Step IICS: clinical skills, and Step III: clinical problem solving) are more like the bar exams. > this is a specious comparison because the exams you cite are professional vocational exams which correspond to a specific career path/domain. Well, you're throwing in the MCAT, which is an aptitude test based on Bloom's Taxonomy of Educational Objectives, more like the GRE than the USMLE or bar exams. > there are so many different ways that things like literacy and numeracy manifest Literacy and numeracy can't be measured? You sure you want to hang your hat on that? Look, to quote Allah, I mean, pg, himself, (http://www.paulgraham.com/wealth.html http://www.paulgraham.com/wealth.html) > To get rich you need to get yourself in a situation with two things, measurement and leverage. You need to be in a position where your performance can be measured, or there is no way to get paid more by doing more. And you have to have leverage, in the sense that the decisions you make have a big effect. We can easily modify this paragraph to most if not all domains and industries. I leave as an exercise to the reader to modify it for "improving education".

jbooth 16y ago

Well, teaching's different from doctoring/lawyering in a few ways. If you judge, say, a couple defense lawyers strictly on their success rate, put one on OJ Simpson's team and give another a long backlog of public defender cases, then Johnny Cochran's going to be the best layer ever. Does that mean he would've done better with the cases where you're set up to fail? Not to say that metrics are useless, but they can definitely be flawed.

MichaelSalib 16y ago

I gotta say, it doesn't take long to identify the docs who did well on those tests: they're generally the best docs. Do you have any evidence to support this notion at all? Or is thus just some religious notion that we're supposed to take on faith?

niels_olson 16y ago
Yes, I'm a doctor. I work with other doctors.
- MichaelSalib 16y ago
  Wait, you mean you have anecdotes? Why didn't you say that? I mean, now that I know you've got some anecdotes, I'm totally convinced.