7 ms·

The thing that bugs me about articles arguing for performance based pay is that it fails to address two critical points. The first is why the teachers that are

by BCM43 16y ago

The thing that bugs me about articles arguing for performance based pay is that it fails to address two critical points. The first is why the teachers that are bad are actually bad. I would be interested in what would happen if the lowest paying teachers were forced to addend additional training. Perhaps all they need is a refresher. It is also possible that they have become disillusioned with what they once thought was a great way to make an effect on children's lives. I could very well imagine this happening to me were I to become a teacher. The other is how they are measuring teacher performance. The only way I have seen this happen is though standardized tests. I do not believe that these actually measure much. The best teachers I have had have not taught me based on what was on the test, which would have meant spoon-feeding me facts and telling me to memorize them, but rather what they felt was important, which often included explaining how I could find the information on my own if I even needed.

niels_olson 16y ago
Your physician was identified for medical school, residency, and fellowship based in large part on standardized test scores. And the schools and residency programs are tyrannized by their students and residents scores. I gotta say, it doesn't take long to identify the docs who did well on those tests: they're generally the best docs. They're the most articulate, ask the most insightful questions during their patient interviews, develop the best diagnostic and management plans, and accomplish the most difficult research. I suspect the same goes for lawyers, law programs, and their bar exams, engineers, engineering programs, and their EIT exams, etc. Why shouldn't kids learn to play by the same rules their parents do? Indeed, isn't that what kids want to do?
- andrewce 16y ago
  Except that the MCAT measures at least some of the core competencies necessary for being a doctor, and includes a lot of vital information which, if you don't have, make it pretty tough to be a physician. Even then, there are plenty of people whose MCAT scores were high enough to get them into medical school who ended up being terrible doctors, just as there are plenty of people who barely got into med school who ended up being terrific doctors. I can't cite any studies, but I imagine that just like anything else the variability within a score band is just as much as the variability between score bands. (Similarly: plenty of bad lawyers have passed the bar, just as plenty of good lawyers failed it before they passed. There's enough variability between the Bar exams of different states to make this possible. I suspect the same is true about engineering programs as well). It may just be that the MCAT is a good indicator of one's ability to become a doctor. Med school admissions are so competitive right now that these schools are able to be pretty selective about who they admit. It may be that these tests were designed at least in part by the professionals in those respective fields to measure knowledge and competencies that directly relate to the domains in question. More to the point, this is a specious comparison because the exams you cite are professional vocational exams which correspond to a specific career path/domain. There's no such vocational test that will adequately cover every child, nor do we even have philosophical agreement about the skills necessary to live a good life, much less the relative importance of such skills, and even if we did, there are so many different ways that things like literacy and numeracy manifest that it'd be downright impossible to construct a test that can even come close to measuring it. The tests you cite are tests that limit who can practice in a given profession, and while they're not perfect, I'll concede their usefulness. More importantly, they're on an individual basis. There'll definitely be some false-positives (people who are good at taking tests but poor at performing the basic skills necessary for competency) and false-negatives, but those can be corrected. It's a whole different ball-game when you're talking about a non-vocational test, particularly one aimed at people who are not yet adults and who have not yet chosen a particular career path.
  - djacobs 16y ago
    The MCAT actually doesn't measure much in the way of knowledge, just of speed. The actual questions are easy, and your score (after a certain knowledge/intelligence threshold) is solely determined by how many practice questions you've waded through.
  - niels_olson 16y ago
    >MCAT arguement The USMLE exams (Step I: basic science, Step IICK: clinical knowledge, Step IICS: clinical skills, and Step III: clinical problem solving) are more like the bar exams. > this is a specious comparison because the exams you cite are professional vocational exams which correspond to a specific career path/domain. Well, you're throwing in the MCAT, which is an aptitude test based on Bloom's Taxonomy of Educational Objectives, more like the GRE than the USMLE or bar exams. > there are so many different ways that things like literacy and numeracy manifest Literacy and numeracy can't be measured? You sure you want to hang your hat on that? Look, to quote Allah, I mean, pg, himself, (http://www.paulgraham.com/wealth.html http://www.paulgraham.com/wealth.html) > To get rich you need to get yourself in a situation with two things, measurement and leverage. You need to be in a position where your performance can be measured, or there is no way to get paid more by doing more. And you have to have leverage, in the sense that the decisions you make have a big effect. We can easily modify this paragraph to most if not all domains and industries. I leave as an exercise to the reader to modify it for "improving education".
- jbooth 16y ago
  Well, teaching's different from doctoring/lawyering in a few ways. If you judge, say, a couple defense lawyers strictly on their success rate, put one on OJ Simpson's team and give another a long backlog of public defender cases, then Johnny Cochran's going to be the best layer ever. Does that mean he would've done better with the cases where you're set up to fail? Not to say that metrics are useless, but they can definitely be flawed.
- MichaelSalib 16y ago
  I gotta say, it doesn't take long to identify the docs who did well on those tests: they're generally the best docs. Do you have any evidence to support this notion at all? Or is thus just some religious notion that we're supposed to take on faith?
  - niels_olson 16y ago
    Yes, I'm a doctor. I work with other doctors.
    - MichaelSalib 16y ago
      Wait, you mean you have anecdotes? Why didn't you say that? I mean, now that I know you've got some anecdotes, I'm totally convinced.
andrewce 16y ago
It's easy to fire "ineffective" teachers (quotations used to show the mutability and variability of the term, rather than sarcasm). Even with the supposed difficulties presented by tenure, so many school districts are struggling for money that it's a simple matter of invoking the RIF-hammer. It's harder to identify what makes an teacher ineffective, and harder still to identify what makes an effective teacher. Even harder still when lesson plans are mandated from up top and are seen as things to "deliver" (like a pizza, or a newspaper). I worked in a school district in which some of my lesson plans were written for me (down to the "Spend 5 minutes on this, 8 minutes on that" level), and was evaluated (on those days) based on my ability to comply with them to the letter. One of the comments I received on an evaluation was "We were hoping you'd show more creativity in your teaching" of a lesson that was assessed on how well I followed a script and how well I got students to follow the script. Frankly, I think the thing that will truly reform education is an entire change in mindset away from students "doing well on tests" and toward students doing good and interesting and challenging work. That said, your notion of attending professional development is good not only for struggling teachers, but for everyone. It's stupid to expect teachers to improve if the only thing we give them is a carrot or a stick.
- gruseom 16y ago
  I agree that standardized tests suck. I distinctly remember how much they suck. A system based on standardized tests is going to suck. But you know what sucks worse? Utterly incompetent teachers. If we can't even identify and get rid of the most egregious offenders - the kind who tell children "I get paid whether you learn or not" and then proceed to read the newspaper in front of the class -- what hope do we have? Everyone always talks about how overwhelmingly complicated these problems are. No doubt that's true, especially if you accept certain premises. But there are also some no-brainers. The ideas that teachers get jobs for life, you can't fire a teacher for underperforming, and layoffs proceed in order of seniority are mind-bogglingly stupid and evil (as Rhee writes, running the schools "for the benefit of the adults in the system, not for the benefit of the kids"). No one can tell me this hasn't had devastating effects. If the best we can do right now is apply crude and mediocre metrics (like standardized tests) to stanch the social hemhorrage that is the failure of our public school system, we'd better do it. Real education, the kind that prepares whole human beings to do great things with their lives, is absolutely what we should shoot for; anyone working as an educator ought to have that in mind at all times. Nevertheless, a system that produces trained rats is not as evil as one that turns out prison fodder.
anamax 16y ago
> The first is why the teachers that are bad are actually bad. Under what circumstance are you willing to buy a poorly made car? After all, maybe the workers could become better given additional retraining, etc. > The other is how they are measuring teacher performance. Feel free to provide a better measurement scheme. Until then, why should we prefer nothing? In practice, we find that the "need better measurement" folk are actually unwilling to accept any measurement scheme. Yes, even one of their own devising.

by BCM43 16y ago

niels_olson 16y ago

Your physician was identified for medical school, residency, and fellowship based in large part on standardized test scores. And the schools and residency programs are tyrannized by their students and residents scores. I gotta say, it doesn't take long to identify the docs who did well on those tests: they're generally the best docs. They're the most articulate, ask the most insightful questions during their patient interviews, develop the best diagnostic and management plans, and accomplish the most difficult research. I suspect the same goes for lawyers, law programs, and their bar exams, engineers, engineering programs, and their EIT exams, etc. Why shouldn't kids learn to play by the same rules their parents do? Indeed, isn't that what kids want to do?

andrewce 16y ago
Except that the MCAT measures at least some of the core competencies necessary for being a doctor, and includes a lot of vital information which, if you don't have, make it pretty tough to be a physician. Even then, there are plenty of people whose MCAT scores were high enough to get them into medical school who ended up being terrible doctors, just as there are plenty of people who barely got into med school who ended up being terrific doctors. I can't cite any studies, but I imagine that just like anything else the variability within a score band is just as much as the variability between score bands. (Similarly: plenty of bad lawyers have passed the bar, just as plenty of good lawyers failed it before they passed. There's enough variability between the Bar exams of different states to make this possible. I suspect the same is true about engineering programs as well). It may just be that the MCAT is a good indicator of one's ability to become a doctor. Med school admissions are so competitive right now that these schools are able to be pretty selective about who they admit. It may be that these tests were designed at least in part by the professionals in those respective fields to measure knowledge and competencies that directly relate to the domains in question. More to the point, this is a specious comparison because the exams you cite are professional vocational exams which correspond to a specific career path/domain. There's no such vocational test that will adequately cover every child, nor do we even have philosophical agreement about the skills necessary to live a good life, much less the relative importance of such skills, and even if we did, there are so many different ways that things like literacy and numeracy manifest that it'd be downright impossible to construct a test that can even come close to measuring it. The tests you cite are tests that limit who can practice in a given profession, and while they're not perfect, I'll concede their usefulness. More importantly, they're on an individual basis. There'll definitely be some false-positives (people who are good at taking tests but poor at performing the basic skills necessary for competency) and false-negatives, but those can be corrected. It's a whole different ball-game when you're talking about a non-vocational test, particularly one aimed at people who are not yet adults and who have not yet chosen a particular career path.
- djacobs 16y ago
  The MCAT actually doesn't measure much in the way of knowledge, just of speed. The actual questions are easy, and your score (after a certain knowledge/intelligence threshold) is solely determined by how many practice questions you've waded through.
- niels_olson 16y ago
  >MCAT arguement The USMLE exams (Step I: basic science, Step IICK: clinical knowledge, Step IICS: clinical skills, and Step III: clinical problem solving) are more like the bar exams. > this is a specious comparison because the exams you cite are professional vocational exams which correspond to a specific career path/domain. Well, you're throwing in the MCAT, which is an aptitude test based on Bloom's Taxonomy of Educational Objectives, more like the GRE than the USMLE or bar exams. > there are so many different ways that things like literacy and numeracy manifest Literacy and numeracy can't be measured? You sure you want to hang your hat on that? Look, to quote Allah, I mean, pg, himself, (http://www.paulgraham.com/wealth.html http://www.paulgraham.com/wealth.html) > To get rich you need to get yourself in a situation with two things, measurement and leverage. You need to be in a position where your performance can be measured, or there is no way to get paid more by doing more. And you have to have leverage, in the sense that the decisions you make have a big effect. We can easily modify this paragraph to most if not all domains and industries. I leave as an exercise to the reader to modify it for "improving education".
jbooth 16y ago
Well, teaching's different from doctoring/lawyering in a few ways. If you judge, say, a couple defense lawyers strictly on their success rate, put one on OJ Simpson's team and give another a long backlog of public defender cases, then Johnny Cochran's going to be the best layer ever. Does that mean he would've done better with the cases where you're set up to fail? Not to say that metrics are useless, but they can definitely be flawed.
MichaelSalib 16y ago
I gotta say, it doesn't take long to identify the docs who did well on those tests: they're generally the best docs. Do you have any evidence to support this notion at all? Or is thus just some religious notion that we're supposed to take on faith?
- niels_olson 16y ago
  Yes, I'm a doctor. I work with other doctors.
  - MichaelSalib 16y ago
    Wait, you mean you have anecdotes? Why didn't you say that? I mean, now that I know you've got some anecdotes, I'm totally convinced.

andrewce 16y ago

It's easy to fire "ineffective" teachers (quotations used to show the mutability and variability of the term, rather than sarcasm). Even with the supposed difficulties presented by tenure, so many school districts are struggling for money that it's a simple matter of invoking the RIF-hammer. It's harder to identify what makes an teacher ineffective, and harder still to identify what makes an effective teacher. Even harder still when lesson plans are mandated from up top and are seen as things to "deliver" (like a pizza, or a newspaper). I worked in a school district in which some of my lesson plans were written for me (down to the "Spend 5 minutes on this, 8 minutes on that" level), and was evaluated (on those days) based on my ability to comply with them to the letter. One of the comments I received on an evaluation was "We were hoping you'd show more creativity in your teaching" of a lesson that was assessed on how well I followed a script and how well I got students to follow the script. Frankly, I think the thing that will truly reform education is an entire change in mindset away from students "doing well on tests" and toward students doing good and interesting and challenging work. That said, your notion of attending professional development is good not only for struggling teachers, but for everyone. It's stupid to expect teachers to improve if the only thing we give them is a carrot or a stick.

gruseom 16y ago
I agree that standardized tests suck. I distinctly remember how much they suck. A system based on standardized tests is going to suck. But you know what sucks worse? Utterly incompetent teachers. If we can't even identify and get rid of the most egregious offenders - the kind who tell children "I get paid whether you learn or not" and then proceed to read the newspaper in front of the class -- what hope do we have? Everyone always talks about how overwhelmingly complicated these problems are. No doubt that's true, especially if you accept certain premises. But there are also some no-brainers. The ideas that teachers get jobs for life, you can't fire a teacher for underperforming, and layoffs proceed in order of seniority are mind-bogglingly stupid and evil (as Rhee writes, running the schools "for the benefit of the adults in the system, not for the benefit of the kids"). No one can tell me this hasn't had devastating effects. If the best we can do right now is apply crude and mediocre metrics (like standardized tests) to stanch the social hemhorrage that is the failure of our public school system, we'd better do it. Real education, the kind that prepares whole human beings to do great things with their lives, is absolutely what we should shoot for; anyone working as an educator ought to have that in mind at all times. Nevertheless, a system that produces trained rats is not as evil as one that turns out prison fodder.

anamax 16y ago

> The first is why the teachers that are bad are actually bad. Under what circumstance are you willing to buy a poorly made car? After all, maybe the workers could become better given additional retraining, etc. > The other is how they are measuring teacher performance. Feel free to provide a better measurement scheme. Until then, why should we prefer nothing? In practice, we find that the "need better measurement" folk are actually unwilling to accept any measurement scheme. Yes, even one of their own devising.