4 ms·

Well Yes but there's a backstory to that. If you talk to teachers who worked in the pre-union days they'll tell you it was a nightmare. Because it was run by

Well Yes but there's a backstory to that. If you talk to teachers who worked in the pre-union days they'll tell you it was a nightmare. Because it was run by low-level politicians (school board members) who were cowardly to the point of causing havok. Put it this way. Most people don't know the political positions of their individual school board members. So if someone comes to their house and says "this member is bad" they'll generally vote against them on the theory of "better safe than sorry". Which meant every crazy parent who thought their kid was treated unfairly would raise a fuss and get teachers fired because the Principal answered to the School Board members and those members had extremely tenuous holds on their position. Meaning teachers were getting fired fairly regularly, often for doing nothing wrong. Teachers Unions were born out of that abuse. The problem now is we've swung to the other extreme. Where even teachers who are bad can't get fired. But I wouldn't say the problem is the union contracts. If anything it's that we haven't found an appropriate metric for measuring teacher performance.

yummyfajitas 15y ago
If anything it's that we haven't found an appropriate metric for measuring teacher performance. Modern VAM (Value Added Modelling) methods tend to work well. This is basically building a statistical predictor of student performance, and then measuring whether individual teachers students meet this predictor. The teachers unions fight tooth and nail against using them, but they work well.
- disgruntledphd2 15y ago
  Care to provide some evidence for this assertion? I am aware of the some of the research into building multilevel models of student and teacher performance, but one of the issues here is that if these metrics are used, there is no test set which can be utilised to validate the model. This is an absolutely huge issue when you are essentially hiring and firing people based on the metric.
  - yummyfajitas 15y ago
    ...there is no test set which can be utilised to validate the model. The collection of students used to develop the model are randomly partitioned - one partition is training data, the other validation data. This is regression 101. This publication goes into detail: http://www.rand.org/pubs/monographs/2004/RAND_MG158.pdf http://www.rand.org/pubs/monographs/2004/RAND_MG158.pdf Note that if you believe VAM doesn't work, you must also believe that we have no statistical evidence that teacher quality matters at all. Is that actually your belief?
- wonderzombie 15y ago
  "student performance" In practice, this means "standardized test scores." Does anyone actually think that's a good metric? I'll bet you most teachers don't think so. And then I see stuff like this: http://voices.washingtonpost.com/answer-sheet/teachers/new-study-blasts-popular-teach.html http://voices.washingtonpost.com/answer-sheet/teachers/new-s.... This links to an EPI study that casts a substantial amount of doubt on VAM: http://epi.3cdn.net/724cd9a1eb91c40ff0_hwm6iij90.pdf http://epi.3cdn.net/724cd9a1eb91c40ff0_hwm6iij90.pdf. Among other things, it turns out there's a lot of fluctuation of how much "value" a teacher adds from year to year. Teachers in a very high percentile one year end up in the bottom 40% in another year, for example. Actually, downthread you provided a link (http://www.rand.org/pubs/monographs/2004/RAND_MG158.pdf http://www.rand.org/pubs/monographs/2004/RAND_MG158.pdf), which says among other things: "At the current time, VAM may show promise for lower-stakes, diagnostic purposes. Examples include identifying teachers who might be low or high performing so that follow-ups can be done to verify the VAM findings. Inferences would need to be circumspect because of possible bias or sensitivity to the measure, but they could be a starting point for administrators (such as principals or superintendents) to target teachers for more thorough review." There's this, too: "The research base is currently insufficient to support the use of VAM for high-stakes decisions. We have identified numerous possible sources of error in teacher effects and any attempt to use VAM estimates for high-stakes decisions must be informed by an understanding of these potential error." This is far more circumspect than "it works well!" Nevertheless it sounds to me like teachers' VA scores are effectively treated as grades, which is not consistent with the report you linked, nor the EPI report.
  - yummyfajitas 15y ago
    In practice, this means "standardized test scores." Does anyone actually think that's a good metric If it isn't, we've got bigger problems - specifically the fact that we have no ability whatsoever to evaluate whether a school is doing a good job. In that case, we need to fix our measurement process and waste as little money on unproven methods in the meantime. Among other things, it turns out there's a lot of fluctuation of how much "value" a teacher adds from year to year. If this is due to statistical noise, then teacher evaluation needs to be done on a multi-year basis. If this is due to internal variation in teacher quality, then maybe we need to either control the variation or accept that teacher quality varies widely and is mostly out of our control. Note that if VAM doesn't work, then all the statistical studies claiming to show that teacher quality matters for student achievement are bunk. All those studies make some attempt at VAM, and show the variation due to teacher quality is large. Are you asserting that this is the case?
tokenadult 15y ago
Eric Hanushek and Steven Rivkin's conference paper "Generalizations about Using Value-Added Measures of Teacher Quality" http://edpro.stanford.edu/hanushek/admin/pages/files/uploads/HanushekRivkin%20AEA2010.CALDER.pdf http://edpro.stanford.edu/hanushek/admin/pages/files/uploads... explores several of the theoretical issues and facts that are simply not known at present about how to evaluate teachers. The paper points out that for public policy, if existing models for teacher evaluation programs by value added are better than what is currently done to evaluate teachers, even if the value-added estimates are not perfect, then they may be worth implementing more generally. This definitely is an issue that deserves further research. On my part, I think a truly effective teacher is worth her weight in gold, and I would like to see the system identify and reward the best teachers, not least because then other teachers will be likely to learn from their techniques and emulate their examples.

yummyfajitas 15y ago

If anything it's that we haven't found an appropriate metric for measuring teacher performance. Modern VAM (Value Added Modelling) methods tend to work well. This is basically building a statistical predictor of student performance, and then measuring whether individual teachers students meet this predictor. The teachers unions fight tooth and nail against using them, but they work well.

disgruntledphd2 15y ago
Care to provide some evidence for this assertion? I am aware of the some of the research into building multilevel models of student and teacher performance, but one of the issues here is that if these metrics are used, there is no test set which can be utilised to validate the model. This is an absolutely huge issue when you are essentially hiring and firing people based on the metric.
- yummyfajitas 15y ago
  ...there is no test set which can be utilised to validate the model. The collection of students used to develop the model are randomly partitioned - one partition is training data, the other validation data. This is regression 101. This publication goes into detail: http://www.rand.org/pubs/monographs/2004/RAND_MG158.pdf http://www.rand.org/pubs/monographs/2004/RAND_MG158.pdf Note that if you believe VAM doesn't work, you must also believe that we have no statistical evidence that teacher quality matters at all. Is that actually your belief?
wonderzombie 15y ago
"student performance" In practice, this means "standardized test scores." Does anyone actually think that's a good metric? I'll bet you most teachers don't think so. And then I see stuff like this: http://voices.washingtonpost.com/answer-sheet/teachers/new-study-blasts-popular-teach.html http://voices.washingtonpost.com/answer-sheet/teachers/new-s.... This links to an EPI study that casts a substantial amount of doubt on VAM: http://epi.3cdn.net/724cd9a1eb91c40ff0_hwm6iij90.pdf http://epi.3cdn.net/724cd9a1eb91c40ff0_hwm6iij90.pdf. Among other things, it turns out there's a lot of fluctuation of how much "value" a teacher adds from year to year. Teachers in a very high percentile one year end up in the bottom 40% in another year, for example. Actually, downthread you provided a link (http://www.rand.org/pubs/monographs/2004/RAND_MG158.pdf http://www.rand.org/pubs/monographs/2004/RAND_MG158.pdf), which says among other things: "At the current time, VAM may show promise for lower-stakes, diagnostic purposes. Examples include identifying teachers who might be low or high performing so that follow-ups can be done to verify the VAM findings. Inferences would need to be circumspect because of possible bias or sensitivity to the measure, but they could be a starting point for administrators (such as principals or superintendents) to target teachers for more thorough review." There's this, too: "The research base is currently insufficient to support the use of VAM for high-stakes decisions. We have identified numerous possible sources of error in teacher effects and any attempt to use VAM estimates for high-stakes decisions must be informed by an understanding of these potential error." This is far more circumspect than "it works well!" Nevertheless it sounds to me like teachers' VA scores are effectively treated as grades, which is not consistent with the report you linked, nor the EPI report.
- yummyfajitas 15y ago
  In practice, this means "standardized test scores." Does anyone actually think that's a good metric If it isn't, we've got bigger problems - specifically the fact that we have no ability whatsoever to evaluate whether a school is doing a good job. In that case, we need to fix our measurement process and waste as little money on unproven methods in the meantime. Among other things, it turns out there's a lot of fluctuation of how much "value" a teacher adds from year to year. If this is due to statistical noise, then teacher evaluation needs to be done on a multi-year basis. If this is due to internal variation in teacher quality, then maybe we need to either control the variation or accept that teacher quality varies widely and is mostly out of our control. Note that if VAM doesn't work, then all the statistical studies claiming to show that teacher quality matters for student achievement are bunk. All those studies make some attempt at VAM, and show the variation due to teacher quality is large. Are you asserting that this is the case?

tokenadult 15y ago

Eric Hanushek and Steven Rivkin's conference paper "Generalizations about Using Value-Added Measures of Teacher Quality" http://edpro.stanford.edu/hanushek/admin/pages/files/uploads/HanushekRivkin%20AEA2010.CALDER.pdf http://edpro.stanford.edu/hanushek/admin/pages/files/uploads... explores several of the theoretical issues and facts that are simply not known at present about how to evaluate teachers. The paper points out that for public policy, if existing models for teacher evaluation programs by value added are better than what is currently done to evaluate teachers, even if the value-added estimates are not perfect, then they may be worth implementing more generally. This definitely is an issue that deserves further research. On my part, I think a truly effective teacher is worth her weight in gold, and I would like to see the system identify and reward the best teachers, not least because then other teachers will be likely to learn from their techniques and emulate their examples.