6 ms·
New Study: Rewarding Good Teachers and Firing Bad Ones Accomplishes Nothing
- Alex3917 8y agoPerhaps the least surprising finding ever, given the previous research: http://www.citevault.com/#vam-efficacy http://www.citevault.com/#vam-efficacy http://www.citevault.com/#vam-statistical-problems http://www.citevault.com/#vam-statistical-problems http://www.citevault.com/#vam-implementation http://www.citevault.com/#vam-implementation
- lend000 8y agoAll this means is that better methods will need to be invented to score teachers, not that it's impossible. For example, one thing I've seen missing in these studies is a system design that attempts to control for the distribution of students in a given incoming class, which seems like the single most predictable metric for end-of-year performance.
- empath75 8y agoOr the aptitude of teachers simply aren’t that relevant and the problem exists elsewhere, as in the curriculum or home environment of the kids, etc.
- Alex3917 8y ago> All this means is that better methods will need to be invented to score teachers, not that it's impossible. No, it is impossible. That's the whole point. If it wasn't impossible it'd be a great idea.
- deleted 8y ago[deleted]
- lend000 8y agoI suspect some in the HN community assume that because public school curriculums were generally so easy and aligned with textbooks such that they got all A's regardless of the teacher, that the teacher does not matter. No, the teacher definitely matters, as does the inflexible and oversimple curriculum and a million other factors. Switch 'teacher' with 'professor' and I highly doubt you would argue that quality no longer matters. The difference is that professors are less bound by stringent curriculums and teach more advanced material. This is where everyone really feels the difference. Whereas for public grade school teachers, an academically inclined student will only struggle under exceptionally bad teachers, less well academically inclined students will suffer without decent teachers even in grade school. The first quarter/semester at a good college typically provides a reality check for students who breezed through primary education.
- neel8986 8y agoCan it be the case that the constant fear of being fired is not an ideal environment for a teacher? Teaching needs a long-term commitment which needs a different kind of incentive than a bonus. I have similarly observed that researcher who wants to do fundamental research are not very comfortable in startup environment which lacks security.
- MBCook 8y agoIt could also be that the quality of teacher just can’t overcome other systematic problems like discipline, required curriculum, lack of flexibility, home environment, materials used, or something else.
- briandear 8y agoWe could argue that the fear of being fired ought to motivate as well. Professional athletes operate under this model. Perhaps we could implement a teacher’s “draft.” Lower performing but still competent enough teachers could be “traded” and stars could result in bidding wars. At the end of each “season” the lowest school gets first pick in that summer’s draft. Substitutes could be like free agents. And winning “seasons” could be financially rewarded with bonuses. There could be a system of measurement based on improvement of students rather than just raw scores, so even lower performing students could help the team “win” by improving even if they are still not meeting the overall standards. You might find teachers wanting to go to tougher schools because bringing a kid from a 20 to a 50 earns more “points” than bringing a kid from an 80 to an 85. I haven’t fully thought this out yet, but we do know one thing: if you are in a hole, stop digging. If something isn’t working, brute force rarely helps. We need innovation in education but not just pedagogical— but with the entire incentive structure for both teachers and students. Crazy idea, I know. But sometimes crazy works.
- chvid 8y agoNext up: fire the educated teachers and just let random grown ups run the class ...
- protonfish 8y agoSudbury Schools https://en.wikipedia.org/wiki/Sudbury_school https://en.wikipedia.org/wiki/Sudbury_school have no curriculum, they let the students do whatever they want. Graduates don't differ significantly from traditional schools in standard educational assessment testing.
- aetherson 8y agoNeat, glad to know that we don't ever need to give teachers raises.
- lawnchair_larry 8y agoAlternate title: we thought money is what good teachers find rewarding, but we were wrong
- karmakaze 8y agoOr: A poorly conducted experiment with small sample sizes and larger external variables is said to "Accomplishes Nothing".
- Latteland 8y agoHighly funded study that has 100s of millions of dollars and the second most wealthy person on earth was unable to accomplish any improvement in schools even with trying very hard to prove his idealogical desires.
- jVinc 8y agoThat's not the story at all. They meassured actual performance and fired teachers who showed bad performance, and rewarded those showing good performance. This was not just a test of dangling money in front of teachers and expecting them to improve their performance. What you could get out of this study is however that it doesn't matter if teachers find the job rewarding or not. It doesn't matter if you keep on bad teachers, in fact the study might point to the fact that teachers are not the bottleneck of student performance at all.
- fancyfish 8y agoGood point here, that we ought to expand the scope of how we look at student performance. We're quick to blame teachers, and indeed teachers probably have as much an impact as anything else, but measuring up teachers and expecting big changes isn't the greatest idea. Yes, it gets more complicated looking at more nuanced measures. Like resources available for the students, their home lives and personal situations, their time available to study and focus. Then again maybe they should look at discrepancies between teachers in districts that pay well and districts that pay less. Perhaps the answer isn't to pay some more and fire others, but rather pay all of them more! I'm a fan of the Education Endowment Foundation(1) on this which looks at all the measures you can think of to try to improve education practices (in the UK). [1]: https://educationendowmentfoundation.org.uk/ https://educationendowmentfoundation.org.uk/
- hueving 8y agoIf it accomplishes nothing then the 'quality' of the teachers is irrelevant.
- rossdavidh 8y agoNot super-surprising to me, but I agree with the postscript: "kudos to the Gates Foundation for running an honest test. They designed it well; they funded it properly; and when it was over they hired a third party to provide an honest assessment of what happened. That’s the way it should be done."
- maxxxxx 8y agoMaybe they should put more effort into figuring how to create an environment where teachers and students can perform well. It's not like the teacher is the only player here. Students and their environments play a big role, curriculum and so on. Reminds me of the 10x programmer nonsense. Give people a decent work environment where they feel respected and most will perform well.
- sizeofchar 8y agoEvery teacher knows that learning performance depends primarily on students. The very same lecture, with same methods, teacher, books, resources, have drastically distinct performance with different classes.
- tw04 8y agoDepending on the age of the student, it also greatly depends on the parents.
- epx 8y agoThis.
- deleted 8y ago[deleted]
- Viliam1234 8y agoThere is a huge informational asymmetry when debating education: almost everyone was a student once, but most people have never been teachers. This is how people know that the quality of education in the same class can change dramatically when you replace a teacher. But most of them don't know that the same is true in the opposite direction: the quality of education can change dramatically when the same teacher teaches different classes (or just when the most disruptive child in the class happens to be sick at home today). There is also no teaching method that would make everyone happy. Some students prefer to work in teams, some prefer to work alone. Some students prefer to have explained how things work, some insist on receiving a list of stuff to memorize. Most people who have a recipe "how to fix education" simply want everyone to do it their preferred way. (So perhaps the best approach would be to have different kinds of school for everyone? Problem is, people would not want to honestly admit some of their preferences. If you listen to internet, everyone prefers understanding to memorization. Then you start explaining stuff, and students follow every sentence you said with "do we have to remember this for the exam?")
- theothermkn 8y agoOne of the canonical motivating sketches for total quality control, developed by Edwards Deming et al, is a mythical factory with three production lines, each with their own line manager. The lines are performing equally well, staying within a bell-shaped distribution about approximately the same mean. (Widgets per hour, or defective unit rates, or what have you.) As factory manager, in an effort to improve the situation, you decide that, at the end of each month, you will promote the top performer and fire/demote the laggard. Congratulations! You promote and demote/fire a line manager every month for six months. However, you notice that the performance of your lines has remained the same. What happened? You were essentially using the performance numbers of your three lines as a random number generator. The system was in control, producing no results outside of three standard deviations. You promoted and fired at random. In order to improve the output of a system, you have to understand it. You can’t just blindly incentivize on the randomly distributed results. If this research leads to a more humane work environment for teachers, freer than before of misguided incentive programs, then that will be a good thing, indeed.
- jeremyt 8y agoYour comment assumes that educational achievement is a "randomly distributed result". It may or may not be, but you can't just assume it.
- pjc50 8y agoEverything is a randomly distributed result unless proved to be affected by a specific input. That's a fundamental assumption of statistical process control.
- reallydude 8y ago> Everything is a randomly distributed result unless proved to be affected by a specific input. Nope. The former is invalidated by the latter, if you take the statement at face value. Everything appears randomly distributed...oh wait that isn't true either as you always have ranges and generally 2 axes (one being time). Most importantly, this doesn't contribute the conversation in any way. -1
- dqpb 8y agoIf teacher salaries were $120k and up I bet you'd get a whole new class of teachers.
- lobotryas 8y agoWould we really? At some point a higher cost will stop resulting in better outcomes.
- zeth___ 8y agoAnd yet we keep paying CEO's more and more. At least teachers are only in control of one class room. The most mischief they can do is to buy animatronic dinosaurs for the one lesson where they mention prehistory to kindergartners.
- innomatics 8y agoThey'd certainly be a different class. I'd imagine a much more competitive selection process for entry into the profession, and the emergence of a generation who excelled at succeeding in that process, but might not be the type to commit to a life of advocating for children. Especially if they possess the skills/drive to switch to other more lucrative occupations. Don't get me wrong though I think that teachers should be well paid, more than enough to live comfortably and happily so they can enjoy what should be a very fulfilling job for them.
- rayiner 8y agoYou’d get better teachers no doubt. But would it have any effect on student outcomes?
- jedberg 8y agoAs anyone who’s been a teacher knows, the parents are the most important part of a kids education. The teacher is there to set up the framework and guide the parents as to what to teach. The parent has to bring it home. The single biggest indicator of future student success is how many of the parents at the school are married with only one working parent. I.e the parents with the most free time to help their kids.
- jdmichal 8y agoI've been told in discussions with school board members that parent involvement is the number one driver of student success. Not teachers, and definitely not dollars per student.
- Eridrus 8y agoI'm left with two thoughts after reading this, either teacher quality doesn't matter or our ability to measure teacher quality is garbage. The latter is no more satisfying because when you can't tell the difference between good and bad you get a market for lemons, so you end up with a pile of bad teachers despite the fact that good ones would help.
- acdha 8y agoI’m not sure this is less depressing but I think it’s not that teacher quality doesn’t master but that we focus on that societally because it’s the easiest to do something about: making teachers’ jobs have higher stakes, etc. is much cheaper than addressing poverty or changing culture so we keep hoping that one more round will do it even though we have close to a century of studies suggesting that we should focus on poverty instead.
- ncphillips 8y agoThe quality of the teacher does matter, but OP says it’s not the constraint for most children. This is basic theory of constraints: a chain is only as strong as its weakest link. The quality of a students education is a function of many variables, and the teacher is just one of those variables. This is why rewarding a teacher based on their classes performance is unfair. By luck they may either get a class filled with exceptional and privileged students, or a class of children with abuse and attention problems. The only way to really get a good measure the quality of a teacher, would be to control the quality of the parents, and all the other variables too.
- curtis 8y agoI find it hard to believe that teacher quality doesn't matter at all. I'm not sure that that's what this study is telling us. Maybe it's just telling us that there is no significant (or at least measurable) difference between teachers that are a little bit below some threshold and teachers that are a little bit above. It surely isn't telling us what would happen if you replaced all the teachers in a school with teachers recruited from the top 5 or 10 percent of all teachers in the country. I really want to know what would happen if you did that. On the other hand, suppose it works really well. We still can't replace all the teachers in the country with the top 5 or 10 percent of all the teachers in the country, so it's not like that would be actionable.
- stevenwoo 8y agoMaybe the teaching system in America is unfixable with the small changes as suggested and tested by the long term but small Gates Foundation study, and revolutionary changes more akin to the Finnish public school system which ironically adopted American academic theories about education wholesale to improve (usual caveat it's easier to change a small country versus the USA or maybe even a state in the USA given political reality). https://www.theguardian.com/education/2015/jun/17/highly-trained-respected-and-free-why-finlands-teachers-are-different https://www.theguardian.com/education/2015/jun/17/highly-tra...
- mlthoughts2018 8y agoI think your final observation is the important part. These studies cost a lot of (often scarce) funding resources. We probably shouldn’t spend the money on hypothetical strategies that cannot be practically implemented. From a practical point of view, we necessarily want to know if there is a way to distinguish between teachers at or around an average level, since this is what necessarily applies to most cases. (Research on MOOC approaches, video lectures, etc., could be a separate strand of experiment though, to see if it can leverage the top decile to a broader audience). This study is saying (modulo the usual limits of generalizing from just one study), that teachers a little above the threshold don’t cause compounding improvements in outcomes over a ~5 year period. But as always it’s rife with possible issues. - maybe bonus incentives weren’t high enough to induce any different effort? - maybe thresholds for distinguishing good and bad teachers were not set right, or need to be dynamically adaptable to different circumstances - maybe there were just confounders, like the mentioned school merger, making student performance primarily causally driven by non-teacher factors over these years in these locations - maybe the particular student cohorts just respond differently to teacher-based treatments than students in general would - maybe students or teachers are affected more by social engagements than learning activities, so that the firing of a beloved, but ineffective, teacher causes a student to remain underperforming, or even resentful of a new, but high-performing teacher. Whatever the case, this single study does not really offer enough to conclusively (or even suggestively) say that rewarding good teachers or firing bad teachers has no effect. It had no effect in this case, with this methodology. Would need to see replications and variations of the type of experiment before aligning any beliefs about budgets or teacher incentives towards this result.
- michaelscott 8y agoThis is unfortunately a complex, multivariate system problem and won't rely solely on the "quality" of a teacher in it. In addition to the behind-the-scenes work most teachers have to deal with, such as departmental objectives, I think great teachers enter the profession in order to benefit children, not to get that fantastic paycheck . It's a socially focused decision, not an financial one, so if I were designing a system to grade a teacher's quality I would focus on attending to their efforts with students, not throwing more money at them.
- exotree 8y agoMoney may not be the end all for people who go into this profession, but it most certainly does matter. See the protests over the past couple of years in areas where teacher pay is low.
- Gimpei 8y agoAnyone check this for the quality of the study?Doesn't look like this was randomized. Also would have been nice to see more pretreatment years for evidence of a trend but maybe that was in the actual report.
- jupiter90000 8y agoIt seems like the primary purpose of school is to make sure kids have been well trained to sit still and follow the rules, learn how to jump through societal hoops, and have a place to be for parents who can't/won't deal with them all day. How much can a teacher really do, trying to enforce all this and get across a subject to 15+ kids in a room, who have various degrees of motivation (sometimes none) to want to learn anything? It seems like there must be a better way to help kids acclimate into adults and give their parents a break/opportunity to work.
- pdfernhout 8y agoOne alternative I suggested around 2010: "Towards a Post-Scarcity New York State of Mind (through homeschooling)" https://www.pdfernhout.net/towards-a-post-scarcity-new-york-state-of-mind.html https://www.pdfernhout.net/towards-a-post-scarcity-new-york-... "New York State current spends roughly 20,000 US dollars per schooled child per year to support the public school system. This essay suggests that the same amount of money be given directly to the family of each homeschooled child. Further, it suggests that eventually all parents would get this amount, as more and more families decide to homeschool because it is suddenly easier financially. It suggests why ultimately this will be a win/win situation for everyone involved (including parents, children, teachers, school staff, other people in the community, and even school administrators :-) because ultimately local schools will grow into larger vibrant community learning centers open to anyone in the community and looking more like college campuses. New York State could try this plan incrementally in a few different school districts across the state as pilot programs to see how it works out. This may seem like an unlikely idea to be adopted at first, but at least it is a starting point for building a positive vision of the future for all children in all our communities. Like straightforward ideas such as Medicare-for-all, this is an easy solution to state, likely with broad popular support, but it may be a hard thing to get done politically for all sorts of reasons. It might take an enormous struggle to make such a change, and most homeschoolers rightfully may say they are better off focusing on teaching their own and ignoring the school system as much as possible, and letting schooled families make their own choices. Still,homeschoolers might find it interesting to think about this idea and how the straightforward nature of it calls into question many assumptions related to how compulsory public schooling is justified. Also, ultimately, the more people who homeschool, the easier it becomes, because there are more families close by with which to meet during the daytime (especially in rural areas). And sometime just knowing an alternative is possible can give one extra hope. Who would have predicted ten years back that NYS would have a governor who was legally blind and whose parents had been forced to change school districts just to get him the education he needed? So, there is always "the optimism of uncertainty", as historian Howard Zinn says. We don't know for sure what is possible and what is not."
- catchmeifyoucan 8y agoThis title is really misleading. There are many other factors that this article does not cover or examine, and it reaches a premature conclusion. Furthermore, standardized exam measures are poor way to understand teacher impact. From what I remember from school, the best teachers made me understand, the poor ones helped me memorize.
- maxander 8y agoI've seen more sophisticated schemes for teacher evaluation, in which the metric takes into account the ability of the students as they enter that teacher's class. So, for a student who got a B, say, in grades N < 5, there's an expected grade value for them to achieve in 5th grade; that student counts towards the teacher's "score" only insofar as they do better or worse than that expected value. Theoretically, this puts teachers on even footing, even when one might be teaching a class of Harvard-prep elementary schoolers and another teaching a class of inner-city kids who have been educationally neglected their entire lives. And data from student performance over time in the current educational system should be entirely sufficient to train the model. It's not clear if the study referenced controlled for student variance this way; if so, it would readily explain the lack of results.
- AmericanChopper 8y ago>It's not clear if the study referenced controlled for student variance this way; if so, it would readily explain the lack of results. This headline is pretty click baity. A “better” teacher is one the produces better outcomes for their students. Firing the “bad” teachers, and rewarding the “good” teachers will obviously lead to better outcomes for students. This experiment was only testing whether the metrics they were using truly measured the quality of the teachers, and whether their insentive structure was productive. Turns out they weren’t.
- turingcompeteme 8y agoMy sister is a teacher, and had been recognized as "high performing". She was even given some type of award. She always taught in pretty well off areas where students came from supportive families. A few years ago she moved to a school on the edge of a native reserve. She is now officially a "low performing" teacher. She insists she works twice as hard as before and has much more positive impact on the lives of her students. She's been robbed and beat up by students. Yet she now devotes her life to these kids, and would probably be fired in this type of experiment. My guess is that the 'good' and 'bad' teachers in the study were almost totally dependent on the kids they had.
- jackfraser 8y agoThis sort of research is amusing in a sad way. IQ studies on large populations have revealed data for nearly a century that would easily have predicted the outcome of your sister's efforts. The scores for teachers have to be normalized against the best available data for the demographic composition of their classroom, if we're going to bother with it at all. Unfortunately, people don't like what the IQ scores show, and so instead the Gates Foundation has to spend millions on a grand experiment that "teaches us nothing" as Mother Jones would put it, mostly because they refuse to acknowledge just how precisely it confirms the uncomfortable things we already know.
- turingcompeteme 8y agoThe parents and grandparents of the children in her class were taken away from their families and put in residential schools where they were abused and mistreated their entire lives. They were then released back into the world with a very poor education, and obvious mistrust for the government. Friend of hers have taught in schools where a cycle of poverty inflicts the population. Most children are raised by single mothers because the father is in jail. When the father is released he has no opportunity to make a honest living for himself and the cycle continues. The uncomfortable things we already know have nothing to do with IQ. It has everything to do with centuries of mistreatment of groups of people.
- 8y ago
- asafira 8y agoWhat are good resources to learn more about what works/ what doesn't in improving education? At the end of the article, they add that the study deserves a lot of credit even though it didn't work. Kudos! I was pretty happy to read that =).
- randyrand 8y agoThis conclusion is amazing! Finally we have a study that says hiring good teachers does not matter. And that going to teaching school to become a better teacher is not important! If this conclusion is correct, teachers can be replaced low wage unskilled laborers! /s This (possibly MotherJones simplified) conclusion must clearly be bogus. Perhaps I should take a look at the actual study, because this article is terrible.
- mabbo 8y agoI feel like the real problem is in how we measure teachers. It shouldn't be about how well the students do, it should be about how well the students do relative to expectations. Imagine a school in a well-to-do area where we expect all the students to do well and get A+'s. If an entire class of students gets A- on a standard test, that means the teacher actually negatively impacted the students. But because they all got A-, we reward the teacher for having done such a great job. I propose this: for each student, far ahead of time, guess how well they're going to do on a standardized test based on previous tests, family situation, zip code they live in, etc. Predict based on all the things other than the teacher. Now measure a teacher on how well they did compared to expectations. To do even better, don't rate teachers based on how well the students do on the test the same year they teach them, but rate them based on the following 3 years. Now you're asking what the long term impact the teacher has on students, on average. That's a useful measure to pay bonuses on. The problem is that politically, it's hard to explain to the general public (or to politicians).
- shanghaiaway 8y agoThat is not a useful measure because there are an unlimited number of factors that influence the outcome.
- candiodari 8y agoSomeone should make a study about management theories. The issue is that the major examples of management theories were ... fraud. Taylor ... was a fraud [1]. Lied about his data in order to get paid more. Even paid the people he "tested his theory on" to act out the outcomes he wanted to see for company higher ups. And he acted like what I've come to expect from managers: first he commits scientific fraud, then writes a book and titles it "Principles of Scientific Management". Because when you go for bullshit, the big lesson is: don't just bullshit a little. Go all out. Weber and Fayol were no different, and even Ford ... let's just say that if you focus on his technical accomplishments and skip all the management bullshit you will not be missing much (besides a big part of Ford's ideas were high wages, so I doubt many managers will be repeating it). As can be seen, among many other places, on wikipedia [2], the fact that Taylorism was an outright fraud has not stopped pretty much every management school in the country from expanding and pushing on with his "work". It's like most of the famous economists. The top ones were simply frauds, who got important positions like chairman of the Fed, or secretary of the Treasury and just happened to be in office during big events. Their theories were wrong (really fraudulent, as they didn't believe in it at the time, which makes it a fraud rather than just being wrong, and Keynes, Hayek, Friedman and Greenspan have admitted as much, and let's face: they were "the greats". If they were frauds, then people like our current Mnuchin, Powell or Geithner aren't even frauds : they're ignorant and actually proud of that (Mnuchin has said in an interview that not understanding much economics makes him a better secretary of the treasury). Yellen actually had credentials, and seems to need another year or two before she'll confirm she was a fraud too, she has already admitted her theories were wrong, so all she has to do is also admit that she knew they were wrong when she decided to implement them. And boy, she was, like Keynes, more than a little bit wrong). Once money or power gets involved, humans, "scientist" or not, become lying bastards. At least this research was honest. But it's not saying what people want to hear, and so it will be ignored, as it wouldn't contribute (and that's being polite) to the careers of the decision makers implementing these policies. Meanwhile, Trump is happily pushing on Obama's efforts in this regard, who simply continued from Bush's policies, ... and so on and so forth. Don't worry: Trump is not about to let such a detail as science stand in the way of looking good to a few people, and let's just not kid ourselves that Hillary or whoever gets elected in 2020 will be better. [1] https://www.theatlantic.com/magazine/archive/2006/06/the-management-myth/304883/ https://www.theatlantic.com/magazine/archive/2006/06/the-man... [2] https://en.wikipedia.org/wiki/Scientific_management#Relationship_to_Fordism https://en.wikipedia.org/wiki/Scientific_management#Relation...
- derpsauce1 8y agoMother Jones is an extremely left leaning website. Teachers Unions are among the core groups supporting democratic and liberal candidates. I love a good study but it is possible that they are misinterpreting the results of this study on purpose. During the last election, I became very sensitive to this type of media manipulation. Tread carefully.