3 ms·
Teaching is a lot like programming in this respect -- if you try to enforce metrics of "performance", you usually end up with optimization for those metrics and
by DarkShikari 15y ago
Teaching is a lot like programming in this respect -- if you try to enforce metrics of "performance", you usually end up with optimization for those metrics and dumber, less-educated students. Similarly, students tend to optimize too, just from a different standpoint. Examples of this:
If you rate teachers based on the grades their students get, they will usually award their students higher grades for the same (or worse) performance. This isn't just a joke; many, many professors have reported their deans bearing down on them for awarding too many bad grades, often to students who are so incompetent that they cannot write basic English sentences or perform long division. The solution -- not remedial courses, or letting them drop out (that would hurt financial aid income) -- but letting them pass even though they lack the basic skills to learn in that class, let alone pass. This also leads to the student mentality that they are being "given" grades (and thus can beg for higher grades) as opposed to earning them through their own effort in the class.
If you rate teachers based on the number of their students pass a standardized test, the teachers will focus solely on the contents of that standardized test and put a huge amount of effort into educating the few students who Don't Get It, at the cost of all the students that do. I had this experience in primary and secondary school, where teachers lamented that they had to go over material for the Standards of Learning tests constantly and repeatedly, dramatically lowering the amount they actually had time to teach. The end result is a quick dropping of everyone to the lowest common denominator -- which often just ends up shifting the entire bell curve to a lower point.
If you rate teachers based on student evaluations, your results are nearly useless. Most students don't bother to write in much detail, since they're just glad to be done with the final exam (I did this too!)-- and often they use it for "revenge" on professors they thought were too hard. The teachers who get the best evaluations are often (though not always) the teachers who gave the easiest work and grades, and whose students turn out to be the most woefully unprepared for what comes next. Just like customers of your software are not always good judges of what they really want, students are often terrible judges of what good education actually is.
As far as I can see, nobody has come up with a magical metric for rating teachers. If anything, most of the metrics out there are outright counterproductive -- and you can see it in schools across America, where teachers are being constantly urged to inflate grades for their 'customers', overlook plagiarism, ignore the most talented students in favor of the least talented ones, and help drive down the quality of education for the next generation.
- moonchrome 15y ago>As far as I can see, nobody has come up with a magical metric for rating teachers. If anything, most of the metrics out there are outright counterproductive -- and you can see it in schools across America, where teachers are being constantly urged to inflate grades for their 'customers', overlook plagiarism, ignore the most talented students in favor of the least talented ones, and help drive down the quality of education for the next generation. You can do things to improve the quality of data - use multiple metrics, don't announce them up front and certainly don't standardize - so that optimizing for a specific test doesn't justify the cost, also change testing as needed/with dynamic feedback and use the results to get insight in to the problem. Computers can only help with this sort of thing. Ultimately it's about defining the objective of education, what environment students are supposed to be prepared by it and exposing them to that environment, just like you want to test the system in production as soon as you can, benchmarks are always synthetic and can't replace "real world" testing. Data won't capture everything and could in fact be 100% misleading - that's the nature of imperfect knowledge - but if used correctly it can help provide insight in to situation. Just don't conflate benchmarks and reality, use it as a tool for improved feedback/analysis. Of course this approach is absolutely incompatible with bureaucratic systems. Heck the entire education system is based around optimizing for arbitrary/artificial evaluation metrics (all that crap about higher education and signaling, that starts with primary education). As for favoring least talented over most talented, that's just different objectives, I don't think it changes with more quantifiable information about students performance, do you want to raise the bottom at the expense of the top.
- yummyfajitas 15y agoIf you rate teachers based on the number of their students pass a standardized test, the teachers will focus solely on the contents of that standardized test and put a huge amount of effort into educating the few students who Don't Get It, at the cost of all the students that do. Your issue here is not with standardized tests. Your issue is with the goals of the school system. I.e., you want schools to maximize mean(performance_data), but they are actually trying to maximize len([p for p in performance_data if p >= pmin]). Gathering good data is not the problem, it just reveals the problem.