7 ms·

I'm surprised to see so many HNers come out against standardized testing. Sure, it's imperfect and this teacher got screwed, but how are administrators supposed

• rapala 14y ago
  So if you have 10 good teachers, you just rank them and kick out the "worst" 2? If your rank the teachers in a strict order, you have to make someone the worst and someone else the best teacher. But is the administration going to check how much worse the worst teacher really is? I doubt it.
  • peterhunt 14y ago
    No. You set performance goals for teachers specific to the teacher and district, evaluate the performance objective and make educated decisions based on that data. It doesn't need to be ranked, but it does need to be empirical.
    • ColinWright 14y ago
      I'm interested to know how you set objective performance goals.
      • peterhunt 14y ago
        "Did test scores improve relative to everybody else year over year?" is a pretty good start. Of course, by definition, this won't work for all districts.
        • ColinWright 14y ago
          Actually, based on my experience of dealing with hundreds of teachers and thousands of students, that won't work either. Test scores are a poor proxy for what students actually know or can do, especially in mathematics. The original linked article provides evidence for that. You put in the caveat "Of course ... this won't work for all districts" but then you're arguing for exceptions to be allowed. That's just a mess. There is no simple fix. There are bad teachers (and I'm leaving the term undefined - it's a bit like porn - undefinable, but recognisable) who get good test results and glowing evaluations, and there are superb teachers who get mediocre test results and undistinguished evaluations. It's easy from the outside or from a limited perspective to suggest "obvious" methods of assessment or "obvious" actions to improve the situation, but in the end, no one has really defined what they mean by "good teaching," so proposing assessments of something undefined will just result in more proxies to be distorted.
        • rapala 14y ago
          Except that now you are ranking, by introducing the notion relative to everybody else. Why should test scores improve year over year? If everyone is getting a 2/6 then yes, but students in the same grade are not going to get smarter every year. There is the major flaw in using student tests to measure teachers IMO. You are going to get a bell curve around 4/6 from the test results for majority of the teachers. When you don't, it is really hard to tell whether the reason is the teaching, the curriculum, the students or the test and its grading.
• Natsu 14y ago
  Good data is helpful. Bad data is worse than useless. You're actually better off with no information than with bad information.
  • peterhunt 14y ago
    I agree with this, but just because a doctor loses a patient once in a while doesn't mean that we should give up on medicine.
    • DougBTX 14y ago
      What if the test makes good doctors quit? (As seems to be the case here.)
      • peterhunt 14y ago
        I don't think that this really fits into my analogy, but if losing a few good teachers is the price we need to pay for preventing a toxic culture of no-accountability where administrators are not allowed to make metrics-based decisions then so be it.
        • kbolino 14y ago
          If you have the wrong metrics, you cannot make the right decisions. If you are throwing away good teachers because the rest are really good at gaming the tests then you have done far more harm than good.
        • Natsu 14y ago
          The analysis shows that the test results contain zero information, unless you believe that teachers change quality randomly every year. If correlation begins to appear in those evaluations, it will be because the test has been gamed.
    • jkn 14y ago
      This is not a correct analogy to "bad information is worse than no information". A better analogy would be a medicine that typically leaves patients worse off than they were before treatment.
• gcp 14y ago
  The problem is that the metrics can be gamed, and there's incentive to do so. Nobody is arguing against testing. They're arguing about doing tests where you know the results will be deeply flawed.
• pbhjpbhj 14y ago
  >Give me a better metric for success that you can measure over the course of a year. // Define success. If one gets the best score but has no social skills, no friends, no fun and no creativity then is that success. This (only considering a test score) is like doing A/B testing but only looking at impressions and not at conversions, or only looking at conversion rates and not at conversion value. In web terms you can succeed by reducing both your impressions and conversion rates (by taking more money of less people and only attracting motivated customers). Sorry that's more destructive than constructive WRT what the right course of action is. Are there [large scale] education systems that consider happiness of pupils, reduction in bullying, positive social interaction and such to be metrics to assess and improve?
  • peterhunt 14y ago
    Sure. More metrics to track should definitely be number of parent complaints, disciplinary actions, and mental health incidents.
• raphman 14y ago
  Unlike website visitors, the teachers in a school are not acting completely independent of each other, but part of a social group. Even if you could adequately measure their 'teaching quality', you might miss factors that contribute to the success of the school as a whole. And measuring 'teaching quality' is not a simple as counting ad click-throughs. Education is at least as much about helping people learn about their role in the world, as it is about learning facts, concepts, and tools. Just measuring the increase of students' scores on standardized tests misses a lot of the impact teachers have on their students' minds. Nobody in his right mind would judge a developer solely on the amount of features he has implemented during the previous year, but also on how robust and maintainable the code is. I you want to take these factors into account, you have to actually read the code, understand it, and ideally talk to the developer about it. Unless you do this, you can not judge the quality of the code. The same holds true for evaluating teachers, imho. Administrators need to talk to students, parents and teachers and form a comprehensive image of a teacher's ability. I really think it is that simple. Certainly not perfect. But not less perfect than standardized testing. As a German, maybe I am misunderstanding some properties of the US educational system.
  • jkn 14y ago
    This is spot on. Does anyone know of a single instance of standardized teacher testing that recognizes important teacher qualities beyond their skill in imparting knowledge to students?
  • ig1 14y ago
    Doesn't that just imply that tests are failing to measure what we want teachers to teach and we should fix the tests ? The whole reason that test results are being used as opposed to manual assesment, is that test results have proven to be a better predictor of long term outcomes. Test results aren't perfect but they're better than what was previously used.
    • bgruber 14y ago
      "test results have proven to be a better predictor of long term outcomes" i'm not sure what long term outcomes you're referring to and how they were measured, but if they're education related the measurement was probably more test results. Yes, test results are a good predictor of future test results.
• jkn 14y ago
  Give me a better metric for success that you can measure over the course of a year. This is assuming that the net effect of the tests is positive, thus we should keep them until we find something better. But a lot of comments are pointing at the harm caused by standardized testing. For people who think the net effect is negative, the rational thing to do is get rid of them until we find something better.
• Jabbles 14y ago
  I don't think you should be optimizing for test scores though, you should be optimizing for "education", which is ill-defined and hard to measure. In the analogy to A/B testing, you wish to maximise your profits, not the number of email addresses you collect.
• adrianhoward 14y ago
  I'm surprised to see so many HNers come out against counting lines of code. Sure, it's imperfect and this developer got screwed, but how are managers supposed to make informed, data-driven decisions on what the right course of action is? Many of us extol the virtues of A/B testing landing page designs and gathering deep metrics and analytics but turn around and slam managers for making decisions based on line code counts.
  • yummyfajitas 14y ago
    Your post is a logical fallacy. The fact that a bad metric exists does not mean all metrics are bad. To address your specific fallacy, the output of coding is heterogeneous while the output of teaching is homogeneous. If I code today, it's a realtime optimization system. Tomorrow it might be a search product. When I taught, the output was always the same: students who understand calculus.
    • yequalsx 14y ago
      The post is a parody. It points out the flawed reasoning used in the parent. It appears you are claiming that when you taught the output was the same each day. Each day you taught your students learned calculus and understood it. Each day you may have taught an aspect of calculus but the topics varied from day to day. Furthermore, assuming undergrad level, it can't be believed that each of your students understood each topic you taught. The dichotomy you've laid out between programming and teaching is not apt.
      • yummyfajitas 14y ago
        The output was the same each semester, not each day. It was the same for me and the guy down the hall. It was the same for every person teaching calc 1 between the last curriculum change and today. Not every student understood every topic. So what?
        • yequalsx 14y ago
          Then the output isn't the same if not all students had the same outcome. Furthermore you used day as the time period when talking about programming. I assumed as a matter of consistency you meant to use roughly the same time period for teaching. Every semester I teach calculus is different. Two different people don't teach the same way even if the curriclumn is the same. Your statements here strengthen the parody. Merge sort is the same for everyone, right? So let's just go by lines of code. The outcome is the same.
• alttag 14y ago
  I had this discussion with a guy who is now the state superintent of public education. Our discussion led to classroom visits of about 10 minutes per instructor, multiple times each semester. Evaluation criteria were: 1) Are learning objectives for the lesson clearly visible or otherwise available? 2) What percent of students were on task? 3) ... I forget the others, but there were five. (It's been about five years, but I think they included the students knowing how they were to be evaluated, and the teaching style used, e.g., lecture, group work, etc., and whether the instructor was using data to inform the approach. It came down to the standard description of leadership: vision, expectations, support, feedback) He was pretty firm that this method would be better—he'd spent years thinking about it—but we both agreed that it required a level of intervention by the administrator that although it could reasonably be expected was unlikely. Using just the test data is the lazy way, which means it's the method most will use. Still, I believe _some_ standardized testing is important. ... but there are two types of tests: norm testing and standards testing, and both types have their uses.