13 ms·

It's fine to grumble about testing, but the debate is not about the tests. Teachers performance in the US has been terrible for many years. Partly this is bec

It's fine to grumble about testing, but the debate is not about the tests. Teachers performance in the US has been terrible for many years. Partly this is because of bad management, partly because of low pay, and partly because of teacher unions preventing any action against the worst teachers and insistance on tenure tracks. The reason testing is good isn't because it is somehow super accurate. It is good because it keeps teachers honest. Without testing, how do you measure teacher performance at all? How can you tell if someone who is capable of teaching well isn't just being lazy or getting distracted? So, if you aren't a fan of testing as a way to measure and improve teacher effectiveness, please find an alternative that works better. Just not having any metric at all is far worse than the imperfect tests we have. It's easy to point out problems. It's useful and important that people find flaws and fault. In this case, however, just pointing out the deficiencies in standardized testing doesn't help anyone unless it either leads to a more effective alternative, or improvements in standardized testing. Standardized tests might be very imperfect at measuring teacher performance, but it's far better to use the tool we have than to just throw up our arms and assume that all teachers are equally competent.

kstenerud 14y ago
"Just not having any metric at all is far worse than the imperfect tests we have." Are you sure about that? How do you know that a bad metric is better than no metric? It sounds a lot more like "We should be doing something. This is something. Therefore we should do it."
- justin_vanw 14y ago
  Sure, I agree that in a general sense just pulling a number out of the air is crap. Testing isn't wrong, it's just not as correlated to the things we would actually like to measure as we would like it to be. This is an open problem in the world, and solving it (besides potentially making the inventor fabulously wealthy) would dramatically change the world. How do you decide if someone is qualified for a job, or will be successful at a university? In reality we don't have a very solid way to figure that out, so lots of people that would have done well are passed over and lots of people get into universities and hired for jobs and end up failing. Testing is correlated to lots of things, and is quite nicely correlated to the quality of the training a student was given. I'm sure there is a better solution out there in the aether, but mankind hasn't discovered it yet.
  - greedo 14y ago
    Actually, we do have a good way to determine how someone will perform. The US military has used the ASVAB test for decades, and studies have shown that performance correlates with test results. http://official-asvab.com/validity_res.htm http://official-asvab.com/validity_res.htm
    - seanmcdirmid 14y ago
      Took the pre-asvab in high school back in the 90s, got a perfect score. The recruiters called a lot. Took the full ASVAB at a recruiting station, still got a very high score. Too easy, how can that be a real test? My ACT/SAT scores were good but not perfect.
- yummyfajitas 14y ago
  This is a question about the philosophy of science. How do you know clinical trials are better than observational methods? How do you know controlled experiments are better than post-hoc reasoning and logical deduction? Standardized testing is just the application of the scientific method to education. If you don't believe in the scientific method then you shouldn't apply it here. But if you are asking for an empirical method to compare the scientific method to "other ways of knowing", there isn't one.
  - deleted 14y ago
    [deleted]
    - deleted 14y ago
      [deleted]
  - vehementi 14y ago
    Standardized testing is an application of the scientific method to education, but it's (accurately) measuring stupid shit that we shouldn't care about in ways that are not relevant.
  - klochner 14y ago
    What about long-run outcomes under standardized testing vs. without standardized testing? Longitudinal data would be difficult to find, but surely countries differ in their respective levels of standardized testing.
  - wpietri 14y ago
    Standardized testing is just the application of the scientific method to education. That's not true in any way. The purpose of the education system is to create a well-informed citizenry prepared for a productive life and for the duty of running a country. The goal of standardized testing is to get some very rough metrics about superficial, easy-to-observe aspects of some basic parts of the apparatus. The two are very tenuously related. If we were applying the scientific method, we wouldn't be doing things like national, uncontrolled experiments that assume that basing major decisions on those rough metrics will result in improvements along the axes we actually care about. And then acting as if those metrics tell us everything about one aspect of the system, teachers, because that's where the political focus is. Applying the scientific method would involve formulating hypotheses and then testing them through lab work and field trials. That's what we do with medicine, at least some of the time. Heck, we could start by testing the hypothesis that getting everybody to focus lots of classroom time on standardized tests improves actual education.
fiatmoney 14y ago
"Teachers performance in the US has been terrible for many years." Terrible at what, exactly?
- justin_vanw 14y ago
  It's all over the map, and there are hardworking teachers. I can tell you about my experience in the 90's. There were exceptions, but generally teachers I had would hand out worksheets and assignments, and then class time would be spent doing what was effectively homework. Very rarely would there be an actual lecture. I had several completely incompetent teachers as well, who were completely tolerated by the schools administration. One teacher, Mrs. Rushing, was completely incompetent. During the lesson on vector addition, she claimed that if you added a bunch of vectors together, it was the same as taking a vector from the beginning of the first vector to the end of the last vector, if you put the source of each vector at the end of the previous one. Well, that's what she was meant to say, she actually insisted that the length of the paths was the same, and she argued for it despite it being obviously wrong. She had at best a weak and often incorrect understanding of the material she was meant to be teaching. She also used her entire planning period each day to sell Avon or Mary Kay or something, which I witnessed every day as I was an aid in the math department and was in the planning room during her planning period. Every day some random lady would come in to the school and chat with her for an hour and try cosmetics during the time she was meant to be preparing for classes. During class she would often get stuck and go ask other teachers questions mid class. So it would have been great if they had tested the outcomes of the students who were in her class, and compared them to the several wonderful teachers teaching the same subject at the same school. Instead, she is still there.
- capnrefsmmat 14y ago
  Standard teaching methods for science and math don't work by many measures. That's not the fault of poor teachers; even well-rated teachers do poorly at improving student understanding. This is largely the fault of our notion that teachers are explanation delivery devices and students explanation receptacles, where the only improvements can be found in better explanations and better student engagement. The notion is wrong. Students need to actively test their understanding by making predictions and failing, and no courses I've ever experienced attempt to do this. If you're interested in the mounds of research: http://www.refsmmat.com/articles/shutup.html http://www.refsmmat.com/articles/shutup.html
lifeisstillgood 14y ago
Testing is only any good if students are assessed at the beginning and end of each year. We care about the difference a teacher makes in the lives and skills of students. Those that make the most difference get the most reward. But hey - that might be ridiculously easy to game - every which way. (Personally I would get my whole class pissed out of their brains on the first test, and teach normally for the rest of the year. With any luck I will see an improvement) And finally - testing is supposed to be about testing the students. Not the teachers. Every Head knows which ones are good and which ones are crap.
- acdha 14y ago
  Even this isn't enough: teachers suffer from small sample sizes and tests which attempt to reduce very complex realities to simple numbers. Imagine teaching 70 students, some of whom have significant problems outside your control (e.g. not showing up for class) - if that number changes by just a couple of students either way, your evaluation can go up or down completely due to random variation. At this point, I'd also note that the people running the numbers tend to have very limited math skills – a great way to recoup some of the money we're pouring into standardized testing would be for each district to hire expert statisticians, scientists, etc. so the comparisons could be done by people who understand how to avoid measurement and analysis errors.
jbooth 14y ago
Software developer performance in the US has been terrible for many years as well. Maybe we should institute some standardized tests? And if you're gonna complain about how measuring KLOC doesn't make any sense, well, it's not enough to just point out problems. You've gotta point out solutions. Be a team player.
- justin_vanw 14y ago
  You're right, it's exactly the same in every way. No strawman here!
  - Tsagadai 14y ago
    It's not the same and that wasn't the poster's point and it isn't a strawman. Metrics are hard. Metrics for complex, creative endeavours are exceptionally hard. Both programmers and teachers are difficult to accurately evaluate and manage. Most developers I know object to arbitrary metrics for the same reasons as most teachers I know do, the metrics rarely measure actual performance and metrics are usually gamed. Perhaps you would like to contribute something to the discussion instead of pointless sarcasm.
- yummyfajitas 14y ago
  Software developer performance in the US has been terrible for many years as well. The cost of software has been going up while the output has not been improving? Um, what? Maybe we should institute some standardized tests? Big companies already do this. They typically have very rigid processes for evaluating consultants/outsourcers and pretty much just follow the numbers. For higher end work it's trickier, but for standard stuff (build X screens in Y CRUD app) it works well. The only real difficulty in measuring the output of individual developers is heterogeneity. Last year I built a search engine, this year I'm building a realtime data framework (and other stuff). That's not a problem for teaching, since the output of teaching is homogeneous (and is defined by the syllabus).
  - drivingmenuts 14y ago
    > That's not a problem for teaching, since the output of teaching is homogeneous (and is defined by the syllabus). The syllabus is typically defined politically, which is anything but homegeneous. In fact, pretty much everything surrounding the teacher is defined by politics, which makes it a huge problem. Until we're mature enough to get rid of the politics surrounding the teaching of even core subjects, like science, we're screwed. And I don't blame the teachers one bit for walking away.
primatology 14y ago
There is a better alternative: surveying students. Standardized tests attempt to assign one number to teacher performance by quantifying student performance on one three-hour test. The survey examines dozens of metrics while allowing students to capture their over 100 hours of experience in the classroom watching their teacher. We're discarding some of our most valuable data. In most schools, student evaluations of teachers aren't even administered, let alone analyzed and weighted in teacher assessment. Skeptical? See the research below. Properly-constructed surveys yield very accurate results; students are surprisingly honest in their responses, and students truly value a hard, fair teacher who actually teaches his students over an "easy A" teacher. Take a look at Ronald Ferguson's work and the MET Project [1]. I can't find the original article, but this New York Times article [2] is a decent summary. [1] http://www.metproject.org/downloads/met-framing-paper.pdf http://www.metproject.org/downloads/met-framing-paper.pdf [2] http://www.nytimes.com/2010/12/11/education/11education.html http://www.nytimes.com/2010/12/11/education/11education.html
- bloaf 14y ago
  Student surveys could certainly contribute to the evaluation of teachers, but I doubt its a viable replacement for academic tests. We must also note that they are being compared to current test-based methods. They claim to get "good agreement" but without a real publication its hard to tell what that means. Does it mean that in most cases the students are honest? (i.e. that the disagreement is caused by dishonesty) Does it mean that in most cases students appreciate teachers that make them learn? (i.e. that the disagreement is caused by students who like their teachers even though the students themselves are not learning) Or is the disagreement inherent in the fact that one measure is better than the other at gauging teacher performance?
Bud 14y ago
So, if you aren't a fan of testing as a way to measure and improve teacher effectiveness, please find an alternative that works better. Just not having any metric at all is far worse than the imperfect tests we have. This is a gigantic assumption which you have tossed out there without any support whatsoever. You are basically making an assumption which encompasses the entire issue in question, and then just blithely assuming you are correct. In fact, the evidence which is staring us in the face indicates that you are not correct, and that we were far better off before we decided to let No Child Left Behind and a group of for-profit "education" companies co-opt our system and turn it into a giant standardized-test machine. You want an alternative that works better? No problem. Go back to doing things the way we did them before January 2001. By the way, this would emphatically NOT mean that we had "no metric at all", of course...we had testing before 2001. We just didn't structure our entire system around constant standardized testing and force teachers to teach to those tests, that's all. We also had other metrics, such as observing how students do in college.
- yummyfajitas 14y ago
  What evidence do you have that things were better before 2001?
  - seanmcdirmid 14y ago
    For anyone that knows anything about the current system, its almost obvious. Education in the states is much more of a wreck now than it was BB (Before Bush). The only thing kids have improved at now is....taking standardized tests.
    - yummyfajitas 14y ago
      tl;dr No evidence but I hate Bush SOOOO MUCH!
      - seanmcdirmid 14y ago
        If you have kids, do you really want them to be educated in fill-in-the-bubble hell? Because that is how school is these days, teachers don't teach, they do test prep and then administer those tests, and the testing companies have gotten rich in basically a corrupt cycle. Read the linked article for crying out loud. Also, NCLB passed with bipartisan support at the beginning of Bush's term, it wasn't his fault, but it was symptomatic of the simplistic shallow thinking that he would use the rest of his term. States are finally able to opt out, thank god, and you can bet I will make sure my kids are educated in one of those states.
curt 14y ago
"partly because of low pay" Teachers are actually some of the highest paid people in society. When you don't include benefits teachers make about $40/hr while engineers make $42/hr. If you do include benefits, a teachers compensation far exceeds that of an engineer due to their pensions. Source, BLS via Forbes.
- sirmarksalot 14y ago
  I looked that up on the BLS website, and while you're not using the same units as them, your numbers don't seem to match up. According to BLS, the median pay is $53,230 per year, which would be closer to $26.50 an hour, assuming a 40-hour work week. http://www.bls.gov/ooh/education-training-and-library/high-school-teachers.htm http://www.bls.gov/ooh/education-training-and-library/high-s... Maybe the Forbes article you're referring to about is only counting classroom hours, which is a pretty absurd notion. Teaching is a full-time job, and any figure based on an assumption to the contrary is going to be flawed. Not meaning to straw-man you, it's just that without a link to the article in question, I can only speculate where your figures come from.
  - yummyfajitas 14y ago
    The correct figure is 38.5 hours/week for 9 months/year. http://www.bls.gov/opub/mlr/2008/03/art4full.pdf http://www.bls.gov/opub/mlr/2008/03/art4full.pdf That's about $33.5/hour for wages alone. It also ignores the value of gold plated health care + pensions, etc.
    - Bud 14y ago
      "Gold plated" (sic) health care? Tuck in your pants. Your bias is peeking out. (Edit after being downvoted, in an attempt to be more constructive): Seriously, you seem to have an animus against teachers having decent health care. What's your problem with that, and why do you feel it's appropriate to label their quite average health care plans as "gold plated", as if they were unfairly receiving more health care than they are entitled to?
lukev 14y ago
The contention here, though, isn't that standardized tests are useless, it's that they're worse than useless and actively cause harm when they're required. It's a very different proposition. As for measuring performance, there are ton of ways to do it (some of which incorporate tests) without mandating standardized testing.
- bloaf 14y ago
  What specifically is the harm they are alleged to cause? Teaching to the test? I wonder what teachers who currently "teach to the test" would teach if they had no test to teach to. I seriously doubt that they would suddenly become paragons of pedagogy, that it was somehow the existence of standardized tests was the cause of their deficient teaching. Of course it could be argued that tests offer a shelter to such teachers; in standardized test's absence, they would be replaced by better teachers. I also think this is an unlikely scenario. The absence of standardization would result in non-uniform results, some districts might come up with good measures of teacher performance and successfully replace teachers, but I suspect most would not. Instead, the formerly "teach to the test" teachers would stay on, and that their teaching would not improve as a result of their new-found lack of accountability. No doubt some replacement for standardized testing could be invented, but that is irrelevant if we are arguing that such tests are worse than useless. I.e. that doing nothing is better than doing these tests.
danielweber 14y ago
Besides sales, every attempt to manage professionals by an objective formula ends up with horrible results. The technocrats will always insist we just need a better formula.
halffullheart 14y ago
Obviously this is a complex issue, and I don't mean to single you out here, but I'm surprised how often I hear the argument that teachers aren't good enough from people who are concerned about the quality of education in the US. It seems to me that in most other industries, the workers in the field are given the benefit of the doubt. In other words, do people really believe that Americans are great at being programmers or fire fighters or lawyers but that we just suck at being teachers? Maybe I'm biased because there are a lot of teachers in my family, but my impression from hearing them discuss it is that the problem is much larger than whether teachers are good enough or not. Many teachers, like the one who wrote the linked article, are working under incredible pressure, essentially in conditions under which is is very easy to fail. Huge class sizes, inadequate funding, etc. I think we should stop blaming teachers and assume like we do with so many other workers that most of them are smart and capable and will do well if we give them the support they need. And it would probably be wise to ask teachers how we can improve education and give them the authority and resources to implement their ideas.
- waterlesscloud 14y ago
  Most of us have been to public schools. We have direct, first-hand observations about the quality of teachers. I had a few stars, most were just getting by, some actively sucked. From everything I've ever heard, most people had the same experience. It's not like we're talking about something none of us know anything about. We've been there, seen it.
  - king_jester 14y ago
    The first hand experience of people having been a student once isn't enough information in which to judge a whole profession. If that were the case, we should be letting only users of software direct how software is made. Also, experiences can be really variable for different people. Some students might have many great teachers and be in a position where school is very positive for them. Other students could be facing an entirely different picture about the kind of schools and teachers available to them, esp. given the variances in local and state politics, class levels, income, and attitudes towards teaching.
wpietri 14y ago
This seems roughly equivalent to the argument that we should measure software developer quality by lines of code written per day. Sure, we know it's not a perfect measurement, but it's the easy one to check, and it keeps people from doing nothing. Of course, we software developers, understanding the complexity of our profession, know that would be idiocy. "But hey," says the MBA, "Some metric is better than no metric."
- Locke1689 14y ago
  Then you create a new metric, you don't just throw your hands up and say, "well, I guess this stuff just can't be measured!"
  - saraid216 14y ago
    Believe it or not, this happens on a regular basis. You never hear about it, though, because standardized testing is our current gold standard and people refuse to use something else.
  - seanmcdirmid 14y ago
    In science, no metric is better than a bad misleading metric, definitely.
    - Locke1689 14y ago
      It's not science if there's no metric. If your hypothesis can't be falsified there's no scientific method. I'm not sure what you're advocating, but it seems to be wish-based education.
      - seanmcdirmid 14y ago
        http://www.washingtonpost.com/blogs/answer-sheet/post/why-faith-in-standardized-testing-industry-is-misplaced/2011/04/10/AFXIkSHD_blog.html http://www.washingtonpost.com/blogs/answer-sheet/post/why-fa...
      - 14y ago
saintgimp 14y ago
The problem of standardized testing has a direct analogue in software engineering that makes the situation crystal clear to any programmer - remember when our industry thought it was a good idea to measure programmer productivity primarily (or even solely) by the number of lines of code written? How did that work out for us? Terribly, of course. LOC metrics have a passing resemblance to the thing you want to measure in software engineering but it turned out to be horribly bad at predicting the thing we actually cared about - working software. Standardized testing bears exactly the same relationship to the outcome we actually care about in education. In both realms, there is no easy single metric you can turn to. In the end, you hire great people and get out of their way. The best measurements come from multi-dimensional feedback from managers, peers, and customers. No it doesn't scale very well but it's the only thing that works.
jseliger 14y ago
>The reason testing is good isn't because it is somehow super accurate. It is good because it keeps teachers honest. Without testing, how do you measure teacher performance at all? How can you tell if someone who is capable of teaching well isn't just being lazy or getting distracted? The other issue is that value-added testing should be used in evaluating teachers, but only a single component of a larger evaluation process. On its own, it's almost as pernicious as the system we have now. For more on these subjects, see * "What Makes a Great Teacher?": http://www.theatlantic.com/magazine/archive/2010/01/what-makes-a-great-teacher/307841/?single_page=true http://www.theatlantic.com/magazine/archive/2010/01/what-mak... * "Why Kids Should Grade Teachers": http://www.theatlantic.com/magazine/archive/2012/10/why-kids-should-grade-teachers/309088/ http://www.theatlantic.com/magazine/archive/2012/10/why-kids... * The collection of articles I listed here: http://jseliger.wordpress.com/2009/11/12/susan-engel-doesnt-get-2/ http://jseliger.wordpress.com/2009/11/12/susan-engel-doesnt-... . >It's easy to point out problems. It's useful and important that people find flaws and fault. The largest issue isn't identifying problems—it's that the people who benefit most from the status quo (teachers, unions) derive very concentrated benefits, while the solutions to the status quo are diffuse (students, parents, society). One sees this pattern in many fields (farm subsidies are the canonical example), and there's an extensive economics literature on it.
daemin 14y ago
I think part of the reason that a lot of student evaluation is now in the form of tests (multiple choice I assume) is because they are relatively easy and quick to mark. Evaluating a piece of design or art work, or asking to write creative prose, or showing all the working out for a lengthy mathematical problem requires significantly more time and energy than feeding a test through a computer.
saraid216 14y ago
Yeah, for me, standardized testing has been a canary in the coal mine more than it has been a legitimate point of contention. The fact that it's such a big deal is a signal that the need for a paradigm shift is strong.
me_again 14y ago
A bad metric can indeed be worse than nothing, because people optimize for the metric, and if it's a poor proxy for the real thing you want, you can actually get less of it. Imagine what would happen if you evaluated software engineers by lines of code written, and you get some idea of the potential downside. Other potential ways to assess teacher effectiveness: - auditing classes & lesson plans - feedback from peers, parents, students and principals

kstenerud 14y ago

"Just not having any metric at all is far worse than the imperfect tests we have." Are you sure about that? How do you know that a bad metric is better than no metric? It sounds a lot more like "We should be doing something. This is something. Therefore we should do it."

justin_vanw 14y ago
Sure, I agree that in a general sense just pulling a number out of the air is crap. Testing isn't wrong, it's just not as correlated to the things we would actually like to measure as we would like it to be. This is an open problem in the world, and solving it (besides potentially making the inventor fabulously wealthy) would dramatically change the world. How do you decide if someone is qualified for a job, or will be successful at a university? In reality we don't have a very solid way to figure that out, so lots of people that would have done well are passed over and lots of people get into universities and hired for jobs and end up failing. Testing is correlated to lots of things, and is quite nicely correlated to the quality of the training a student was given. I'm sure there is a better solution out there in the aether, but mankind hasn't discovered it yet.
- greedo 14y ago
  Actually, we do have a good way to determine how someone will perform. The US military has used the ASVAB test for decades, and studies have shown that performance correlates with test results. http://official-asvab.com/validity_res.htm http://official-asvab.com/validity_res.htm
  - seanmcdirmid 14y ago
    Took the pre-asvab in high school back in the 90s, got a perfect score. The recruiters called a lot. Took the full ASVAB at a recruiting station, still got a very high score. Too easy, how can that be a real test? My ACT/SAT scores were good but not perfect.
yummyfajitas 14y ago
This is a question about the philosophy of science. How do you know clinical trials are better than observational methods? How do you know controlled experiments are better than post-hoc reasoning and logical deduction? Standardized testing is just the application of the scientific method to education. If you don't believe in the scientific method then you shouldn't apply it here. But if you are asking for an empirical method to compare the scientific method to "other ways of knowing", there isn't one.
- deleted 14y ago
  [deleted]
  - deleted 14y ago
    [deleted]
- vehementi 14y ago
  Standardized testing is an application of the scientific method to education, but it's (accurately) measuring stupid shit that we shouldn't care about in ways that are not relevant.
- klochner 14y ago
  What about long-run outcomes under standardized testing vs. without standardized testing? Longitudinal data would be difficult to find, but surely countries differ in their respective levels of standardized testing.
- wpietri 14y ago
  Standardized testing is just the application of the scientific method to education. That's not true in any way. The purpose of the education system is to create a well-informed citizenry prepared for a productive life and for the duty of running a country. The goal of standardized testing is to get some very rough metrics about superficial, easy-to-observe aspects of some basic parts of the apparatus. The two are very tenuously related. If we were applying the scientific method, we wouldn't be doing things like national, uncontrolled experiments that assume that basing major decisions on those rough metrics will result in improvements along the axes we actually care about. And then acting as if those metrics tell us everything about one aspect of the system, teachers, because that's where the political focus is. Applying the scientific method would involve formulating hypotheses and then testing them through lab work and field trials. That's what we do with medicine, at least some of the time. Heck, we could start by testing the hypothesis that getting everybody to focus lots of classroom time on standardized tests improves actual education.

fiatmoney 14y ago

"Teachers performance in the US has been terrible for many years." Terrible at what, exactly?

justin_vanw 14y ago
It's all over the map, and there are hardworking teachers. I can tell you about my experience in the 90's. There were exceptions, but generally teachers I had would hand out worksheets and assignments, and then class time would be spent doing what was effectively homework. Very rarely would there be an actual lecture. I had several completely incompetent teachers as well, who were completely tolerated by the schools administration. One teacher, Mrs. Rushing, was completely incompetent. During the lesson on vector addition, she claimed that if you added a bunch of vectors together, it was the same as taking a vector from the beginning of the first vector to the end of the last vector, if you put the source of each vector at the end of the previous one. Well, that's what she was meant to say, she actually insisted that the length of the paths was the same, and she argued for it despite it being obviously wrong. She had at best a weak and often incorrect understanding of the material she was meant to be teaching. She also used her entire planning period each day to sell Avon or Mary Kay or something, which I witnessed every day as I was an aid in the math department and was in the planning room during her planning period. Every day some random lady would come in to the school and chat with her for an hour and try cosmetics during the time she was meant to be preparing for classes. During class she would often get stuck and go ask other teachers questions mid class. So it would have been great if they had tested the outcomes of the students who were in her class, and compared them to the several wonderful teachers teaching the same subject at the same school. Instead, she is still there.
capnrefsmmat 14y ago
Standard teaching methods for science and math don't work by many measures. That's not the fault of poor teachers; even well-rated teachers do poorly at improving student understanding. This is largely the fault of our notion that teachers are explanation delivery devices and students explanation receptacles, where the only improvements can be found in better explanations and better student engagement. The notion is wrong. Students need to actively test their understanding by making predictions and failing, and no courses I've ever experienced attempt to do this. If you're interested in the mounds of research: http://www.refsmmat.com/articles/shutup.html http://www.refsmmat.com/articles/shutup.html

lifeisstillgood 14y ago

Testing is only any good if students are assessed at the beginning and end of each year. We care about the difference a teacher makes in the lives and skills of students. Those that make the most difference get the most reward. But hey - that might be ridiculously easy to game - every which way. (Personally I would get my whole class pissed out of their brains on the first test, and teach normally for the rest of the year. With any luck I will see an improvement) And finally - testing is supposed to be about testing the students. Not the teachers. Every Head knows which ones are good and which ones are crap.

acdha 14y ago
Even this isn't enough: teachers suffer from small sample sizes and tests which attempt to reduce very complex realities to simple numbers. Imagine teaching 70 students, some of whom have significant problems outside your control (e.g. not showing up for class) - if that number changes by just a couple of students either way, your evaluation can go up or down completely due to random variation. At this point, I'd also note that the people running the numbers tend to have very limited math skills – a great way to recoup some of the money we're pouring into standardized testing would be for each district to hire expert statisticians, scientists, etc. so the comparisons could be done by people who understand how to avoid measurement and analysis errors.

jbooth 14y ago

Software developer performance in the US has been terrible for many years as well. Maybe we should institute some standardized tests? And if you're gonna complain about how measuring KLOC doesn't make any sense, well, it's not enough to just point out problems. You've gotta point out solutions. Be a team player.

justin_vanw 14y ago
You're right, it's exactly the same in every way. No strawman here!
- Tsagadai 14y ago
  It's not the same and that wasn't the poster's point and it isn't a strawman. Metrics are hard. Metrics for complex, creative endeavours are exceptionally hard. Both programmers and teachers are difficult to accurately evaluate and manage. Most developers I know object to arbitrary metrics for the same reasons as most teachers I know do, the metrics rarely measure actual performance and metrics are usually gamed. Perhaps you would like to contribute something to the discussion instead of pointless sarcasm.
yummyfajitas 14y ago
Software developer performance in the US has been terrible for many years as well. The cost of software has been going up while the output has not been improving? Um, what? Maybe we should institute some standardized tests? Big companies already do this. They typically have very rigid processes for evaluating consultants/outsourcers and pretty much just follow the numbers. For higher end work it's trickier, but for standard stuff (build X screens in Y CRUD app) it works well. The only real difficulty in measuring the output of individual developers is heterogeneity. Last year I built a search engine, this year I'm building a realtime data framework (and other stuff). That's not a problem for teaching, since the output of teaching is homogeneous (and is defined by the syllabus).
- drivingmenuts 14y ago
  > That's not a problem for teaching, since the output of teaching is homogeneous (and is defined by the syllabus). The syllabus is typically defined politically, which is anything but homegeneous. In fact, pretty much everything surrounding the teacher is defined by politics, which makes it a huge problem. Until we're mature enough to get rid of the politics surrounding the teaching of even core subjects, like science, we're screwed. And I don't blame the teachers one bit for walking away.

primatology 14y ago

There is a better alternative: surveying students. Standardized tests attempt to assign one number to teacher performance by quantifying student performance on one three-hour test. The survey examines dozens of metrics while allowing students to capture their over 100 hours of experience in the classroom watching their teacher. We're discarding some of our most valuable data. In most schools, student evaluations of teachers aren't even administered, let alone analyzed and weighted in teacher assessment. Skeptical? See the research below. Properly-constructed surveys yield very accurate results; students are surprisingly honest in their responses, and students truly value a hard, fair teacher who actually teaches his students over an "easy A" teacher. Take a look at Ronald Ferguson's work and the MET Project [1]. I can't find the original article, but this New York Times article [2] is a decent summary. [1] http://www.metproject.org/downloads/met-framing-paper.pdf http://www.metproject.org/downloads/met-framing-paper.pdf [2] http://www.nytimes.com/2010/12/11/education/11education.html http://www.nytimes.com/2010/12/11/education/11education.html

bloaf 14y ago
Student surveys could certainly contribute to the evaluation of teachers, but I doubt its a viable replacement for academic tests. We must also note that they are being compared to current test-based methods. They claim to get "good agreement" but without a real publication its hard to tell what that means. Does it mean that in most cases the students are honest? (i.e. that the disagreement is caused by dishonesty) Does it mean that in most cases students appreciate teachers that make them learn? (i.e. that the disagreement is caused by students who like their teachers even though the students themselves are not learning) Or is the disagreement inherent in the fact that one measure is better than the other at gauging teacher performance?

Bud 14y ago

So, if you aren't a fan of testing as a way to measure and improve teacher effectiveness, please find an alternative that works better. Just not having any metric at all is far worse than the imperfect tests we have. This is a gigantic assumption which you have tossed out there without any support whatsoever. You are basically making an assumption which encompasses the entire issue in question, and then just blithely assuming you are correct. In fact, the evidence which is staring us in the face indicates that you are not correct, and that we were far better off before we decided to let No Child Left Behind and a group of for-profit "education" companies co-opt our system and turn it into a giant standardized-test machine. You want an alternative that works better? No problem. Go back to doing things the way we did them before January 2001. By the way, this would emphatically NOT mean that we had "no metric at all", of course...we had testing before 2001. We just didn't structure our entire system around constant standardized testing and force teachers to teach to those tests, that's all. We also had other metrics, such as observing how students do in college.

yummyfajitas 14y ago
What evidence do you have that things were better before 2001?
- seanmcdirmid 14y ago
  For anyone that knows anything about the current system, its almost obvious. Education in the states is much more of a wreck now than it was BB (Before Bush). The only thing kids have improved at now is....taking standardized tests.
  - yummyfajitas 14y ago
    tl;dr No evidence but I hate Bush SOOOO MUCH!
    - seanmcdirmid 14y ago
      If you have kids, do you really want them to be educated in fill-in-the-bubble hell? Because that is how school is these days, teachers don't teach, they do test prep and then administer those tests, and the testing companies have gotten rich in basically a corrupt cycle. Read the linked article for crying out loud. Also, NCLB passed with bipartisan support at the beginning of Bush's term, it wasn't his fault, but it was symptomatic of the simplistic shallow thinking that he would use the rest of his term. States are finally able to opt out, thank god, and you can bet I will make sure my kids are educated in one of those states.

curt 14y ago

"partly because of low pay" Teachers are actually some of the highest paid people in society. When you don't include benefits teachers make about $40/hr while engineers make $42/hr. If you do include benefits, a teachers compensation far exceeds that of an engineer due to their pensions. Source, BLS via Forbes.

sirmarksalot 14y ago
I looked that up on the BLS website, and while you're not using the same units as them, your numbers don't seem to match up. According to BLS, the median pay is $53,230 per year, which would be closer to $26.50 an hour, assuming a 40-hour work week. http://www.bls.gov/ooh/education-training-and-library/high-school-teachers.htm http://www.bls.gov/ooh/education-training-and-library/high-s... Maybe the Forbes article you're referring to about is only counting classroom hours, which is a pretty absurd notion. Teaching is a full-time job, and any figure based on an assumption to the contrary is going to be flawed. Not meaning to straw-man you, it's just that without a link to the article in question, I can only speculate where your figures come from.
- yummyfajitas 14y ago
  The correct figure is 38.5 hours/week for 9 months/year. http://www.bls.gov/opub/mlr/2008/03/art4full.pdf http://www.bls.gov/opub/mlr/2008/03/art4full.pdf That's about $33.5/hour for wages alone. It also ignores the value of gold plated health care + pensions, etc.
  - Bud 14y ago
    "Gold plated" (sic) health care? Tuck in your pants. Your bias is peeking out. (Edit after being downvoted, in an attempt to be more constructive): Seriously, you seem to have an animus against teachers having decent health care. What's your problem with that, and why do you feel it's appropriate to label their quite average health care plans as "gold plated", as if they were unfairly receiving more health care than they are entitled to?

lukev 14y ago

The contention here, though, isn't that standardized tests are useless, it's that they're worse than useless and actively cause harm when they're required. It's a very different proposition. As for measuring performance, there are ton of ways to do it (some of which incorporate tests) without mandating standardized testing.

bloaf 14y ago
What specifically is the harm they are alleged to cause? Teaching to the test? I wonder what teachers who currently "teach to the test" would teach if they had no test to teach to. I seriously doubt that they would suddenly become paragons of pedagogy, that it was somehow the existence of standardized tests was the cause of their deficient teaching. Of course it could be argued that tests offer a shelter to such teachers; in standardized test's absence, they would be replaced by better teachers. I also think this is an unlikely scenario. The absence of standardization would result in non-uniform results, some districts might come up with good measures of teacher performance and successfully replace teachers, but I suspect most would not. Instead, the formerly "teach to the test" teachers would stay on, and that their teaching would not improve as a result of their new-found lack of accountability. No doubt some replacement for standardized testing could be invented, but that is irrelevant if we are arguing that such tests are worse than useless. I.e. that doing nothing is better than doing these tests.

danielweber 14y ago

Besides sales, every attempt to manage professionals by an objective formula ends up with horrible results. The technocrats will always insist we just need a better formula.

halffullheart 14y ago

Obviously this is a complex issue, and I don't mean to single you out here, but I'm surprised how often I hear the argument that teachers aren't good enough from people who are concerned about the quality of education in the US. It seems to me that in most other industries, the workers in the field are given the benefit of the doubt. In other words, do people really believe that Americans are great at being programmers or fire fighters or lawyers but that we just suck at being teachers? Maybe I'm biased because there are a lot of teachers in my family, but my impression from hearing them discuss it is that the problem is much larger than whether teachers are good enough or not. Many teachers, like the one who wrote the linked article, are working under incredible pressure, essentially in conditions under which is is very easy to fail. Huge class sizes, inadequate funding, etc. I think we should stop blaming teachers and assume like we do with so many other workers that most of them are smart and capable and will do well if we give them the support they need. And it would probably be wise to ask teachers how we can improve education and give them the authority and resources to implement their ideas.

waterlesscloud 14y ago
Most of us have been to public schools. We have direct, first-hand observations about the quality of teachers. I had a few stars, most were just getting by, some actively sucked. From everything I've ever heard, most people had the same experience. It's not like we're talking about something none of us know anything about. We've been there, seen it.
- king_jester 14y ago
  The first hand experience of people having been a student once isn't enough information in which to judge a whole profession. If that were the case, we should be letting only users of software direct how software is made. Also, experiences can be really variable for different people. Some students might have many great teachers and be in a position where school is very positive for them. Other students could be facing an entirely different picture about the kind of schools and teachers available to them, esp. given the variances in local and state politics, class levels, income, and attitudes towards teaching.

wpietri 14y ago

This seems roughly equivalent to the argument that we should measure software developer quality by lines of code written per day. Sure, we know it's not a perfect measurement, but it's the easy one to check, and it keeps people from doing nothing. Of course, we software developers, understanding the complexity of our profession, know that would be idiocy. "But hey," says the MBA, "Some metric is better than no metric."

Locke1689 14y ago
Then you create a new metric, you don't just throw your hands up and say, "well, I guess this stuff just can't be measured!"
- saraid216 14y ago
  Believe it or not, this happens on a regular basis. You never hear about it, though, because standardized testing is our current gold standard and people refuse to use something else.
- seanmcdirmid 14y ago
  In science, no metric is better than a bad misleading metric, definitely.
  - Locke1689 14y ago
    It's not science if there's no metric. If your hypothesis can't be falsified there's no scientific method. I'm not sure what you're advocating, but it seems to be wish-based education.
    - seanmcdirmid 14y ago
      http://www.washingtonpost.com/blogs/answer-sheet/post/why-faith-in-standardized-testing-industry-is-misplaced/2011/04/10/AFXIkSHD_blog.html http://www.washingtonpost.com/blogs/answer-sheet/post/why-fa...
    - 14y ago

saintgimp 14y ago

The problem of standardized testing has a direct analogue in software engineering that makes the situation crystal clear to any programmer - remember when our industry thought it was a good idea to measure programmer productivity primarily (or even solely) by the number of lines of code written? How did that work out for us? Terribly, of course. LOC metrics have a passing resemblance to the thing you want to measure in software engineering but it turned out to be horribly bad at predicting the thing we actually cared about - working software. Standardized testing bears exactly the same relationship to the outcome we actually care about in education. In both realms, there is no easy single metric you can turn to. In the end, you hire great people and get out of their way. The best measurements come from multi-dimensional feedback from managers, peers, and customers. No it doesn't scale very well but it's the only thing that works.

jseliger 14y ago

>The reason testing is good isn't because it is somehow super accurate. It is good because it keeps teachers honest. Without testing, how do you measure teacher performance at all? How can you tell if someone who is capable of teaching well isn't just being lazy or getting distracted? The other issue is that value-added testing should be used in evaluating teachers, but only a single component of a larger evaluation process. On its own, it's almost as pernicious as the system we have now. For more on these subjects, see * "What Makes a Great Teacher?": http://www.theatlantic.com/magazine/archive/2010/01/what-makes-a-great-teacher/307841/?single_page=true http://www.theatlantic.com/magazine/archive/2010/01/what-mak... * "Why Kids Should Grade Teachers": http://www.theatlantic.com/magazine/archive/2012/10/why-kids-should-grade-teachers/309088/ http://www.theatlantic.com/magazine/archive/2012/10/why-kids... * The collection of articles I listed here: http://jseliger.wordpress.com/2009/11/12/susan-engel-doesnt-get-2/ http://jseliger.wordpress.com/2009/11/12/susan-engel-doesnt-... . >It's easy to point out problems. It's useful and important that people find flaws and fault. The largest issue isn't identifying problems—it's that the people who benefit most from the status quo (teachers, unions) derive very concentrated benefits, while the solutions to the status quo are diffuse (students, parents, society). One sees this pattern in many fields (farm subsidies are the canonical example), and there's an extensive economics literature on it.

daemin 14y ago

I think part of the reason that a lot of student evaluation is now in the form of tests (multiple choice I assume) is because they are relatively easy and quick to mark. Evaluating a piece of design or art work, or asking to write creative prose, or showing all the working out for a lengthy mathematical problem requires significantly more time and energy than feeding a test through a computer.

saraid216 14y ago

Yeah, for me, standardized testing has been a canary in the coal mine more than it has been a legitimate point of contention. The fact that it's such a big deal is a signal that the need for a paradigm shift is strong.

me_again 14y ago

A bad metric can indeed be worse than nothing, because people optimize for the metric, and if it's a poor proxy for the real thing you want, you can actually get less of it. Imagine what would happen if you evaluated software engineers by lines of code written, and you get some idea of the potential downside. Other potential ways to assess teacher effectiveness: - auditing classes & lesson plans - feedback from peers, parents, students and principals