3 ms·

Your links do not remotely come close to debunking the concept of VAM. Your first (accurately) says "measurements are noisy" and the inaccurately says "and oth

Your links do not remotely come close to debunking the concept of VAM. Your first (accurately) says "measurements are noisy" and the inaccurately says "and other things matter for learning too". Unless I missed something, it doesn't provide a reason to expect bias in the other things. The second link says that some VAM models assume student assignment is random when it is not, and also that measured educational outputs don't matter much for outcomes anyway. The logical conclusion of this is simply that VAM needs modification, and maybe we should reduce education. Claiming this debunks the idea of VAM is like saying "hah, this paper proves OLS doesn't always work well, therefore regression is debunked". The third is really long and I couldn't figure out it's point. The fourth one says VAM is noisy. If you believe VAM is unable to measure the effects of teachers, are you willing to follow through with the logical conclusions? Namely, the effect teachers have on students is so small it can't be measured and we should therefore focus on other things (e.g. reducing cost) instead?

Alex3917 12y ago
> Unless I missed something, it doesn't provide a reason to expect bias in the other things. "VAM’s instability can result from differences in the characteristics of students assigned to particular teachers in a particular year, from small samples of students (made even less representative in schools serving disadvantaged students by high rates of student mobility), from other influences on student learning both inside and outside school, and from tests that are poorly lined up with the curriculum teachers are expected to cover, or that do not measure the full range of achievement of students in the class." > If you believe VAM is unable to measure the effects of teachers, are you willing to follow through with the logical conclusions? Namely, the effect teachers have on students is so small it can't be measured ... We already knew that the contributions of teachers to student achievement was relatively small long before VAM was invented. The only people who seem to think that teachers have a very large impact are the pro-VAM folks (e.g. Bill Gates and Malcolm Gladwell), who all cite the same flawed study. That said, the reasons why VAM can't measure teacher performance isn't because the contributions of teachers are small per se, but rather because there are all sorts of problems with all of the proposed measurement systems. In order to create a VAM system that actually worked, assuming such a thing is possible, you'd have to make a number of changes to the way that the school system is designed... e.g. assigning students to teachers at random, designing standardized tests to measure teacher quality, implementing standardized tests in all subject areas, etc. But implementing all of those changes may well be worse for student learning than firing all the bad teachers or whatever. Not to mention that there is currently a major crisis in teacher retention, with half of all teachers either getting fired or quitting in their first five years. (After spending two years and god knows how much money getting their masters degrees to go into the profession.)
- yummyfajitas 12y ago
  VAM’s instability Your quote concerns variance, not bias. (The main exception is the bit about "full range of achievement". It's quite true that teachers of upper class Asian students will likely have lower VAM scores than they deserve since the test caps out at 100%.) We already knew that the contributions of teachers to student achievement was relatively small long before VAM was invented. You dodged the question. Do you think policymakers and politicians should stop trying to improve teachers, and instead simply try to reduce costs? I.e., pay teachers less, increase their turnover (so that we don't have to pay expensive but ineffective senior ones), etc?
  - Alex3917 12y ago
    > Your quote concerns variance, not bias. What were you originally referencing in terms of bias? > Do you think policymakers and politicians should stop trying to improve teachers, and instead simply try to reduce costs? I.e., pay teachers less, increase their turnover (so that we don't have to pay expensive but ineffective senior ones), etc? I mean the answer to any of those types of questions really depend on what kind of school system you want to create as a whole, you could go either way on any of them and have either a good or a bad school system depending on the overall design choices. But my point was that I don't think that VAM has much to say about any of these questions, because whether or not it's theoretically possible to measure teacher performance doesn't say much about whether or not good and bad teachers actually exist and whether or not they have an impact on students learning/lives. My point was that the reason we can't measure teacher performance isn't because the difference between good and bad teachers is small, even though it is relatively small in the grand scheme of things.
    - yummyfajitas 12y ago
      http://en.wikipedia.org/wiki/Bias_(statistics) http://en.wikipedia.org/wiki/Bias_(statistics) http://en.wikipedia.org/wiki/Variance http://en.wikipedia.org/wiki/Variance Alex, I'd suggest you refrain from opining that standard statistical methods fail in education until after you master basic statistics.
    - 12y ago

Alex3917 12y ago

> Unless I missed something, it doesn't provide a reason to expect bias in the other things. "VAM’s instability can result from differences in the characteristics of students assigned to particular teachers in a particular year, from small samples of students (made even less representative in schools serving disadvantaged students by high rates of student mobility), from other influences on student learning both inside and outside school, and from tests that are poorly lined up with the curriculum teachers are expected to cover, or that do not measure the full range of achievement of students in the class." > If you believe VAM is unable to measure the effects of teachers, are you willing to follow through with the logical conclusions? Namely, the effect teachers have on students is so small it can't be measured ... We already knew that the contributions of teachers to student achievement was relatively small long before VAM was invented. The only people who seem to think that teachers have a very large impact are the pro-VAM folks (e.g. Bill Gates and Malcolm Gladwell), who all cite the same flawed study. That said, the reasons why VAM can't measure teacher performance isn't because the contributions of teachers are small per se, but rather because there are all sorts of problems with all of the proposed measurement systems. In order to create a VAM system that actually worked, assuming such a thing is possible, you'd have to make a number of changes to the way that the school system is designed... e.g. assigning students to teachers at random, designing standardized tests to measure teacher quality, implementing standardized tests in all subject areas, etc. But implementing all of those changes may well be worse for student learning than firing all the bad teachers or whatever. Not to mention that there is currently a major crisis in teacher retention, with half of all teachers either getting fired or quitting in their first five years. (After spending two years and god knows how much money getting their masters degrees to go into the profession.)

yummyfajitas 12y ago
VAM’s instability Your quote concerns variance, not bias. (The main exception is the bit about "full range of achievement". It's quite true that teachers of upper class Asian students will likely have lower VAM scores than they deserve since the test caps out at 100%.) We already knew that the contributions of teachers to student achievement was relatively small long before VAM was invented. You dodged the question. Do you think policymakers and politicians should stop trying to improve teachers, and instead simply try to reduce costs? I.e., pay teachers less, increase their turnover (so that we don't have to pay expensive but ineffective senior ones), etc?
- Alex3917 12y ago
  > Your quote concerns variance, not bias. What were you originally referencing in terms of bias? > Do you think policymakers and politicians should stop trying to improve teachers, and instead simply try to reduce costs? I.e., pay teachers less, increase their turnover (so that we don't have to pay expensive but ineffective senior ones), etc? I mean the answer to any of those types of questions really depend on what kind of school system you want to create as a whole, you could go either way on any of them and have either a good or a bad school system depending on the overall design choices. But my point was that I don't think that VAM has much to say about any of these questions, because whether or not it's theoretically possible to measure teacher performance doesn't say much about whether or not good and bad teachers actually exist and whether or not they have an impact on students learning/lives. My point was that the reason we can't measure teacher performance isn't because the difference between good and bad teachers is small, even though it is relatively small in the grand scheme of things.
  - yummyfajitas 12y ago
    http://en.wikipedia.org/wiki/Bias_(statistics) http://en.wikipedia.org/wiki/Bias_(statistics) http://en.wikipedia.org/wiki/Variance http://en.wikipedia.org/wiki/Variance Alex, I'd suggest you refrain from opining that standard statistical methods fail in education until after you master basic statistics.
  - 12y ago