> The particular VAM system used to estimate this metric may be flawed or even completely broken. But at the point in the article this pull quote is located, th

> The particular VAM system used to estimate this metric may be flawed or even completely broken. But at the point in the article this pull quote is located, that argument had not been made. > My biggest problem with the article is it doesn't describe alternate methods of teacher evaluation. VAM may be flawed, but how does it compare to the other methods? If we accept that teacher evaluation based on student performance is necessary (is it? I have no idea!), what's a better way to do it? The main point I took from the article is that the VAM approach is a black box. It is data driven and we have some reason to suspect it should be useful, but we simply have no way to audit that on specific cases. Combine that with the fact that this particular teacher is exemplary, yet got an abysmal score, we are faced with the worst possible result: Someone that should get a great score received a terrible one, and we don't know why. So we can't figure out what went wrong or how to fix it. This isn't a new issue. Statistical models used to be more human-understandable, then over several decades neural networks and big data and so forth have become popular. They often achieve good results, but often remain partially or entirely black boxes. It makes sense to use a black box method to recognize handwriting, for example, if it does well on average. It's fine if it messes up on some samples. But teachers deserve to be treated fairly, each and every single one. That's the tension here - better methods on average tend to be less auditable on individual cases.