7 ms·

All of those are examples of jobs with fairly to extremely reliable performance measurements... E.g. if a salesperson sells makes 20 sales in a quarter, then th

All of those are examples of jobs with fairly to extremely reliable performance measurements... E.g. if a salesperson sells makes 20 sales in a quarter, then there is zero chance of them getting fired because their managers think they actually made -7 sales. Whoah, suddenly you are applying a much lower level of skepticism to performance measurements now that we aren't talking about education. As you know, the mean standardized test score of students in a class (with no attempt to adjust for student quality) is just as objectively known as the number of sales. So I guess if we just did that, we'd be fine? Of course, the relevant question is how does that compare to a baseline # of sales that this salesperson could be expected to get? I knew a salesgirl doing enterprise software. In a given month she might make 0 sales, or 1, in a very good month 2. Is she a bad salesgirl because she had a month with 0 sales? That's variance, which according to you 100% invalidates a performance metric. You are making isolated demands for rigour in education that you don't seem to apply elsewhere. See this article for a description of this fallacy: http://slatestarcodex.com/2014/08/14/beware-isolated-demands-for-rigor/ http://slatestarcodex.com/2014/08/14/beware-isolated-demands... Now ask yourself, why are you doing this? Since you seem to believe good teachers exist, what evidence do you have of this? The lighting of fires in the belly of invisible dragons? http://www.godlessgeeks.com/LINKS/Dragon.htm http://www.godlessgeeks.com/LINKS/Dragon.htm

Alex3917 10y ago
Teachers aren't ranked based on mean test scores, they're ranked based on their estimated contribution to student test scores. There are two issues here: 1) How accurate are the metrics 2) how stable are the metrics over time. Here are the differences between sales and teaching: - With VAM, not only do you have stability to worry about, but you also have the question of whether or not VAM is an accurate measure of teacher contribution to student performance in the first place. With sales you might argue that the number of sales or total contribution metrics don't accurately predict CLV or something, but the ambiguity there is trivial compared with VAM. - With respect to stability, with teaching you only get test scores once a year, which means that it takes about five years until the VAM scores of teachers become relatively stable. Whereas with sales people, you get new metrics every quarter. - Since what matters the most in business is having enough cashflow to stay in business, the short term view matters more than the longterm view. Whereas with students, what they achieve five and ten years out is much more important than how they perform at the end of each year. - With sales, there is zero barrier to entry, and if you get fired then you can just get another job two weeks later. With teaching you need to spend 2+ years and tens of thousands of dollars to get into the profession, and if you get fired then the best case scenario is that it takes an entire year to get another job. But often you're basically just banned from the profession. - With sales you're also making several times more money than you would as a teacher.
- yummyfajitas 10y ago
  If teachers were estimated on mean test scores, we could rank them the same way we often rank salespeople. Would you be ok with this? If not, why not? And more importantly, why don't your newfound rationalizations apply equally well to salespeople or traders? (Incidentally, traders also are measured against a statistical model - risk adjustments + benchmark rate.) VAM is an accurate measure of teacher contribution to student performance in the first place. And with sales, you need to determine whether your attribution/commission model is an accurate measure of salesperson compensation to company profits. It's the exact same problem, it uses similar statistical methods, and it has variance. * Whereas with sales people, you get new metrics every quarter.* So test every quarter and evaluate on that basis. Done. Further, many salespeople have less information per month than teachers - for example, the enterprise salesgirl I mentioned earlier who has 0, 1 or 2 sales/month. You seem to be desperately reaching for rationalizations that explain why teaching is fundamentally different than every other profession. Why is that?
  - nitrogen 10y ago
    ...teaching is fundamentally different than every other profession. Why is that? The payoff of a good teacher shows up years down the road, not once per quarter. Teaching is extremely politicized; parents and random joes like to manipulate the system and are likewise manipulable by less scrupulous members of the public with a hidden agenda -- this is the "think of the children" phenomenon writ large. Teacher class assignments are politically motivated, not random, so student quality is not random. There are substantial movements by parents to eliminate standardized tests because they don't want their children labeled or profiled, and yet you want to increase testing to every quarter. The fears of those parents are somewhat justified because testing software keeps all kinds of behavioral metrics on students that are never revealed to parents. You have an insanely powerful union to deal with that doesn't necessarily represent the true interests of teachers.
  - Alex3917 10y ago
    > If teachers were estimated on mean test scores, we could rank them the same way we often rank salespeople. Good sales people create value by bringing in money, and sales are measured in money. Good teachers create value by teaching well, and mean test scores don't measure teaching well. That might be slightly simplistic, but that's the big picture idea. > And with sales, you need to determine whether your attribution/commission model is an accurate measure of salesperson compensation to company profits I'm not arguing that sales people are always fairly compensated, only that there is less complexity in doing so and the epistemological issues are more straightforward. > So test every quarter and evaluate on that basis. In sales, measuring is free, whereas testing is more like growing carrots... If you keep pulling up your carrots to check how big they are, then at the end of the summer you're not going to have any carrots. > explain why teaching is fundamentally different than every other profession I never said or implied this.
nitrogen 10y ago
One problem with standardized tests in my region is that teachers are not allowed to count them on grades, so students will just walk into the testing center, click "A" 150 times, and stare blankly for the rest of the time. That's hardly a reflection on teacher quality. Student quality (and administrative or support staff manipulation of student-to-teacher assignments) varies a lot from teacher to teacher and year to year, probably much more than teacher quality. You talk about using quantitative measurements, but then treat variance like a qualitative thing that is equally ignorable regardless of scope.

Alex3917 10y ago

Teachers aren't ranked based on mean test scores, they're ranked based on their estimated contribution to student test scores. There are two issues here: 1) How accurate are the metrics 2) how stable are the metrics over time. Here are the differences between sales and teaching: - With VAM, not only do you have stability to worry about, but you also have the question of whether or not VAM is an accurate measure of teacher contribution to student performance in the first place. With sales you might argue that the number of sales or total contribution metrics don't accurately predict CLV or something, but the ambiguity there is trivial compared with VAM. - With respect to stability, with teaching you only get test scores once a year, which means that it takes about five years until the VAM scores of teachers become relatively stable. Whereas with sales people, you get new metrics every quarter. - Since what matters the most in business is having enough cashflow to stay in business, the short term view matters more than the longterm view. Whereas with students, what they achieve five and ten years out is much more important than how they perform at the end of each year. - With sales, there is zero barrier to entry, and if you get fired then you can just get another job two weeks later. With teaching you need to spend 2+ years and tens of thousands of dollars to get into the profession, and if you get fired then the best case scenario is that it takes an entire year to get another job. But often you're basically just banned from the profession. - With sales you're also making several times more money than you would as a teacher.

yummyfajitas 10y ago
If teachers were estimated on mean test scores, we could rank them the same way we often rank salespeople. Would you be ok with this? If not, why not? And more importantly, why don't your newfound rationalizations apply equally well to salespeople or traders? (Incidentally, traders also are measured against a statistical model - risk adjustments + benchmark rate.) VAM is an accurate measure of teacher contribution to student performance in the first place. And with sales, you need to determine whether your attribution/commission model is an accurate measure of salesperson compensation to company profits. It's the exact same problem, it uses similar statistical methods, and it has variance. * Whereas with sales people, you get new metrics every quarter.* So test every quarter and evaluate on that basis. Done. Further, many salespeople have less information per month than teachers - for example, the enterprise salesgirl I mentioned earlier who has 0, 1 or 2 sales/month. You seem to be desperately reaching for rationalizations that explain why teaching is fundamentally different than every other profession. Why is that?
- nitrogen 10y ago
  ...teaching is fundamentally different than every other profession. Why is that? The payoff of a good teacher shows up years down the road, not once per quarter. Teaching is extremely politicized; parents and random joes like to manipulate the system and are likewise manipulable by less scrupulous members of the public with a hidden agenda -- this is the "think of the children" phenomenon writ large. Teacher class assignments are politically motivated, not random, so student quality is not random. There are substantial movements by parents to eliminate standardized tests because they don't want their children labeled or profiled, and yet you want to increase testing to every quarter. The fears of those parents are somewhat justified because testing software keeps all kinds of behavioral metrics on students that are never revealed to parents. You have an insanely powerful union to deal with that doesn't necessarily represent the true interests of teachers.
- Alex3917 10y ago
  > If teachers were estimated on mean test scores, we could rank them the same way we often rank salespeople. Good sales people create value by bringing in money, and sales are measured in money. Good teachers create value by teaching well, and mean test scores don't measure teaching well. That might be slightly simplistic, but that's the big picture idea. > And with sales, you need to determine whether your attribution/commission model is an accurate measure of salesperson compensation to company profits I'm not arguing that sales people are always fairly compensated, only that there is less complexity in doing so and the epistemological issues are more straightforward. > So test every quarter and evaluate on that basis. In sales, measuring is free, whereas testing is more like growing carrots... If you keep pulling up your carrots to check how big they are, then at the end of the summer you're not going to have any carrots. > explain why teaching is fundamentally different than every other profession I never said or implied this.

nitrogen 10y ago

One problem with standardized tests in my region is that teachers are not allowed to count them on grades, so students will just walk into the testing center, click "A" 150 times, and stare blankly for the rest of the time. That's hardly a reflection on teacher quality. Student quality (and administrative or support staff manipulation of student-to-teacher assignments) varies a lot from teacher to teacher and year to year, probably much more than teacher quality. You talk about using quantitative measurements, but then treat variance like a qualitative thing that is equally ignorable regardless of scope.