3 ms·
For teachers, the metric is (roughly speaking) "% of students capable of multiplying/dividing numbers up to 4 digits in a standardized test setting". How do you
by yummyfajitas 14y ago
For teachers, the metric is (roughly speaking) "% of students capable of multiplying/dividing numbers up to 4 digits in a standardized test setting". How do you game this metric?
The fact that one company used a couple of bad objective metrics doesn't mean all objective metrics are bad. They are used with a great deal of success in many fields. Sales people are paid on commission, traders are paid proportionally to (risk adjusted) profits, etc. All it means is that if you set the wrong goals for your company, you'll probably succeed at the wrong thing.
So tell us - is "maximize the % of students capable of multiplying/dividing 4 digit numbers" the wrong goal? If so, what is the right goal, and why can't it be measured?
- WalterBright 14y agoOne popular way of gaming the metric is systematic cheating on the tests by the teachers. I say popular because it has happened on a large scale, most recently in Atlanta as I recall. Another way to manipulate the test results is to manipulate which students are in your class or your school. The point is, people are endlessly creative in subverting rules to their own benefit - so they conform to the letter of the rules but not the spirit. Consider also the "work to rule" technique used by unions as a bargaining tactic. It's as simple as the workers literally adhering to their job descriptions. It doesn't work out well for the company.
- yummyfajitas 14y agoTo prevent cheating, test administration should be handled by someone other than teachers. The fact that the current system has a bunch of cheaters is not an argument against more carefully and objectively measuring the current system. What next - bankers sometimes engage in rogue trading, so we should reduce monitoring of their behavior? Another way to manipulate the test results is to manipulate which students are in your class or your school. This is very difficult with VAM, since the goal is to increase (actual score - statistically predicted score). You need to reliably identify students who will do better than their statistical predictor. I.e., you need to discover students who will improve drastically this year and then pack your student body with them.
- DanBC 14y ago> "% of students capable of multiplying/dividing numbers up to 4 digits in a standardized test setting". How do you game this metric? You use the time that you used to use for teaching reading and devote it to teaching methods for multiplying and dividing numbers up to 4 digits in a standardized test setting. You might say this isn't gaming the test, it's just a stupid way for schools to optimise their results for that metric. But that metric is mechanical; there's no measurement for deep understanding of the principles, just success or failure for applying mechanical rules by rote. Most US standardised test settings use machine marked multiple choices. I'm told that there are lessons for 'bubbling in' - lesson on how to fill in the multiple choice answer bubbles to ensure fewer mis-marked answers.
- yummyfajitas 14y agoYou use the time that you used to use for teaching reading and devote it to teaching methods for multiplying and dividing numbers up to 4 digits in a standardized test setting. Clever - you caught me. I got lazy and didn't feel like typing up all the goals of a 3'rd grade education system into an HN comment, preferring only to provide a simple example and hoping that a reasonable reader would extrapolate. Mea culpa - I'll stop assuming reasonable readers in the future. Or are you actually suggesting that the school system might forget to include reading when defining their goals?
- DanBC 14y ago> up all the goals of a 3'rd grade education system Can all the goals of a 3rd grade education system be reduced to a purely mechanical list of stuff? Let's try applying your example of arithmetic to reading. What are the goals? To get children to read individual words? Or to get children to read a sentence, and obtain meaning from it? If it's just to read words, do you get those words from a defined vocab? Should they all be real words, or do you include nonsense words too? While schools may not have stopped teaching students to read they have cut out other parts of the curriculum to focus on what's being tested. And there's a risk of a cut off point - there's a bunch of children who will read, and there's a few children who are struggling to read. Do you spend extra time and money on the few strugglers (some of whom are going to fail whatever you do), or do you concentrate on the majority (and get most of them through the test and thus look good)?
- DataWorker 14y agoA few come to mind: cheating (see Atlanta Journal-Constitution series on this) encouraging absenteeism amongst low performers damaging the test materials of low performing students to get their scores excluded getting low performers into IEPs so they don't test, take the alternate or receive accommodations sabotage other teachers in various ways vie for the most favorable students by pulling strings with the administration I'm sure if there's actual motivation one might find even more.
- yummyfajitas 14y agoMost of these issues can be addressed simply by completely separating education and evaluation. I.e., teachers take the day off when tests are administered, some bureaucrat shows up and does it instead. Treating damaged test materials/absent students as a score of 0 is the simplest way to prevent hacking the student body. vie for the most favorable students by pulling strings with the administration The most favorable students are those who will perform better than their statistical predictor. How do teachers know who those will be?
- WalterBright 14y agoThe teachers know who the good students are, just like the students know who the good teachers are.
- yummyfajitas 14y agoIf the statistical predictor in vam also knows who the good students are, this won't help. All it will do is raise the bar for that teacher.
- WalterBright 14y agoThe teachers know who the good students are, just like the students know who the good teachers are.
- WalterBright 14y agoConsider also how athletes game the drug tests - such as Lance Armstrong.