8 ms·

I was a public school teacher that worked in a district that attempted to measure teacher value add. There was limited pay for performance based on that metric.

by dwater 2y ago

I was a public school teacher that worked in a district that attempted to measure teacher value add. There was limited pay for performance based on that metric. And the district I worked in was majority racial minority and majority living in poverty. "Using the expiration of preexisting collective bargaining agreements as a source of exogenous variation in the timing of changes in pay, I show that the introduction of flexible pay raised salaries of high-quality teachers, increased teacher quality (due to the arrival of high-quality teachers from other districts and increased effort), and improved student achievement." "The main dataset contains information on the universe of Wisconsin teachers, linked to student test scores to calculate teacher VA." "Student Test Scores and Demographics.—Student-level data include math and reading test scores in the Wisconsin Knowledge and Concepts Examination (WKCE 2007–2014) and the Badger test (2015–2016), for all students in grades 3 to 8, as well as demographic characteristics such as gender, race and ethnicity, socioeconomic (SES) status, migration status, English-learner status, and disability." What this study is saying is that paying teachers more for improved performance by their students on standardized tests results in higher standardized test scores of their students. That does not surprise me. In my experience, this system encouraged teaching to the test (focusing the majority of instruction on test prep rather than traditional instruction) and widespread cheating. Both of these efforts raise standardized test scores. The study assumes that standardized test scores are a direct measure of teacher quality and student achievement. In my experience that is not true.

azinman2 2y ago
My fear as well. Is there a better way to assess teacher quality, especially in challenging student environments?
- readyman 2y ago
  Don't assess teacher quality. Eliminate poverty by eliminating capitalism, a system that literally depends on it.
  - CoastalCoder 2y ago
    > Eliminate poverty by eliminating capitalism, a system that literally depends on it. Could you give an example of where this has worked?
    - dmix 2y ago
      Why even engage this sort of thing? It's just going to turn into a flame thread.
      - CoastalCoder 2y ago
        Just in case I had something to learn. Apologies for the risk of conflagration :)
    - snapplebobapple 2y ago
      If you define pverty as earning less than 15 % of the median country wage and then pay 90 % of people 2 dollars a day and 10 % of the people infinity dollars a day you have no poverty by definition. Some variation of that is how every other system of government/economy has beaten poverty. There is no point engaging these people unless you are a psychiatrist trying to help them with their mental illness.
      - philbin 2y ago
        [dead]
      - lupusreal 2y ago
        Even easier than that, simply define poverty as a property of capitalism and if anybody says that poverty also exists in your communist society, you throw them into the gulag for spreading capitalist propaganda.
  - driscoll42 2y ago
    Even in a non-capitalist society, you still need high quality teachers and ideally some way to measure them. And it's not like eliminating capitalism in any way would prove to end poverty.
- soco 2y ago
  None of those other ways can be done without effort and extra costs. And as nowadays everything has to be done cheaper and quicker, it will stay as that.
- dwater 2y ago
  First you have to define teacher quality. What should a good teacher provide to their students? Knowledge, ability, curiosity, safety, integrity, self-worth, honor, respect, kindness? Can these be quantified and compared against each other?
bena 2y ago
Bingo. My wife is a special education teacher and so much of her work does not get reflected on a test.
gadders 2y ago
>>and widespread cheating It amazes me that teachers would be so unethical. They seem worse than the general population.
- cess11 2y ago
  Where I live schools are commonly for profit, so shareholders and bosses exert pressure to get higher grades. It works, without the students learning more. Our universities are complaining that students arrive that can hardly read even though their grades say they are able to. And yeah, we have standardised tests administered centrally so we can actually to some degree measure 'grade inflation' too, and know very well that employment/pay based on grading undermines whatever slim value grading would have otherwise. Edit: Oh, and parents also exert such pressure.
- dwater 2y ago
  They are the general population, look around. Teaching no longer provides a stable middle class life in most areas. They live paycheck to paycheck, struggle to afford housing, feel crippled by student loan payments, and have little opportunity for advancement.
- jddj 2y ago
  "You get what you measure" is broadly applicable and systems need to accomodate that. There's nothing about teachers that makes them less ethical than the the average <s>car emissions system engineer</s> <s>KPMG auditor</s> person
  - gadders 2y ago
    The expectation is that they would be more ethical, in the same way you would expect EG a doctor to be.
    - Clubber 2y ago
      It's easier to be incorruptible by money when you already have a lot of money. One of the big societal problems with pushing down wages for 50 years is that it makes society less ethical because people aren't comfortable enough in their finances. When people aren't comfortable with their finances, they are less likely to be able to quickly ignore unethical things that might help their financial situation even a little bit. Quick example: You see a guy drop a $20 when stuffing his ATM withdrawal into his wallet. I would say, "hey bud, you dropped a 20." Someone else might just let him walk out and pick it up for themselves because it would help them a lot more than me.
jorvi 2y ago
A secondary question is: is the pay increase high enough to induce more talented people to become a teacher? If all it does is pull higher quality teachers from other places, it is zero sum and all you eventually end up with is the same quality schooling for a higher price. I do think teachers as a whole should be paid well, but that is a separate discussion.
- dwater 2y ago
  When I left teaching my salary went up 50% on day 1, to 100% after about a year, to 150% after 2 or 3 years, and has continued climbing significantly since. If I had stayed my salary would have lagged inflation. The pay for performance bonuses I've heard of have never been more than 20% of a teacher's salary. I'm not typical because I have CS and Math degrees, but still, I have no intention of ever considering returning to teaching. Not part of this discussion, but I did work with people who left public schools to work at charter schools because you could get 25-50% more there, but they rarely stayed more than 2 years because of the working environment being more toxic than at public schools.
brodouevencode 2y ago
What if the standardized tests aren't administered by the teacher? That seems like it would be an easy solution to get around the cheating problem. As for teaching to the test, what's the real downfall of doing that? Colleges use standardized test scores as the biggest or one of the biggest criteria in admission.
- bena 2y ago
  > As for teaching to the test, what's the real downfall of doing that? This comment is a perfect encapsulation of AI hype. Recitation is not competence. It doesn't matter if Bobby can say "4" when I say "What is 2 + 2?". What matters is if Bobby can add.
  - ryathal 2y ago
    Test questions are written to test varying levels of competence from basic recall to true mastery. It's partially why so many people hate word problems, they don't actually understand the concepts. Many others just can't read.
    - Hizonner 2y ago
      Test questions can be written to assess true mastery, although it's not so easy as you're trying to make it sound... especially if you want the test to take a sane amount of time, be consistently gradable by different graders, and have any kind of reasonable signal to noise ratio. But sure, you can get there, or at least go a long way toward it. If you try to actually create tests that way, you will discover that the institutions and political atmosphere around you will punish you harshly. It's "unfair" to give a test that you can't pass by memorizing things, you see. And the people who will ultimately make the decisions about what you're allowed to do with your test are rarely going to care much about actual mastery, and often wouldn't be able to recognize mastery if they did care. In practice, standardized tests are always going to be easily gamed, so if you make people's and institutions' rewards dependent on them, you will end up diverting more time, energy, and other resources away from actually teaching mastery and into meaningless gaming.
readyman 2y ago
The sick idea that profit is the onlyway to motivate humans is historically idiotic. It is an outcome of pathetic efforts to legitimate capitalism, a system of exploitation.
- deleted 2y ago
  [deleted]
- CoastalCoder 2y ago
  You were downvoted, but I think you're correct. When I worked for the DoD, most of us were focused on doing the right thing for the country, and we worked diligently towards that. The pay was mediocre, and potential performance bonuses were minimal. But the job stability let us focus on the big picture, and we did. I haven't worked anywhere else that had the same dynamic, even much better paying jobs in the private sector.
- jackcosgrove 2y ago
  I agree but I wouldn't pin all the blame on the big C. Marxism places a lot of emphasis on materialism, which was always one of the biggest targets for its critics. One man's profit is another's stolen labor value. Both systems of economic (that should be the clue here) thought are very materialist and don't do a good enough job assigning value to intrinsic motivation, believing in something, following a passion, etc.
bitshiftfaced 2y ago
The question is whether test scores as a measure of teacher quality is better than the next best alternative. The authors suggest that this would be seniority, but it could also be subjective measures like complaints from parents, likeability, and office politics. I agree that there are problems with test-based evaluation, but overall I believe it's the least bad solution. It gives teachers a tangible incentive to figure out why what they're doing may not be working and try something different.

by dwater 2y ago

azinman2 2y ago

My fear as well. Is there a better way to assess teacher quality, especially in challenging student environments?

readyman 2y ago
Don't assess teacher quality. Eliminate poverty by eliminating capitalism, a system that literally depends on it.
- CoastalCoder 2y ago
  > Eliminate poverty by eliminating capitalism, a system that literally depends on it. Could you give an example of where this has worked?
  - dmix 2y ago
    Why even engage this sort of thing? It's just going to turn into a flame thread.
    - CoastalCoder 2y ago
      Just in case I had something to learn. Apologies for the risk of conflagration :)
  - snapplebobapple 2y ago
    If you define pverty as earning less than 15 % of the median country wage and then pay 90 % of people 2 dollars a day and 10 % of the people infinity dollars a day you have no poverty by definition. Some variation of that is how every other system of government/economy has beaten poverty. There is no point engaging these people unless you are a psychiatrist trying to help them with their mental illness.
    - philbin 2y ago
      [dead]
    - lupusreal 2y ago
      Even easier than that, simply define poverty as a property of capitalism and if anybody says that poverty also exists in your communist society, you throw them into the gulag for spreading capitalist propaganda.
- driscoll42 2y ago
  Even in a non-capitalist society, you still need high quality teachers and ideally some way to measure them. And it's not like eliminating capitalism in any way would prove to end poverty.
soco 2y ago
None of those other ways can be done without effort and extra costs. And as nowadays everything has to be done cheaper and quicker, it will stay as that.
dwater 2y ago
First you have to define teacher quality. What should a good teacher provide to their students? Knowledge, ability, curiosity, safety, integrity, self-worth, honor, respect, kindness? Can these be quantified and compared against each other?

bena 2y ago

Bingo. My wife is a special education teacher and so much of her work does not get reflected on a test.

gadders 2y ago

>>and widespread cheating It amazes me that teachers would be so unethical. They seem worse than the general population.

cess11 2y ago
Where I live schools are commonly for profit, so shareholders and bosses exert pressure to get higher grades. It works, without the students learning more. Our universities are complaining that students arrive that can hardly read even though their grades say they are able to. And yeah, we have standardised tests administered centrally so we can actually to some degree measure 'grade inflation' too, and know very well that employment/pay based on grading undermines whatever slim value grading would have otherwise. Edit: Oh, and parents also exert such pressure.
dwater 2y ago
They are the general population, look around. Teaching no longer provides a stable middle class life in most areas. They live paycheck to paycheck, struggle to afford housing, feel crippled by student loan payments, and have little opportunity for advancement.
jddj 2y ago
"You get what you measure" is broadly applicable and systems need to accomodate that. There's nothing about teachers that makes them less ethical than the the average <s>car emissions system engineer</s> <s>KPMG auditor</s> person
- gadders 2y ago
  The expectation is that they would be more ethical, in the same way you would expect EG a doctor to be.
  - Clubber 2y ago
    It's easier to be incorruptible by money when you already have a lot of money. One of the big societal problems with pushing down wages for 50 years is that it makes society less ethical because people aren't comfortable enough in their finances. When people aren't comfortable with their finances, they are less likely to be able to quickly ignore unethical things that might help their financial situation even a little bit. Quick example: You see a guy drop a $20 when stuffing his ATM withdrawal into his wallet. I would say, "hey bud, you dropped a 20." Someone else might just let him walk out and pick it up for themselves because it would help them a lot more than me.

jorvi 2y ago

A secondary question is: is the pay increase high enough to induce more talented people to become a teacher? If all it does is pull higher quality teachers from other places, it is zero sum and all you eventually end up with is the same quality schooling for a higher price. I do think teachers as a whole should be paid well, but that is a separate discussion.

dwater 2y ago
When I left teaching my salary went up 50% on day 1, to 100% after about a year, to 150% after 2 or 3 years, and has continued climbing significantly since. If I had stayed my salary would have lagged inflation. The pay for performance bonuses I've heard of have never been more than 20% of a teacher's salary. I'm not typical because I have CS and Math degrees, but still, I have no intention of ever considering returning to teaching. Not part of this discussion, but I did work with people who left public schools to work at charter schools because you could get 25-50% more there, but they rarely stayed more than 2 years because of the working environment being more toxic than at public schools.

brodouevencode 2y ago

What if the standardized tests aren't administered by the teacher? That seems like it would be an easy solution to get around the cheating problem. As for teaching to the test, what's the real downfall of doing that? Colleges use standardized test scores as the biggest or one of the biggest criteria in admission.

bena 2y ago
> As for teaching to the test, what's the real downfall of doing that? This comment is a perfect encapsulation of AI hype. Recitation is not competence. It doesn't matter if Bobby can say "4" when I say "What is 2 + 2?". What matters is if Bobby can add.
- ryathal 2y ago
  Test questions are written to test varying levels of competence from basic recall to true mastery. It's partially why so many people hate word problems, they don't actually understand the concepts. Many others just can't read.
  - Hizonner 2y ago
    Test questions can be written to assess true mastery, although it's not so easy as you're trying to make it sound... especially if you want the test to take a sane amount of time, be consistently gradable by different graders, and have any kind of reasonable signal to noise ratio. But sure, you can get there, or at least go a long way toward it. If you try to actually create tests that way, you will discover that the institutions and political atmosphere around you will punish you harshly. It's "unfair" to give a test that you can't pass by memorizing things, you see. And the people who will ultimately make the decisions about what you're allowed to do with your test are rarely going to care much about actual mastery, and often wouldn't be able to recognize mastery if they did care. In practice, standardized tests are always going to be easily gamed, so if you make people's and institutions' rewards dependent on them, you will end up diverting more time, energy, and other resources away from actually teaching mastery and into meaningless gaming.

readyman 2y ago

The sick idea that profit is the onlyway to motivate humans is historically idiotic. It is an outcome of pathetic efforts to legitimate capitalism, a system of exploitation.

deleted 2y ago
[deleted]
CoastalCoder 2y ago
You were downvoted, but I think you're correct. When I worked for the DoD, most of us were focused on doing the right thing for the country, and we worked diligently towards that. The pay was mediocre, and potential performance bonuses were minimal. But the job stability let us focus on the big picture, and we did. I haven't worked anywhere else that had the same dynamic, even much better paying jobs in the private sector.
jackcosgrove 2y ago
I agree but I wouldn't pin all the blame on the big C. Marxism places a lot of emphasis on materialism, which was always one of the biggest targets for its critics. One man's profit is another's stolen labor value. Both systems of economic (that should be the clue here) thought are very materialist and don't do a good enough job assigning value to intrinsic motivation, believing in something, following a passion, etc.

bitshiftfaced 2y ago

The question is whether test scores as a measure of teacher quality is better than the next best alternative. The authors suggest that this would be seniority, but it could also be subjective measures like complaints from parents, likeability, and office politics. I agree that there are problems with test-based evaluation, but overall I believe it's the least bad solution. It gives teachers a tangible incentive to figure out why what they're doing may not be working and try something different.