5 ms·

Check out "The Tyranny of Metrics" book for why this won't work. Weird to say it's simple.

by diob 4y ago

WalterBright 4y ago
The current metric is gamed pretty thoroughly, wouldn't you say? Besides, businesses operate on metrics every day. It's the bottom line. It works. You just have to pick metrics that are the goal of the organization.
dmix 4y ago
I’m sure I already know the answers from experience after becoming obsessed with KPIs as the solution to startup problems to seeing how companies just game vanity metrics. But could you provide some examples of why it doesn’t work in a school environment? My best guess: It is possible to weed out non-productive KPIs, they are still extremely useful for breaking down reality distortion fields, but it takes an honest and thoughtful person with a good grasp of both the business and how the stats are collected - combined with management buy in. I have a feeling highly bureaucratic gov organizations won’t have either the talent, iterative approach, nor incentives to actually find the right metrics that matter.
- WalterBright 4y ago
  Math tests are hard to game.
- hattmall 4y ago
  It's too variable. A teacher has a variety of classes with a variety of children that changes completely every year. The outliers are enough to impact the metrics significantly but leaving them out paints an unclear picture. As well you have additional high variability of an individual students performance at any given time. This variability is greatest in the non-outliers which matter the most.
  - WalterBright 4y ago
    I answered that in the OP.
- erosenbe0 4y ago
  Metrics work when kids are prepared to learn in school, but many are not. Black boys in our worst neighborhoods have a >5% chance of being shot at least once by age 21. In the worst cities, 1/200 Black men and boys are shot every year. If you consider only those aged 13-55 in the worst areas it's more like 1/100 -- per year. This is not a simple environment to educate in. Kids don't have a safe place to play. Kids are in gangs. Kids are obese. Kids don't go to the dentist. Kids fight. Kids have high lead levels, asthma, and various trauma.
  - dmix 4y ago
    I don't follow your point. Are you saying we shouldn't commit to education metrics because a subset of the population aren't capable of fully engaging with education system? What about the kids who are engaged in education? Should they have to put their systems on hold or change it as a way to help(?) those who can't fully participate? Those seem like two different problems with two different solutions. For ex: solving the environmental issues for the smaller subset and better correlating grades with actual outcomes in education for everyone. Especially if it risks masking problems and may ultimately harm more than it helps.
    - erosenbe0 4y ago
      Oh I think we are referring to two different things. One poster suggested that performance-based pay for teachers is a simple solution and that it could be implemented with metrics. I countered because that is not simple. In addition, many teachers are doing support roles, library, phys ed, social work, reading specialist, etc. OTOH metrics and objective standards are definitely worthwhile indicators. Agreed. In fact, in the modern age of data I think we ought to look at more things to see if there is a latent pattern positive or negative. Reading and math scores are not enough. I want to see physical fitness data. I want to know how many words per minute kids can type. How many dental cavities do they have. Can they pass a civics quiz on state or federal government. Do they know basic geography and have some literacy about the world.
      - willcipriano 4y ago

Check out "The Tyranny of Metrics" book for why this won't work. Weird to say it's simple.

by diob 4y ago

Check out "The Tyranny of Metrics" book for why this won't work. Weird to say it's simple.

The current metric is gamed pretty thoroughly, wouldn't you say? Besides, businesses operate on metrics every day. It's the bottom line. It works. You just have to pick metrics that are the goal of the organization.

dmix 4y ago

I’m sure I already know the answers from experience after becoming obsessed with KPIs as the solution to startup problems to seeing how companies just game vanity metrics. But could you provide some examples of why it doesn’t work in a school environment? My best guess: It is possible to weed out non-productive KPIs, they are still extremely useful for breaking down reality distortion fields, but it takes an honest and thoughtful person with a good grasp of both the business and how the stats are collected - combined with management buy in. I have a feeling highly bureaucratic gov organizations won’t have either the talent, iterative approach, nor incentives to actually find the right metrics that matter.

WalterBright 4y ago
Math tests are hard to game.
hattmall 4y ago
It's too variable. A teacher has a variety of classes with a variety of children that changes completely every year. The outliers are enough to impact the metrics significantly but leaving them out paints an unclear picture. As well you have additional high variability of an individual students performance at any given time. This variability is greatest in the non-outliers which matter the most.
- WalterBright 4y ago
  I answered that in the OP.
erosenbe0 4y ago
Metrics work when kids are prepared to learn in school, but many are not. Black boys in our worst neighborhoods have a >5% chance of being shot at least once by age 21. In the worst cities, 1/200 Black men and boys are shot every year. If you consider only those aged 13-55 in the worst areas it's more like 1/100 -- per year. This is not a simple environment to educate in. Kids don't have a safe place to play. Kids are in gangs. Kids are obese. Kids don't go to the dentist. Kids fight. Kids have high lead levels, asthma, and various trauma.
- dmix 4y ago
  I don't follow your point. Are you saying we shouldn't commit to education metrics because a subset of the population aren't capable of fully engaging with education system? What about the kids who are engaged in education? Should they have to put their systems on hold or change it as a way to help(?) those who can't fully participate? Those seem like two different problems with two different solutions. For ex: solving the environmental issues for the smaller subset and better correlating grades with actual outcomes in education for everyone. Especially if it risks masking problems and may ultimately harm more than it helps.
  - erosenbe0 4y ago
    Oh I think we are referring to two different things. One poster suggested that performance-based pay for teachers is a simple solution and that it could be implemented with metrics. I countered because that is not simple. In addition, many teachers are doing support roles, library, phys ed, social work, reading specialist, etc. OTOH metrics and objective standards are definitely worthwhile indicators. Agreed. In fact, in the modern age of data I think we ought to look at more things to see if there is a latent pattern positive or negative. Reading and math scores are not enough. I want to see physical fitness data. I want to know how many words per minute kids can type. How many dental cavities do they have. Can they pass a civics quiz on state or federal government. Do they know basic geography and have some literacy about the world.
    - willcipriano 4y ago