5 ms·

This is easy to say but hard to do. "Based on excellence" is the mantra I've heard from software development circles for years and yet... 1. How many of us ha

This is easy to say but hard to do. "Based on excellence" is the mantra I've heard from software development circles for years and yet... 1. How many of us have lost productivity because some other developer ran rough-shod through code caring not for maintainability, readability, or even extensibility? Which one was praised and which one rewarded? Doesn't apply to teachers? Ever have a dud who couldn't do simple math in charge of the algebra class? Guess what the next teacher has to deal with. 2. How do "normal" people in managerial roles judge competency between two people in such roles? When looking at our own industry we see how difficult it is for "normal" people to make those sorts of judgement calls. Some schools require only 2 years teaching experience before moving on to superintendent roles. Some states more. With such divergence what can we expect? 3. How will we measure such excellence? What metric? What new system will evolve to capitalize on this metric despite the spirit of the metric? Unintended consequences?

ig1 16y ago
Generally people seem to support either performance improvement measured by exams (i.e. how much students improved over the course of the term) or some sort of peer-review system. Both have their own problems, but both seem to be much better than the arbitrary system of seniority. There is evidence to show teachers get better with experience, but almost all of that improvement happens within the first few years of a teachers career.
- tptacek 16y ago
  It's actually not obvious that a testing-based teacher measurement regime would be better than the seniority system. The seniority system does lock in mediocre teachers, but it has minimal knock-on effects to the actual curriculum. Test-based measurement has already made some curricula toxic to students, and those test results have minimal impact to the teachers themselves. How much worse would it be if the teach-to-the-test incentive was "afford a new car"?
  - ig1 16y ago
    I'm guessing less worse than the incentive to do the minimum amount possible knowing you're going to get a pay raise regardless. We can change the test if it's not measuring the right thing (we should do that anyway regardless of it's use for teacher measurement). Test scores are what parents, universities, etc. measure the performance of children by, we shouldn't have a double standard of having one metric for children and another for those who teach them.
    - _delirium 16y ago
      Imo, universities measuring applicants mainly by test scores is a bad thing, done only because they can't find a better way to do it, but I'd hardly want to subordinate all of education to that. K-12 education should teach kids useful stuff, meta-learning habits, civic information, attempt to inspire them, etc., not solely aim to maximize their SAT scores. To the extent I found K-12 education useful at all, it was only the parts that weren't narrowly aiming at maximizing my test scores, so it'd be a shame to declare those irrelevant and ditch them. Plus, universities do take some other things into account. I got significant credit on my university applications for an open-source project I worked on in high school, and a website I maintained (in the late 90s websites weren't exactly rare, but it wasn't yet to the "everyone has a blog" level of pervasiveness). Will a teacher who successfully inspires kids to do that sort of extracurricular stuff get credit in these evaluation schemes? Alternatively, if you consider preparing kids for college to be the sole purpose of K-12 education, why not just measure it directly? For 2nd-grade teachers it would be difficult to measure due to the large time lag, but it'd be possible for high-school teachers. Instead of the state inventing its own tests, just use the % of students who go on to college as the metric, and then do some statistical analysis over a 5-year window to estimate the effect a particular teacher had on that rate. There are a lot of variables and each student is influenced by many teachers, but over a window you can make an estimate of whether a particular teacher had any influence (positive or negative) on college-admission rates, compared to a hypothetical average teacher of the same subject and of a hypothetically equivalent set of students (according to whatever demographic/etc. variables). A teacher version of http://en.wikipedia.org/wiki/Value_over_replacement_player http://en.wikipedia.org/wiki/Value_over_replacement_player, basically.
deleted 16y ago
[deleted]
alphaoverlord 16y ago
This might sound trite, but I think the solution would be to couple standardized testing with performance based pay. Unlike code, it's hard to aim for solely short-term gains. As long as teachers aren't cheating, I can't imagine many ways to teach students that aren't inherently beneficial to the student. Teaching better study habits, etc might not be directly related to their material, but it still helps the students. Also, I think there are fairly generalizable development timelines for children. While standardized testing can never be a gold standard in determining progress, I think we can both agree such testing does measure a worthwhile metric. I would definitely agree that there are edge cases, both in terms of individual students and in terms of school districts, but linking pay with progress seems on face value to be a good idea. That, plus independent school districts, is system-based evolution. Such selection depends on creating the right relationship between effort and reward.
- HaloZero 16y ago
  The argument against standard test scores as the metric is that it covers two basic competencies, reading and mathematics. Which are of course fundamental but there are edge cases of people who have no interest in those fields but would prefer foreign languages, arts, etc. Teachers in the arts can inspire and help provide to students but don't have any standardized evaluation method. Plus for students who are younger where one teacher teaches all subjects, the focus on mathematics/reading will increase compared to arts. The arts education significantly reduced during No Child Left Behind b/c of the requirements. You can obviously argue that competency in basics should be achieved before arts too though. Just pointing out an issue
  - yummyfajitas 16y ago
    We can't have standardized foreign language or art tests?
    - HaloZero 16y ago
      Foreign language yes, but art? I could memorize a bunch of facts about art, styles, etc. But that's not the same as doing well in the class. I could be a brilliant artist who learned a lot in the class because the teacher motivated me but I have no idea what the difference between penciling vs stenciling. (I have no knowledge of art, I just said that example as something that sounds similar but could be very different).
WalterBright 16y ago
There is no bureaucratic rule that can be devised that can separate good teachers from bad ones. Every such rule can and will be gamed. It's like the difference between art and porn - the difference is obvious by inspection, but impossible to codify. With a free market school system, people will know who the good teachers are and will pay for them (supply and demand). With a government school system, this simply will never work, because the people paying the salaries will never be the ones judging the teachers.
- _delirium 16y ago
  Unless the teachers are going to be individually contracted by parents, don't private-sector schools have the same evaluation problem? They still have to separate good and bad teachers via some sort of process, probably overseen by management (aka bureaucracy). The similarity of the agency problems is one reason corporations internally often look so much like governments.
  - enjo 16y ago
    And fundamentally why the idea that private enterprise is always more efficient is a fundamentally flawed premise.
  - WalterBright 16y ago
    To some extent, sure, the larger the organization is the more bureaucratic and inefficient it gets at about everything. But there's still one fundamental difference - with public schools, one is forced to attend (truancy laws) and forced to pay for it.
  - tptacek 16y ago
    Private schools can fire teachers subjectively. A good private school principal can use gut feel to evaluate a teacher. But that doesn't scale, unless we find a way to identify and recruit very smart, very trustworthy principals on a large scale.
    - gruseom 16y ago
yummyfajitas 16y ago
Regarding 2), you do it via Value Added Modeling. You build a statistical predictor of student performance (based on standardized test scores) then measure Performance = Avg(Actual - predicted).
- tptacek 16y ago
  This makes sense but suggests that we need to simultaneously invest in better standardized tests. The most common standardized tests we have today make for poor year-round curricula.
  - jseliger 16y ago
    I'm always in favor of better standardized tests -- who isn't? -- but one handy thing about value-added modeling is that if you use the same flawed tests, you should get similar outcomes: if a kid is in the 10th percentile of his class when he starts his sophomore year and the 20th when he ends, as measured by similar tests, then he's progressed.
    - tptacek 16y ago
      Objection. The kid may have improved specifically in his ability to score well on a specific standardized test, and in no other way. And meanwhile, the process that delivered that superficial improvement could have immeasurably harmed other students who might have otherwise genuinely excelled.
      - forkandwait 16y ago
        Is there a study that shows that scoring well on standardized tests does NOT reflect real learning? Perhaps there is, but... There is a lot rhetoric to the effect that "scoring well on tests just means you are good at taking tests", but (1) there is nothing wrong with getting good at taking tests, and (2) are we sure? Also, the process MAY or MAY not have harmed other students -- there is no reason to believe it would, that I can tell -- again, please respond if there is.

ig1 16y ago

Generally people seem to support either performance improvement measured by exams (i.e. how much students improved over the course of the term) or some sort of peer-review system. Both have their own problems, but both seem to be much better than the arbitrary system of seniority. There is evidence to show teachers get better with experience, but almost all of that improvement happens within the first few years of a teachers career.

tptacek 16y ago
It's actually not obvious that a testing-based teacher measurement regime would be better than the seniority system. The seniority system does lock in mediocre teachers, but it has minimal knock-on effects to the actual curriculum. Test-based measurement has already made some curricula toxic to students, and those test results have minimal impact to the teachers themselves. How much worse would it be if the teach-to-the-test incentive was "afford a new car"?
- ig1 16y ago
  I'm guessing less worse than the incentive to do the minimum amount possible knowing you're going to get a pay raise regardless. We can change the test if it's not measuring the right thing (we should do that anyway regardless of it's use for teacher measurement). Test scores are what parents, universities, etc. measure the performance of children by, we shouldn't have a double standard of having one metric for children and another for those who teach them.
  - _delirium 16y ago
    Imo, universities measuring applicants mainly by test scores is a bad thing, done only because they can't find a better way to do it, but I'd hardly want to subordinate all of education to that. K-12 education should teach kids useful stuff, meta-learning habits, civic information, attempt to inspire them, etc., not solely aim to maximize their SAT scores. To the extent I found K-12 education useful at all, it was only the parts that weren't narrowly aiming at maximizing my test scores, so it'd be a shame to declare those irrelevant and ditch them. Plus, universities do take some other things into account. I got significant credit on my university applications for an open-source project I worked on in high school, and a website I maintained (in the late 90s websites weren't exactly rare, but it wasn't yet to the "everyone has a blog" level of pervasiveness). Will a teacher who successfully inspires kids to do that sort of extracurricular stuff get credit in these evaluation schemes? Alternatively, if you consider preparing kids for college to be the sole purpose of K-12 education, why not just measure it directly? For 2nd-grade teachers it would be difficult to measure due to the large time lag, but it'd be possible for high-school teachers. Instead of the state inventing its own tests, just use the % of students who go on to college as the metric, and then do some statistical analysis over a 5-year window to estimate the effect a particular teacher had on that rate. There are a lot of variables and each student is influenced by many teachers, but over a window you can make an estimate of whether a particular teacher had any influence (positive or negative) on college-admission rates, compared to a hypothetical average teacher of the same subject and of a hypothetically equivalent set of students (according to whatever demographic/etc. variables). A teacher version of http://en.wikipedia.org/wiki/Value_over_replacement_player http://en.wikipedia.org/wiki/Value_over_replacement_player, basically.

deleted 16y ago

[deleted]

alphaoverlord 16y ago

This might sound trite, but I think the solution would be to couple standardized testing with performance based pay. Unlike code, it's hard to aim for solely short-term gains. As long as teachers aren't cheating, I can't imagine many ways to teach students that aren't inherently beneficial to the student. Teaching better study habits, etc might not be directly related to their material, but it still helps the students. Also, I think there are fairly generalizable development timelines for children. While standardized testing can never be a gold standard in determining progress, I think we can both agree such testing does measure a worthwhile metric. I would definitely agree that there are edge cases, both in terms of individual students and in terms of school districts, but linking pay with progress seems on face value to be a good idea. That, plus independent school districts, is system-based evolution. Such selection depends on creating the right relationship between effort and reward.

HaloZero 16y ago
The argument against standard test scores as the metric is that it covers two basic competencies, reading and mathematics. Which are of course fundamental but there are edge cases of people who have no interest in those fields but would prefer foreign languages, arts, etc. Teachers in the arts can inspire and help provide to students but don't have any standardized evaluation method. Plus for students who are younger where one teacher teaches all subjects, the focus on mathematics/reading will increase compared to arts. The arts education significantly reduced during No Child Left Behind b/c of the requirements. You can obviously argue that competency in basics should be achieved before arts too though. Just pointing out an issue
- yummyfajitas 16y ago
  We can't have standardized foreign language or art tests?
  - HaloZero 16y ago
    Foreign language yes, but art? I could memorize a bunch of facts about art, styles, etc. But that's not the same as doing well in the class. I could be a brilliant artist who learned a lot in the class because the teacher motivated me but I have no idea what the difference between penciling vs stenciling. (I have no knowledge of art, I just said that example as something that sounds similar but could be very different).

WalterBright 16y ago

There is no bureaucratic rule that can be devised that can separate good teachers from bad ones. Every such rule can and will be gamed. It's like the difference between art and porn - the difference is obvious by inspection, but impossible to codify. With a free market school system, people will know who the good teachers are and will pay for them (supply and demand). With a government school system, this simply will never work, because the people paying the salaries will never be the ones judging the teachers.

_delirium 16y ago
Unless the teachers are going to be individually contracted by parents, don't private-sector schools have the same evaluation problem? They still have to separate good and bad teachers via some sort of process, probably overseen by management (aka bureaucracy). The similarity of the agency problems is one reason corporations internally often look so much like governments.
- enjo 16y ago
  And fundamentally why the idea that private enterprise is always more efficient is a fundamentally flawed premise.
- WalterBright 16y ago
  To some extent, sure, the larger the organization is the more bureaucratic and inefficient it gets at about everything. But there's still one fundamental difference - with public schools, one is forced to attend (truancy laws) and forced to pay for it.
- tptacek 16y ago
  Private schools can fire teachers subjectively. A good private school principal can use gut feel to evaluate a teacher. But that doesn't scale, unless we find a way to identify and recruit very smart, very trustworthy principals on a large scale.
  - gruseom 16y ago

yummyfajitas 16y ago

Regarding 2), you do it via Value Added Modeling. You build a statistical predictor of student performance (based on standardized test scores) then measure Performance = Avg(Actual - predicted).

tptacek 16y ago
This makes sense but suggests that we need to simultaneously invest in better standardized tests. The most common standardized tests we have today make for poor year-round curricula.
- jseliger 16y ago
  I'm always in favor of better standardized tests -- who isn't? -- but one handy thing about value-added modeling is that if you use the same flawed tests, you should get similar outcomes: if a kid is in the 10th percentile of his class when he starts his sophomore year and the 20th when he ends, as measured by similar tests, then he's progressed.
  - tptacek 16y ago
    Objection. The kid may have improved specifically in his ability to score well on a specific standardized test, and in no other way. And meanwhile, the process that delivered that superficial improvement could have immeasurably harmed other students who might have otherwise genuinely excelled.
    - forkandwait 16y ago
      Is there a study that shows that scoring well on standardized tests does NOT reflect real learning? Perhaps there is, but... There is a lot rhetoric to the effect that "scoring well on tests just means you are good at taking tests", but (1) there is nothing wrong with getting good at taking tests, and (2) are we sure? Also, the process MAY or MAY not have harmed other students -- there is no reason to believe it would, that I can tell -- again, please respond if there is.