5 ms·

I'm pretty skeptical of the claims in this - it'd be interesting to see actual data. Evaluations are one of the extremely few levers students actually have to

I'm pretty skeptical of the claims in this - it'd be interesting to see actual data. Evaluations are one of the extremely few levers students actually have to pull when dealing with a terrible teacher. I'd expect that given an entire class' worth of evaluations you'd be able to strip outliers and get some genuine useful responses. Though the worst professor I had at RPI held our grades hostage until the evaluations were in, so they're not perfect.

cimmanom 8y ago
Tying them directly indeed seems like a bad idea (like pretty much any other naive reputation score), but that doesn’t mean you have to give up on the feedback mechanism. Just set a threshold - such as “scores an average of < 4/10”. If the score is below that threshold, invest a little more time and effort into getting more detailed evaluations and figuring out why the score is low and whether it’s a genuine problem with the professor vs something like their having high standards and lazy students. Then train or fire as appropriate.
- the_jeremy 8y ago
  Strict thresholds don't usually work. My school had public metrics, and teachers preferred classes made a huge difference. The teachers that taught the harder classes (component design rings a bell) always got significantly lower scores than the ones teaching easier classes (physics I, circuits for Mechanical Engineers)
  - cimmanom 8y ago
    That’s why you don’t use the threshold to trigger anything more punitive than further investigation. And sure, tune the threshold to the department. But also consider that maybe teaching is better on average in some departments than others. (For instance, the drop-off in teaching quality from high school to college in my experience was enormous in math and arts but slight in both sciences and humanities. Which seems to me more like a problem with the math and art departments’ approaches to hiring or teaching than with math or arts as subjects.)
    - joe_the_user 8y ago
      That’s why you don’t use the threshold to trigger anything more punitive than further investigation. And by "you", you mean "no one ever". Teacher evaluations have immediate and universal impact. There is essentially no filter and no interpretation on this. It's like tweets. Once a social signal is out-there, public, everyone is acting on the implications regardless of hypothetical mitigating factors. A lot of people talk about this stuff in the abstract, as some hypothetical, like we'll do this and we'll have safeguards in place. The news is, this is how things have been for college teachers for a while. You get an evaluation and it has an impact and there's no mitigation and no contextualizing.
olliej 8y ago
In America teachers get tenure so your feedback has no impact on them improving. Also if you look at literally any study of them TAs for harder courses get harsher ratings, they get harsher ratings from weaker students. I also spent a number of years as a TA I can say anecdotally that is my exact first hand experience - good reviews in easy courses like graphics, atrocious scores in design of concurrent systems. In fact for concurrent systems it was well known that the TA for the course would literally always get the lowest review rating in the department, and that was independent of their rating in any other courses that they TAd. TAs also get punished if the actual instructor is bad - because students blame everyone in the course. I mean the American system has its own slew of problems - lecturers run the tests themselves, they by design don’t provide prior exams (hint: if knowing the prior years exam questions tells you enough to be considered your exams are bad and you should feel bad)
- Y7ZCQtNo39 8y ago
  Very few professors are granted tenure, as was the case in the past. It's weakly non-zero: times have changed. There are far more adjunct professors today. Universities are more run like businesses than in decades past.
  - bocklund 8y ago
    I don’t have data, but I think most tenure track professors get tenure (if not at their current university, than another). Adjunct professors are not tenure track, so there is really no expectation that they would get tenure. Though I have seen the numbers that say the number of adjuncts is increasing.
    - vonmoltke 8y ago
      The point is that far fewer professors are tenure track to begin with, compared to past decades.
      - olliej 8y ago
        Almost every lecturer my wife has is tenure track or already tenured. One of them just plays videos of himself in his lectures. He doesn't actually go to them. He is actually around during less than half of his scheduled office hours. He complains to (and at least once has shouted at) the TAs about how students keep asking him questions. He has tenure.
monksy 8y ago
Now that I think about that.. I did have a professor in my undergrad who held grades until the evaluations were done. Suffice to say, I regret the evaluation I gave.
jseliger 8y ago
Evaluations are one of the extremely few levers students actually have to pull when dealing with a terrible teacher. I'd expect that given an entire class' worth of evaluations you'd be able to strip outliers and get some genuine useful responses. Schools by and large reward research, not teaching (https://jakeseliger.com/2010/09/26/how-universities-work-or-what-i-wish-id-known-freshman-year-a-guide-to-american-university-life-for-the-uninitiated/ https://jakeseliger.com/2010/09/26/how-universities-work-or-...), and, even if they do reward teaching, it's not clear that students on evals do more than reward grades.
setr 8y ago
I’ve found that the best professors tend to come with higher standards for grading, more involved assignments, and just generally more work involved. This is great if you’re actively interested in learning the subject; most students are not. In which case, its far worse than the shitty teacher, and its reflected in the feedback. From the perspective of most students, the ideal professor has low grading standards, “good enough” teaching, and most importantly, a “fun” class (primarily by means of humor). Most students are there to get a degree, to get a job. The best teacher according to students is the one that best fits that model (which is generally at odds with the goals of classical academia).
- tmin 8y ago
  One of the best classes that I had was very hard in homeworks but was easy on the grading. It was literature and art class taught by a lawyer. I felt that I grew up a bit after that class. The class was also rated high by students
  - setr 8y ago
    I was thinking primarily of my history with stem (really, specifically the t and e); I feel like the arts are still more classically oriented, because a degree isn’t anywhere near as required, and theres weaker guarantees on income streams, such that its been relatively unaffected by the university trying to fulfill the role if trade schools. The same would be true of mathematics, phds in general, and probably most of the hard sciences. Undergrad feedback I wouldn’t trust at all regardless of major, Masters is major-dependent (ce, cs are particularly screwed) and phd candidates are probably fine. Simply by looking at the why the general population is even in the major (eg ask an undergrad and the reason is probably not actual interest in the subject: parents, jobs, money, dropout-major, couldn’t decide, best grades in hs, etc). Their feedback will naturally reflect the misaligned incentives If your interest in feedback is to make sure the teachers are actually good at teaching, anyways.

cimmanom 8y ago

Tying them directly indeed seems like a bad idea (like pretty much any other naive reputation score), but that doesn’t mean you have to give up on the feedback mechanism. Just set a threshold - such as “scores an average of < 4/10”. If the score is below that threshold, invest a little more time and effort into getting more detailed evaluations and figuring out why the score is low and whether it’s a genuine problem with the professor vs something like their having high standards and lazy students. Then train or fire as appropriate.

the_jeremy 8y ago
Strict thresholds don't usually work. My school had public metrics, and teachers preferred classes made a huge difference. The teachers that taught the harder classes (component design rings a bell) always got significantly lower scores than the ones teaching easier classes (physics I, circuits for Mechanical Engineers)
- cimmanom 8y ago
  That’s why you don’t use the threshold to trigger anything more punitive than further investigation. And sure, tune the threshold to the department. But also consider that maybe teaching is better on average in some departments than others. (For instance, the drop-off in teaching quality from high school to college in my experience was enormous in math and arts but slight in both sciences and humanities. Which seems to me more like a problem with the math and art departments’ approaches to hiring or teaching than with math or arts as subjects.)
  - joe_the_user 8y ago
    That’s why you don’t use the threshold to trigger anything more punitive than further investigation. And by "you", you mean "no one ever". Teacher evaluations have immediate and universal impact. There is essentially no filter and no interpretation on this. It's like tweets. Once a social signal is out-there, public, everyone is acting on the implications regardless of hypothetical mitigating factors. A lot of people talk about this stuff in the abstract, as some hypothetical, like we'll do this and we'll have safeguards in place. The news is, this is how things have been for college teachers for a while. You get an evaluation and it has an impact and there's no mitigation and no contextualizing.

olliej 8y ago

In America teachers get tenure so your feedback has no impact on them improving. Also if you look at literally any study of them TAs for harder courses get harsher ratings, they get harsher ratings from weaker students. I also spent a number of years as a TA I can say anecdotally that is my exact first hand experience - good reviews in easy courses like graphics, atrocious scores in design of concurrent systems. In fact for concurrent systems it was well known that the TA for the course would literally always get the lowest review rating in the department, and that was independent of their rating in any other courses that they TAd. TAs also get punished if the actual instructor is bad - because students blame everyone in the course. I mean the American system has its own slew of problems - lecturers run the tests themselves, they by design don’t provide prior exams (hint: if knowing the prior years exam questions tells you enough to be considered your exams are bad and you should feel bad)

Y7ZCQtNo39 8y ago
Very few professors are granted tenure, as was the case in the past. It's weakly non-zero: times have changed. There are far more adjunct professors today. Universities are more run like businesses than in decades past.
- bocklund 8y ago
  I don’t have data, but I think most tenure track professors get tenure (if not at their current university, than another). Adjunct professors are not tenure track, so there is really no expectation that they would get tenure. Though I have seen the numbers that say the number of adjuncts is increasing.
  - vonmoltke 8y ago
    The point is that far fewer professors are tenure track to begin with, compared to past decades.
    - olliej 8y ago
      Almost every lecturer my wife has is tenure track or already tenured. One of them just plays videos of himself in his lectures. He doesn't actually go to them. He is actually around during less than half of his scheduled office hours. He complains to (and at least once has shouted at) the TAs about how students keep asking him questions. He has tenure.

monksy 8y ago

Now that I think about that.. I did have a professor in my undergrad who held grades until the evaluations were done. Suffice to say, I regret the evaluation I gave.

jseliger 8y ago

Evaluations are one of the extremely few levers students actually have to pull when dealing with a terrible teacher. I'd expect that given an entire class' worth of evaluations you'd be able to strip outliers and get some genuine useful responses. Schools by and large reward research, not teaching (https://jakeseliger.com/2010/09/26/how-universities-work-or-what-i-wish-id-known-freshman-year-a-guide-to-american-university-life-for-the-uninitiated/ https://jakeseliger.com/2010/09/26/how-universities-work-or-...), and, even if they do reward teaching, it's not clear that students on evals do more than reward grades.

setr 8y ago

I’ve found that the best professors tend to come with higher standards for grading, more involved assignments, and just generally more work involved. This is great if you’re actively interested in learning the subject; most students are not. In which case, its far worse than the shitty teacher, and its reflected in the feedback. From the perspective of most students, the ideal professor has low grading standards, “good enough” teaching, and most importantly, a “fun” class (primarily by means of humor). Most students are there to get a degree, to get a job. The best teacher according to students is the one that best fits that model (which is generally at odds with the goals of classical academia).

tmin 8y ago
One of the best classes that I had was very hard in homeworks but was easy on the grading. It was literature and art class taught by a lawyer. I felt that I grew up a bit after that class. The class was also rated high by students
- setr 8y ago
  I was thinking primarily of my history with stem (really, specifically the t and e); I feel like the arts are still more classically oriented, because a degree isn’t anywhere near as required, and theres weaker guarantees on income streams, such that its been relatively unaffected by the university trying to fulfill the role if trade schools. The same would be true of mathematics, phds in general, and probably most of the hard sciences. Undergrad feedback I wouldn’t trust at all regardless of major, Masters is major-dependent (ce, cs are particularly screwed) and phd candidates are probably fine. Simply by looking at the why the general population is even in the major (eg ask an undergrad and the reason is probably not actual interest in the subject: parents, jobs, money, dropout-major, couldn’t decide, best grades in hs, etc). Their feedback will naturally reflect the misaligned incentives If your interest in feedback is to make sure the teachers are actually good at teaching, anyways.