5 ms·

I have had a chance to look at course evaluation data over time and the scary thing about it is how quickly ratings can change from year to year. One year a pr

I have had a chance to look at course evaluation data over time and the scary thing about it is how quickly ratings can change from year to year. One year a professor might be considered an absolute rock star with students raving about the course, but a problematic upgrade to a piece of software used in the course can drive students absolutely mad the next year. I'm convinced that there is a group think that occurs very quickly in the term and profs are immediately labeled as either stars or bozos, and it is hard to change that perception once it has formed. Another huge factor in course evaluations is the economic environment and prospects for upcoming graduates. Tight labor markets seem to correlate with higher course evaluations as students value subjects that will hopefully make them more marketable. Conversely, a lot of courses get hammered when the job market is good because students start getting a "just give me the piece of paper so I can get out there" feeling.

Fomite 11y ago
The thing that strikes me is how small the sample sizes are - most students really can't be bothered to fill out evaluations, and those who do are probably not an unbiased sample. There's often maybe a handful of evaluations per year for anything but massive lecture classes. It's hard to even make meaningful or realistic estimates of the quality of a teacher from that data.
- bobochan 11y ago
  Very true, but there are certainly ways that schools can up the response rates if they are interested. Some schools authentication systems that store a student's ID on one system when they complete the evaluation and the actual response anonymously on another system. You can then set policy to say, "Your final grades will not be released until you complete the course evaluation." I have also noticed that giving students 10 minutes at the beginning of the last class, with the professor leaving the room, helps the rate.

Fomite 11y ago

The thing that strikes me is how small the sample sizes are - most students really can't be bothered to fill out evaluations, and those who do are probably not an unbiased sample. There's often maybe a handful of evaluations per year for anything but massive lecture classes. It's hard to even make meaningful or realistic estimates of the quality of a teacher from that data.

bobochan 11y ago
Very true, but there are certainly ways that schools can up the response rates if they are interested. Some schools authentication systems that store a student's ID on one system when they complete the evaluation and the actual response anonymously on another system. You can then set policy to say, "Your final grades will not be released until you complete the course evaluation." I have also noticed that giving students 10 minutes at the beginning of the last class, with the professor leaving the room, helps the rate.