18 ms·

Students Evaluating Teachers Doesn’t Just Hurt Teachers, It Hurts Students

179 points by chmaynard 8y ago

blakesterz 8y ago
This is a short commentary piece, not exactly much there. There's some interesting comments, the one at the top right now says "They are no doubt imprecise, but for a majority of the faculty the scores form a tight bunch around a mean. Newer faculty are likely to have lower scores, so it is often times useful to sit down with them and review the feedback and talk about ways to improve. A few faculty members continually score higher than average, and I want to know why--perhaps they can help do some mentoring with the lower scoring faculty."
- jimhefferon 8y ago
  > mentoring with the lower scoring faculty Of course this comment assumes that higher scores means better teaching. FWIW, my experience both as a prof and as an evaluator of profs is that when students are reasonable good their comments, in general, are reasonably useful. After all, they are in the room and they see what is happening. The problem is that when students diverge from reasonably well-prepared and reasonably hard-working, their responses diverge from valuable. It is a kind of Dunning–Kruger, where you get evaluations that are hard to reconcile with any kind of learning goals standard. All this would be mitigated if there were multiple measures. But there are not, at least in practice that I've seen. It is a problem. As an evaluator you want to give people credit for doing a good job. But it can be hard to tell. As a prof for many years, I know how to increase evaluations. (I don't agree that it is grades; I recently had a fifth year review and calculated the correlation between my grades and class evaluations for those years and r^2 was basically zero.) But the things that increase evaluations are not very tied to increasing learning and are certainly not tied to increasing the amount of material covered.
fossuser 8y ago
I'm pretty skeptical of the claims in this - it'd be interesting to see actual data. Evaluations are one of the extremely few levers students actually have to pull when dealing with a terrible teacher. I'd expect that given an entire class' worth of evaluations you'd be able to strip outliers and get some genuine useful responses. Though the worst professor I had at RPI held our grades hostage until the evaluations were in, so they're not perfect.
- cimmanom 8y ago
  Tying them directly indeed seems like a bad idea (like pretty much any other naive reputation score), but that doesn’t mean you have to give up on the feedback mechanism. Just set a threshold - such as “scores an average of < 4/10”. If the score is below that threshold, invest a little more time and effort into getting more detailed evaluations and figuring out why the score is low and whether it’s a genuine problem with the professor vs something like their having high standards and lazy students. Then train or fire as appropriate.
  - the_jeremy 8y ago
    Strict thresholds don't usually work. My school had public metrics, and teachers preferred classes made a huge difference. The teachers that taught the harder classes (component design rings a bell) always got significantly lower scores than the ones teaching easier classes (physics I, circuits for Mechanical Engineers)
    - cimmanom 8y ago
      That’s why you don’t use the threshold to trigger anything more punitive than further investigation. And sure, tune the threshold to the department. But also consider that maybe teaching is better on average in some departments than others. (For instance, the drop-off in teaching quality from high school to college in my experience was enormous in math and arts but slight in both sciences and humanities. Which seems to me more like a problem with the math and art departments’ approaches to hiring or teaching than with math or arts as subjects.)
      - joe_the_user 8y ago
        That’s why you don’t use the threshold to trigger anything more punitive than further investigation. And by "you", you mean "no one ever". Teacher evaluations have immediate and universal impact. There is essentially no filter and no interpretation on this. It's like tweets. Once a social signal is out-there, public, everyone is acting on the implications regardless of hypothetical mitigating factors. A lot of people talk about this stuff in the abstract, as some hypothetical, like we'll do this and we'll have safeguards in place. The news is, this is how things have been for college teachers for a while. You get an evaluation and it has an impact and there's no mitigation and no contextualizing.
olliej 8y ago
No shit basing compensation on reviews by people who have an overt bias to blame the instructor. The purpose of the review should be purely to provide feedback to the instructor and TA. It should not be available to the administration - the purpose is to help the TA and instructors improve their teaching. Especially if the instructors have tenure and so can’t be fired for failing to teach. Ratings should also be scaled to control for biases - what are the average grades given by each student, how do they compare to other students in their classes, and to their course grade.
- eastWestMath 8y ago
  My university explicitly rewards TAs with high reviews. I swear I’ve walked in on people handing out assignment solutions.
  - olliej 8y ago
    In the US at least you're essentially expected to be a TA as part of most PhD programs, so if your ability to complete your PhD is dependent on the will of students who you might be causing to fail - or you might just not be a good teach - why would you not do that?
rb808 8y ago
My best advice I got about college teaching: University is a place of higher learning, not higher teaching. If you're relying on being spoon fed the course, you're doing it wrong.
- invalidOrTaken 8y ago
  What exactly are universities selling in that case? From an individual standpoint, sure, a student is always well-advised to take charge of their own learning process. But if you can do that, what do you need a university for? At this point I'm pretty sure the answer boils down to "a piece of paper and a dating pool."
  - thrower123 8y ago
    Pieces of paper that open the gateways to a comfortable white-collar middle class life. At least putatively. The more expensive ones sell connections that can open the doors to wealth and power.
  - novia 8y ago
    BaaS (Boss as a Service)
  - baroffoos 8y ago
    Honestly I wouldn't be surprised if uni for tech becomes a thing of the past. Uni didn't teach me anything I didn't already know from working on personal projects and im fairly sure my public git repos had far more effect in job interviews than uni did. I have talked to others who have finished and they hardly know anything compared to those I know who are self taught.
  - MaxBarraclough 8y ago
    No, because there's far more to a good learning environment than having good teachers. I could pretend that I could've learnt the same amount by sitting at home with a reading-list for four years, but there's no way I would have. I commented on this recently in the context of MOOCs -- https://news.ycombinator.com/item?id=18511176 https://news.ycombinator.com/item?id=18511176
Y7ZCQtNo39 8y ago
We pushed very hard at my alma mater to publicize course/teacher evaluations. We eventually got the University to accept iff a minimum participation rate was met (on a per course basis-- so it wasn't all or nothing. We'd get a subset of the data when enough participation occurred. We considered this reasonable, since if very few students participate, the data isn't all that meaningful). Regarding professors wanting to keep review data tightly sealed: in my view, if you can't by public disclosure of your evaluation, then you either don't feel you're meeting expectations, or have no desire to improve in areas where students feel improvement could be made. Also, the biases pointed out in these reviews aren't unique to academia. Gender and age biases exist everywhere. This article sounds like it's just pushing the idea that students should have less influence in the hiring and promotion decisions of professors. You know, the very people that the teachers first and foremost serve at a university.
- paulie_a 8y ago
  If anything you would think professors would welcome reviews. They already get reviewed anyways. It's just word of mouth. At least a student could see a much wider range of opinions vs one person's experience.
  - Y7ZCQtNo39 8y ago
    We made our own review system, to compete with the contracted-out system they used for their siloed data we didn't have access to. We agreed to not launch the system, in exchange for limited access to the data, as I described above. Professors en masse opposed opening up this data. That opposition alone made me feel we were doing good work. If they had to stand by their public reviews, hopefully they could stand by the quality of the instructional experience provided. I ended up making a startup to deal with issues like this, to sell SaaS solutions directly to student governance groups, rather than to institutions. Both control fairly significant budgets (student governance groups at mid-to-large sized institutions have 6/7 figure budgets to easily afford enterprise pricing). My platforms are student-first. It's more a passion project than trying to get me Zuckerberg levels of wealth.
  - monksy 8y ago
tamaharbor 8y ago
I had taught two semesters of a basic pharmacy course at a local community college. Of the 20 or so student reviews I received, 80% of them mentioned my 'ta-tas'. Enough said.
- dgellow 8y ago
  What does “ta-tas” mean?
  - travisjungroth 8y ago
    Breasts.
- Fomite 8y ago
  I've read the evaluations of a few of my female colleagues. Without fail, a good portion of them are toe-curlingly sexist.
  - jhasse 8y ago
    Care to give an example? Are you male or female?
    - Fomite 8y ago
      I'm male. Lots of students under the impression that commenting on their dress is appropriate. A large chunk of those being suggesting they dress in ways more visually interesting. Some direct references to their appearance (usually "positive"). One particularly egregious one speculating on who she slept with to have her position. Tons of gendered expectations in language. Expectations to "be more nurturing" and things like that. Far more comments about them being "young", "inexperienced" and "new" than my own evaluations, despite being considerably more experienced than I am (and no, there was no way my class was just more polished - it was thrown together last minute).
      - belorn 8y ago
starbeast 8y ago
I would also assume that the students, taken as an average, would also be some of the worst qualified people to evaluate educators, pretty much by definition.
RcouF1uZ4gsC 8y ago
An interesting way to evaluate professors would be to contact alumni who have graduated and ask them about the professors that they remember. Chances are that the really good and the really bad will stand out. I still remember my calculus professor and statistics professor who both did a great job. I remember some other professors on the other hand who were bad at teaching undergraduates. My opinions on my college professors seems independent of the grade I received or how hard the class was. Often times, time and experience give a different perspective on how valuable something was.
- Fomite 8y ago
  > Often times, time and experience give a different perspective on how valuable something was. Indeed. I'm a professor in a STEM field. One of the most valuable courses I ever took was in medieval feminist art history.
epx 8y ago
I was a visiting professor in a local university for a while. The student evaluations were directly proportional to the grades - easier courses = better grades. I refused to see the evaluations and refused to discuss them with my supervisor, it was pointless. BTW my long-term relationship with ex-students is great, at least with the ones I care about.
ccleve 8y ago
If the school would force teachers to grade on a curve, the problem would go away. Teachers would have only a limited budget of high grades to give away. Grading on a curve is not without problems, particularly when class sizes are small, but it has a lot of benefits. It makes it easier to compare students across schools when the schools use the same curve.
- azhenley 8y ago
  That is a horrible idea. Then students will try to take courses with weak students. Oh, that smart group is taking CS340 next semester? Guess I'll wait until the following semester. Do not turn it into even more of a competition than it already is. Why should someone else's ability influence my grade???
  - thaumasiotes 8y ago
    Because the meaning of your grade is that it gets compared to their grade.
    - azhenley 8y ago
      Says who? Why can't it be used to evaluate my individual learning/effort? Grades only make sense to compare within the same class at the same university during the same semester.
      - the_jeremy 8y ago
        Grades are relative to other people. That's the point of places asking for GPA. If you were just evaluating your effort / learning, your grades could just say "learned a lot" or "made a good effort". But GPA is a way to discriminate between potential candidates for schools/businesses, and thus needs to be quantifiable and comparable.
        azhenley 8y ago
        Some schools have found how useless GPAs are and have adopted narrative evaluations [1]. The grading in my course is not relative. I set a rubric that I assess each individual on. Sure, within the class you can compare grades, but I wouldn't say it is particularly meaningful. Comparing between classes or universities is completely meaningless. People just want a simple number that can accurately represent someone's knowledge/learning/skill/effort, but it doesn't exist. [1] https://en.wikipedia.org/wiki/Narrative_evaluation https://en.wikipedia.org/wiki/Narrative_evaluation
waterpigcow 8y ago
It is my suspicion that the attitude of american students towards their professors is perhaps a function of the privatization of schools. since students are paying for their education they feel entitled to a good grade regardless of performance. I would like to know others thoughts on this. I don't know much about post secondary education in Europe but it seems different from the American collegiate system somehow.
- sdinsn 8y ago
  > privatization of schools Hmm? There has always been private schools. Nothing is new about them
  - dragonwriter 8y ago
    Over the last several decades, US public colleges and universities have generally seen a declining share of funding from the chartering governments, relying more on business activity plus student fees. This is probably what is being referred to.
    - travisoneill1 8y ago
      The student fees and tuition are mostly paid for by government backed student loans. There really isn't any private education in the US past high school.
      - waterpigcow 8y ago
        I don't see the government paying for my college... In my experience a majority of my friends have or are on track to have student debt from having to pay for college.
        Fomite 8y ago
        Technically you wouldn't see it even if it was there. For most public universities, it comes as a direct allocation from the state legislature to the university.
katsura 8y ago
To be honest, I'm not sure how teachers get and keep their jobs, but as students are graded on their studying skills, I feel like teachers should be graded every fews years on their teaching skills. My girlfriend is studying in a distance university, so she basically never meets any of her teachers. This year she has to do practices at a chosen company, and she got a teacher assigned to her who is supposed to "help" her. In her study guides it is written that the teacher has to give her classes every week. After first contact the teacher said that she will give every information by email (or other kind of online communication), and won't waste the students' precious time with classes. At first this sounded awesome, because my girlfriend has a lot to do in her last year. BUT... Since then 2 months have passed, the teacher has been repeatedly asked to keep at least a few classes, because things are not moving forward. She ignores her messages for days, answers in very short sentences, and although it is mandatory for her to keep the classes, she is always "busy" at those times and doesn't offer any other dates, so it is impossible to meet her. My girlfriend spends crazy hours combing through PDFs hoping to find information on how to do her practices, what she should prepare, how to write her work diary, etc. And the other day, when she tried to contact the teacher's higher-ups to do something about this, one guy basically shouted at her, on the phone, saying that she is making her teacher look bad, even though the said theacher works really hard, and she should just listen to the teacher and let her be. After some of this preaching he just put the phone down. It has been a really frustrating experience for us.
- joe_the_user 8y ago
  To be honest, I'm not sure how teachers get and keep their jobs, but as students are graded on their studying skills, I feel like teachers should be graded every fews years on their teaching skills. Well, to be honest, I really wish everyone who wants to one more hurdle in front of teachers should understand the correlation between bad teaching and teachers having a low-paid, unrewarding and effectively abusive job. Lousy teachers keep their job because no one highly competent wants these jobs. Lousy teachers keep their job because a huge array of bureaucratic song-and-dance exists and people good at that can be bad at teaching. I'm sorry you had that experience but you might consider looking at the system. Corporate customer support is terrible, let's all call for more test-of-competence for these low paid flunkies too. That will address the problem.
duchenne 8y ago
A possible solution would be to split the job, to avoid conficts of interest. One teacher teaches. Another one grades. When I was studying in France in "prépa", all students wanted to get a real evaluation of their level, because the really important exam to enter the best universities was organized nation-wide. But, this is hard to organize in private universities where students are also clients. Which parent would pay 40k$/y to be told that his kid does not study well? Maybe, here also, to avoid conflicts of interest, the job can be split. One university teaches. Another organization grades.
- chmod775 8y ago
  This is precisely what some private schools in Germany do (yes, there are a few of those). They might've started doing so to combat the notion that private education is worth less because it just might be easier.
nijave 8y ago
This article seems to suggest larger problem with grades than reviews. It seems like reviews are just aggravating the grade problem.
Overtonwindow 8y ago
Possible alternative method is to hire and train students to be observers, both covert and overt. Put them into classes as if they were students, and those students Would then Evaluate that professor. It could work especially cross colleges, where those students will never naturally take classes by that professor.
- pas 8y ago
  Around here the student associations have seats on the accreditation bodies and they do indeed sit on in classes as they tour the county. I don't think this is effective, but it happens.
avani 8y ago
Student reviews are also ridiculously sexist in the average case. In a recent study, the students rated identical teachers differently enough by gender that actual teaching performance was lost in the noise: https://academic.oup.com/jeea/advance-article-abstract/doi/10.1093/jeea/jvx057/4850534?redirectedFrom=fulltext https://academic.oup.com/jeea/advance-article-abstract/doi/1... Another pair of studies showed that identical courses where students thought instructors were male rated higher than those that had a female professor, and their qualitative answers were also very different: https://www.insidehighered.com/news/2018/03/14/study-says-students-rate-men-more-highly-women-even-when-theyre-teaching-identical https://www.insidehighered.com/news/2018/03/14/study-says-st... I'm grateful to be a computer scientist, where the consolation prize for not getting tenure is a cushy industry job, but I feel deeply for my pre-tenure colleagues in the life sciences who have to work twice as hard for their evaluations while still maintaining their research and service requirements.
- craigzucchini 8y ago
  From what I gathered from the mobile version (which may be paired down), the difference was not that significant, particularly in the analysis of evaluations and RMP submissions. Would have liked to find more on class composition, and would be curious how reproducible the relatively minor absolute differences would be, but "ridiculously sexist" seems a bit hyperbolic. Do you have a specific take on those studies that inform your opinion? I could be very well working with not enough information here.
- YorkshireSeason 8y ago
  students rated identical teachers Did you actually read critically the papers you cite? [1] claims to be comparing "identical teachers" but [1]'s sole claim to courses taught by male and female instructors is "neither students’ grades nor self-study hours are affected by the instructor’s gender". Clearly those are hardly the only factors relevant to teacher quality. Moreover [1] claims to use "objective measure of the instructors’ performance", and which includes -- I kid you not -- "self-reported number of hours students spent studying for the course". [2] is similarly vague, and claims that "the courses were identical: all lectures, assignments, and content were exactly the same in all sections" only to to state in the next sentence that the "only aspects of the course that varied between Dr. Mitchell’s and Dr. Martin’s sections were the course grader and contact with the instructor". Well, isn't "contact with the instructor" significant? Both [1, 2] use p-values [3], which doesn't increase confidence in the results. As an aside, neither paper discusses potential bias the authors might have, in particular their own social desirability bias [4]. [1] https://academic.oup.com/jeea/advance-article-abstract/doi/10.1093/jeea/jvx057/4850534?redirectedFrom=fulltext https://academic.oup.com/jeea/advance-article-abstract/doi/1... [2] https://www.insidehighered.com/news/2018/03/14/study-says-students-rate-men-more-highly-women-even-when-theyre-teaching-identical https://www.insidehighered.com/news/2018/03/14/study-says-st... [3] https://en.wikipedia.org/wiki/P-value https://en.wikipedia.org/wiki/P-value [4] https://en.wikipedia.org/wiki/Social_desirability_bias https://en.wikipedia.org/wiki/Social_desirability_bias
booleandilemma 8y ago
It’s been awhile since I was a student, but I remember the reviews on ratemyprofessors.com to be pretty accurate. There were some asshole teachers employed at my college and they were called out for it accordingly.
killjoywashere 8y ago
Teachers should be judged on the longitudinal performance of their students. Khan Academy and BlackBoard must have some utterly fascinating data in this regard.
gdsimoes 8y ago
One of the main problems with the educational system is that students are evaluated by the same person that teaches them. If the teacher wasn’t the one grading the mean grade of the students would be a decent evaluation of the performance of teacher.
- gnicholas 8y ago
  This is what “high stakes testing” is for. Not that it’s perfect, but the idea is to have objective third-party measurements.
- khawkins 8y ago
  If another professor wrote the exams for a class they didn't sit in, you'd have the students screaming in unison, "this wasn't discussed in class!" Not exactly disagreeing with the idea, but students have been normalized to the idea that, with enough studying and concentration, any exam can be aced. When they feel like they've been cheated by the system they get discouraged, making a harmful learning experience.
  - pas 8y ago
    Pretty small price to get fair evaluations. Furthermore it forces teachers to communicate the requurements, the curriculum and sillabus well to the exam writers and graders.
  - michaelt 8y ago
    If another professor wrote the exams for a class they didn't sit in, you'd have the students screaming in unison, "this wasn't discussed in class!" In standard testing at other levels, this is solved by a syllabus. For example, in electronic engineering I would expect a second-year class in switch mode power supply design at any university to look extremely similar. Admittedly there may be classes in cutting-edge research, opinion-based subjects or extremely niche topics, where you can't find two academics in the country who could agree on a year's syllabus and set of answers. In my degree I'd say less than 25% of courses were that way, though.
  - tk75x 8y ago
tgsovlerkhgsel 8y ago
As I expected, this is an opinion piece from a teacher whining because they got bad ratings: > I got a terrible rating, and its publication humiliated me. While this doesn't mean that the article can't make valid points, large parts of it also complain about "today's youth". I suspect this got upvoted mostly based on the contrarian headline...
gaius 8y ago
It’s funny, a teacher’s whole job consists of marking work and assigning grades, but see how reluctant they are to have their own work graded! The entire teaching racket needs to be torn up and started again from scratch.
zeroname 8y ago
Grade inflation in "American Literature"? Shocking, indeed. Not only are these poor students paying ludicrous sums of money for worthless education, their meaningless grades are now even easier to obtain. Folks, let's just be honest and admit that this kind of education is for entertainment purposes only and therefore the consumers are the rightful arbitrators on who does the entertainment.
augbog 8y ago
As someone who had a teacher who literally was tenured, had the absolute worst reviews ever on the website and wherever, literally recorded his lectures once and now goes to lectures just to answer questions (which btw he barely answers em he sometimes just posts to the lecture video) I strongly feel evaluating teachers is a necessary thing.
- wahern 8y ago
  Is something preventing you from communicating your concerns directly to the administration (in person or anonymously) outside the context of a perfunctory, after-the-fact review? I would imagine a truly poor teacher would standout for the number of unsolicited complaints.
  - augbog 8y ago
    Truthfully I have no idea what he even does with the required review papers that are given to us a the end of the class. Lots of people have reviewed him on Askmyprofessor.
wisty 8y ago
There's a much better way to hold educators to account - have other educators sit in on classes. But this is often very strongly resisted, apparently it's 'unprofessional'.
- AndyMcConachie 8y ago
  I'd be curious to hear who thinks this is unprofessional. Not any educators I know.
  - kiriakasis 8y ago
    well, forcing another teacher in the class with the intent of supervision of the teacher is obviously a terrible idea. Having two teacher in every class collaborating during the lecture or as an aide to supervise the students can be a obviously good idea. Generally speaking, in my opinion, teaching is a very complex and personal interaction and micromanaging only makes it worse.
  - jcagalawan 8y ago
    While I can't find any specific sources in the field of education, I was taught in engineering school that engineers reviewing the work of another engineer, unless their job specifically required them to review work, was against the code of ethics in Ontario. It looks like this is true as well in the USA which has this tidbit in their code of ethics.[1] "a. An Engineer in private practice will not review the work of another engineer for the same client, except with the knowledge of such engineer, or unless the connection of such engineer with the work has been terminated." [1]http://ethics.iit.edu/ecodes/node/4099 http://ethics.iit.edu/ecodes/node/4099
    - nickloewen 8y ago
      This is different though, because of the “except with knowledge of said engineer” line. This code is trying to prevent clients from, for example, getting a “second opinion” on a project if they are unhappy that the first engineer deemed it unsafe. This isn’t really an issue in teaching. I think that it would be impolite to just drop in on a colleagues class unannounced, especially if it is a bit lecture where you might go unnoticed, but I don’t see why a prearranged visit would be unprofessional per se. (Perhaps because it would cause students to doubt the quality of their instructor?) FWIW my understanding is that at my university the response to poor evals is a visit from a colleague, to assess whether the teacher really is weak or if the material is just difficult. Depending on that, the instructor may get additional coaching in teaching technique. This seems like a sensible approach to me.
sonnyblarney 8y ago
In my experience this has been good for weeding out really bad profs. At my Grad school they turfed a prof 1/2 way through that was bones bad. That said - it's perverse at the other end and I do believe that there is unconscious sexism. We had a really smart bunch of people, and some of the female teachers were a little more apprehensive in front of us which I think signals to people subconsciously. Another way: the feedback has to be interpreted. I suggest that the profs should not be 'graded' by students - rather there should just be an open ended opportunity for feedback.
enriquto 8y ago
As a teacher, I disagree strongly with the conclusions of this article. Anonymous student evaluations are an invaluable tool to improve my teaching. The students say so many fair and useful things!
- kiriakasis 8y ago
  I think the problem is when student evaluation have an affect (positive or negative) on the teacher career.
  - enriquto 8y ago
    Yes, when that measure becomes a target it may have some unintended consequences. But the device is too useful to be discarded for that reason.
    - narag 8y ago
      The teachers can do it voluntarily and for their own eyes, without making it an evaluation form.
danmaz74 8y ago
There could be sexism at work or not, but how can you even define "identical" teachers? I never saw two teachers that were identical (even if they had the same gender, ethnicity, age and sexual orientation).
DanielBMarkham 8y ago
This is the age-old doctor question. You go to the doctor's office because you have a medical problem. Doctor 1 comes in on-time, she's smart, capable, friendly, and spends a good amount of time concerned about you and your situation. Doctor 2 comes in late, she's irascible, pedantic, hurried, and only listens long enough to tell you what to do. She also insults you and your family in various ways and doesn't seem to care. After the visit, you go to Yelp. Which one was the better doctor? You have no idea! But do you think that's going to stop somebody from forming and posting an opinion anyway? You can make the argument that these other qualities are good to have in a doctor. I agree. I want a doctor like the first one. But people go to doctors first and foremost to get better, not to make new social acquaintances. If the first doc is incompetent and hurting people, and the second doc a genius, and her attitude is actually causing more people to comply with what's best for them out of a sense anger at her style, perhaps? That's the thing. Who the hell cares if the outside looks cool/sexy/friendly or not? So when we ask for reviews from students who have never worked in the field they're studying for, they're like that Yelp doctor reviewer. I don't see how this situation is good for anybody. In fact, it'll probably lead to a bunch friendly, good-looking morons with great people skills entertaining bunch of kids who aren't learning anything. That's the only natural consequence here.
- pas 8y ago
  It's not like there can't be an accurate way to measure this. Pre-register patient problems, and see how fast doctors solve problems on average/median. Then it doesn't matter how jovial/amicable the doc is. With enough data you can fish out potential signals (and then test them properly), maybe bedside manner really doesn't matter, maybe being on time is more important. (Because then people spend less time in the clinic among other sick people.) And as others mentioned the teacher eval problem can be solved by splitting the lecturer, exam writer and grader roles. (Grading should be done by a book written as edge cases accumulate, edge cases should be handled without names to prevent bias, as much as possible.) And then aggregate stats can be published about each class and school.
  - DanielBMarkham 8y ago
    Sorry no, it doesn't work like that. In either case. Doctors, like professors, are evaluated by former patients based on a mix of factors. In your doctor example, doctors don't "solve problems". They see patients. Many times these patients present with conflicting and hard-to-resolve symptoms. I imagine with the internet this has gotten worse. They use a mix of Occam's Razor, deductive reasoning, and social psychology for the patient's long-term health. You don't want a doctor that finds the answers to complex problems and gives out morphine. You want a doctor that over decades has patients with the best outcomes. In my example, the one doctor may have ticked you off so much you decide never to go back -- and quit smoking just to prove her wrong. That's a win for both you and the doc, although there's no problem being solved. You can speculate various ways to go at this. Perhaps the doctor is evaluated by other doctors working at that location. Or perhaps it's a matrix of various things. As long as you're just pulling stuff out of the air, you can invent any kind of system you'd like. And tech will deliver it for you. Human interventions are messy affairs, and thank goodness for that. We are not robots. Your proposal might do a great job of sorting out professors who teach students well enough to take the test, but heck if I see where that is the end of the analysis. That's just the beginning. We're just getting started talking about professors who can lead a class to mostly pass a test. That should be a baseline for any professor. This is the problem we see in software development. You can come up with all sorts of metrics to stand in place of evaluating whether a team/person is good or not, but whatever you come up with mostly seem to work -- and has very bad effects on those being measured. (With the possible exception of business-value-loop feedback time) I don't think this is tractable in the way you seem to want it. If I were picking a college or professor, I might look at five or six variables and weigh them in various ways depending on my personal needs and goals at the time -- which might change tomorrow. I might want to go interview them. Meet some current and former students. There is a severe and onerous "platform danger" in tech. It starts with the conceit that we can build platforms for anything. It ends up with people not-as-smart-as-they-used-to-be having Google tell them where to go for dinner. Or Wally in Dilbert coding himself up a minivan. [0] I'm not saying the metric definitions and accumulations aren't interesting. They deserve to be considered. I'm saying that by reducing human judgment to single vector, it makes us as a species dumber. That's one of those intuitive things that sound good but are in reality quite bad. 0. https://dilbert.com/strip/1995-11-13 https://dilbert.com/strip/1995-11-13
nocobot 8y ago
I'm a CS student in Germany and contrary to the authors claim we actually do evaluate teachers (to be fair that might not have been common place when she was teaching). This is obviously anecdotal but in my experience these ratings seem to reflect my perceived quality of the lectures quite well.
xtrapolate 8y ago
> "A teacher (also called a school teacher or, in some contexts, an educator) is a person who helps others to acquire knowledge, competences or values." The target audience is a crucial part of the feedback loop. Removing the students from the equation sounds counter intuitive. What are the proposed alternative metrics? Who delivers the feedback? And what is the feedback based on, if not on the direct opinions of the people on the receiving end of the service?
- JacKTrocinskI 8y ago
  I think the main idea here is that student evaluations only make sense if we assume that students are actually interested in acquiring knowledge, competences, or values, however if their main aim in a course is a good grade then their evaluations suddenly hold little value since they will rate "easy" instructors highest.
  - dual_basis 8y ago
    When I was a college instructor I found that when I tried hard - putting a lot of thought into pedagogy, having weekly (open notes) quizzes, assigning challenging but fair homework - I got terrible evaluations, even though the average grade in my class was higher than other teachers who were teaching the same course. When I phoned it in, didn't really try, was very lax with the homework and often made the quizzes take-home, I got excellent reviews but the average grade in the class was worse.
    - thatcat 8y ago
      Did your reviews not ask for justification of the ratings?
      - dual_basis 8y ago
        There are a bunch of ratings 1-10, and then a place to optionally add comments. It's hard to consolidate information from 200+ comments, but, then again, most students do not fill out the comment section anyway. (In a sense, I guess the whole thing is optional - after all, it is anonymous, and we don't check that every student has filled the whole thing out. It also switched from in-class to an online form during the time I was teaching, and the response rate went down quite a bit as a result, but this was after the situation I mentioned above.)
skywhopper 8y ago
This strikes me as a very whiny article, not worthy of publishing in the Chronicle. A lifetime of hurt feelings and resentments bubbling up into a rant full of straw men and frankly irresponsible misinformation about the role student evaluations play in instructor assessment. It’s too bad, too, because there certainly are criticisms to be made, and this article hints at some of them. But a well designed evaluation instrument along with critical interpretation of the results and comments can be an invaluable tool for instructors who want to improve their teaching. But unfortunately this article also spreads untruths about the weight given to student evaluations, and dismisses common-sense security protocols like having a student deliver the results instead of the instructor (not perfect of course, but the student has much less incentive to tamper) as demeaning rather than a sensible precaution. If the author had presented any evidence whatsoever to support her claims linking student evals to grade inflation (possibly related, and I’m sure studies have been done, but evals are very far from the only obvious contributing factor), or even made a passing attempt to explain what percentage of weight is given to raw eval scores for actual tenure considerations (very little in the broad scheme of things), and cut the junk out about how universities seeing students as customers is a new thing (it’s not new at all as any skim of the history of universities as an institution will make clear, but someone at a US state university should realize that it’s skyrocketing tuition rates, and not whiny students, that give students a bigger sense of entitlement) the article would have been much better.
- michaelmior 8y ago
  While I think it's true that the article overemphasized the weight placed on student evaluations somewhat, I think this does get to the real problem. There's nothing wrong with students evaluating instructors. The problem comes when this is the only method used to evaluate teaching effectiveness. I'm a newly appointed professor, but I've heard many stories from colleagues in the past about students thanking them for being demanding in their course as what they learned was helpful in the future. Demanding classes tend to have negative impact on student evaluations, but that's obviously not capturing the whole picture.
- wahern 8y ago
  How does a sense of entitlement result in grade inflation? Well, the obvious way is if you pervasively require students evaluations, students systematically evaluate professors based on the grades they received, and these evaluations are used to make staffing and compensation decisions by the administration, incentivizing strategic grading behavior by professors. Solution: stop the evaluations. Even most entitled students won't make unsolicited complaints. And of the ones who do? As long as student complaints aren't used to rank every professor relative to each other, but rather only to weed out the most egregious instructors, there's significantly less pressure for grade inflation. It's when you provide the administration a regular, systematized, "objective" stream of data points that the temptation to assess professors this way becomes irresistible for the administration.
weliketocode 8y ago
Let me a add a wrinkle to the mix specifically in terms of grade inflation. The value of most coursework and any learnings gleaned from it, has dropped significantly. The best value from entering education institutions is instead from job opportunities after. Unfortunately, these opportunities are often based on grades. If the grades are more important than the coursework, then of course students will optimize for it. Students should choose institutions & teachers based on both quality of opportunities and ease of coursework. Going a step further, teachers that don't understand the changing interests of their students should be reprimanded. At my university, most major recruiting for internships and such happened in one semester. Now, I specifically remember the dichotomy of two of my professors' approach to this. Prof John - If you miss a class/deadline/exam due to recruiting, you get a 0. Prof George - I understand recruiting is happening, reach out to me ASAP if you feel overwhelmed or need a date changed. Now, the coursework has largely been forgotten. But, I will never forget John's idiocy.
- sevensor 8y ago
  I think you'll come to regret this attitude when you reach your mid-30s with no skills other than the concrete procedural skills you picked up on the job. I use my education constantly, and in ways I wouldn't have anticipated. I often see people doing things the hard way because they don't know there's any other way, or passing on opportunities to do something interesting because they don't realize it's even possible.
  - quacked 8y ago
    The comment you're responding to isn't an attitude that the poster has, it's an explanation of how students are responding to changes in incentives. As the total number of college graduates grows, they're competing in job markets that are smaller than the total number of college graduates. This ratio is unbalanced and becoming more unbalanced with every year. The people hiring college graduates are developing stricter and stricter filters in order to decide what body of people they're even going to allow to compete in the job market; the biggest, easiest filter to sort out who's truly competitive for positions is grades, and so it makes sense that students compete to get the best grades. rather than competing to be the most competent. Most grads are leaving college in tremendous debt with the understanding that the longer they go without employment, the less likely it is they'll end up employed. This means that while they're in college, rather than focusing on the material they're learning, they're focusing on whatever will allow them to have the largest advantage when they enter the job market. In order for this trend to reverse, the cost of college needs to be dramatically lessened; I think this could be accomplished by reducing the total percentage of high school graduates that attend college immediately, increasing the prevalence of online schools, and reducing university overhead by cutting services to students and administrative positions.
chuckgreenman 8y ago
Sure, evalutations are flawed, but it appears that universities are unable or unwilling to develop other quality measures. For example, this semester I've had a networking professor that would get half way into explaining something interesting and then state that he didn't care about the topic so he didn't prepare to talk about it and we'd be moving on to something else. I've got an A in that class, but I wouldn't say that I learned much from it due to that attitude. I don't think that the CS department here has any way to tell that something like that is happening without evaluations.
- bane 8y ago
  Right. I think anybody reading comments in these review sites can see if students are ranking professors well for being "easy" or if they're ranking them well for being great educators. The universities I went to for my undergrad and graduate programs both had an informal rating system for professors and I definitely encountered some of both. One of the most highly rated professors I encountered was also one of the best educators at the school. He had a reputation as being a brutally hard grader. But most of the students who left his classes with some bruises and (what would have been in any other class) a mediocre grade, ached to take more from him because you felt the immense value of his teaching. I had other professors who taught poorly, tolerated cheating, or gave non-sequitur exams on material they never covered and so on. There was absolutely no recourse or way of providing feedback on these professors and even ones who received universally poor reviews at the end of the semester stayed on and even attained tenure in some cases. The only real recourse was for students to rank professors amongst themselves and simply starve out poor professors with lack of registrations. That's the only signal universities seem to respond to.
  - chuckgreenman 8y ago
    I got the impression that the author was talking about department run evaluation sheets rather than review sites. Student evaluations shouldn't be seen as an evaluation metric but a smoke detector that indicates a staff member might need some more direction. Teaching is an art, just because someone isn't doing it well doesn't mean they can't do it well. In fact in some cases I imagine they are doing poorly because they would rather be working on their research area. Providing the opportunity to course correct rather than starve out is good.
cyberpanther 8y ago
This is also a problem with developer bootcamps since they get a lot of "sales" from reviews. They can often cater to students to the detriment of the learning experience. At times it is about catering to creature comforts, but can bleed over into lessening the learning experience. And especially in bootcamps since things are so fast paced and often times emotional, the reviews are not objective. Many are too positive or too negative.

179 points by chmaynard 8y ago

This is a short commentary piece, not exactly much there. There's some interesting comments, the one at the top right now says "They are no doubt imprecise, but for a majority of the faculty the scores form a tight bunch around a mean. Newer faculty are likely to have lower scores, so it is often times useful to sit down with them and review the feedback and talk about ways to improve. A few faculty members continually score higher than average, and I want to know why--perhaps they can help do some mentoring with the lower scoring faculty."

jimhefferon 8y ago
> mentoring with the lower scoring faculty Of course this comment assumes that higher scores means better teaching. FWIW, my experience both as a prof and as an evaluator of profs is that when students are reasonable good their comments, in general, are reasonably useful. After all, they are in the room and they see what is happening. The problem is that when students diverge from reasonably well-prepared and reasonably hard-working, their responses diverge from valuable. It is a kind of Dunning–Kruger, where you get evaluations that are hard to reconcile with any kind of learning goals standard. All this would be mitigated if there were multiple measures. But there are not, at least in practice that I've seen. It is a problem. As an evaluator you want to give people credit for doing a good job. But it can be hard to tell. As a prof for many years, I know how to increase evaluations. (I don't agree that it is grades; I recently had a fifth year review and calculated the correlation between my grades and class evaluations for those years and r^2 was basically zero.) But the things that increase evaluations are not very tied to increasing learning and are certainly not tied to increasing the amount of material covered.

fossuser 8y ago

I'm pretty skeptical of the claims in this - it'd be interesting to see actual data. Evaluations are one of the extremely few levers students actually have to pull when dealing with a terrible teacher. I'd expect that given an entire class' worth of evaluations you'd be able to strip outliers and get some genuine useful responses. Though the worst professor I had at RPI held our grades hostage until the evaluations were in, so they're not perfect.

cimmanom 8y ago
Tying them directly indeed seems like a bad idea (like pretty much any other naive reputation score), but that doesn’t mean you have to give up on the feedback mechanism. Just set a threshold - such as “scores an average of < 4/10”. If the score is below that threshold, invest a little more time and effort into getting more detailed evaluations and figuring out why the score is low and whether it’s a genuine problem with the professor vs something like their having high standards and lazy students. Then train or fire as appropriate.
- the_jeremy 8y ago
  Strict thresholds don't usually work. My school had public metrics, and teachers preferred classes made a huge difference. The teachers that taught the harder classes (component design rings a bell) always got significantly lower scores than the ones teaching easier classes (physics I, circuits for Mechanical Engineers)
  - cimmanom 8y ago
    That’s why you don’t use the threshold to trigger anything more punitive than further investigation. And sure, tune the threshold to the department. But also consider that maybe teaching is better on average in some departments than others. (For instance, the drop-off in teaching quality from high school to college in my experience was enormous in math and arts but slight in both sciences and humanities. Which seems to me more like a problem with the math and art departments’ approaches to hiring or teaching than with math or arts as subjects.)
    - joe_the_user 8y ago
      That’s why you don’t use the threshold to trigger anything more punitive than further investigation. And by "you", you mean "no one ever". Teacher evaluations have immediate and universal impact. There is essentially no filter and no interpretation on this. It's like tweets. Once a social signal is out-there, public, everyone is acting on the implications regardless of hypothetical mitigating factors. A lot of people talk about this stuff in the abstract, as some hypothetical, like we'll do this and we'll have safeguards in place. The news is, this is how things have been for college teachers for a while. You get an evaluation and it has an impact and there's no mitigation and no contextualizing.

olliej 8y ago

No shit basing compensation on reviews by people who have an overt bias to blame the instructor. The purpose of the review should be purely to provide feedback to the instructor and TA. It should not be available to the administration - the purpose is to help the TA and instructors improve their teaching. Especially if the instructors have tenure and so can’t be fired for failing to teach. Ratings should also be scaled to control for biases - what are the average grades given by each student, how do they compare to other students in their classes, and to their course grade.

eastWestMath 8y ago
My university explicitly rewards TAs with high reviews. I swear I’ve walked in on people handing out assignment solutions.
- olliej 8y ago
  In the US at least you're essentially expected to be a TA as part of most PhD programs, so if your ability to complete your PhD is dependent on the will of students who you might be causing to fail - or you might just not be a good teach - why would you not do that?

rb808 8y ago

My best advice I got about college teaching: University is a place of higher learning, not higher teaching. If you're relying on being spoon fed the course, you're doing it wrong.

invalidOrTaken 8y ago
What exactly are universities selling in that case? From an individual standpoint, sure, a student is always well-advised to take charge of their own learning process. But if you can do that, what do you need a university for? At this point I'm pretty sure the answer boils down to "a piece of paper and a dating pool."
- thrower123 8y ago
  Pieces of paper that open the gateways to a comfortable white-collar middle class life. At least putatively. The more expensive ones sell connections that can open the doors to wealth and power.
- novia 8y ago
  BaaS (Boss as a Service)
- baroffoos 8y ago
  Honestly I wouldn't be surprised if uni for tech becomes a thing of the past. Uni didn't teach me anything I didn't already know from working on personal projects and im fairly sure my public git repos had far more effect in job interviews than uni did. I have talked to others who have finished and they hardly know anything compared to those I know who are self taught.
- MaxBarraclough 8y ago
  No, because there's far more to a good learning environment than having good teachers. I could pretend that I could've learnt the same amount by sitting at home with a reading-list for four years, but there's no way I would have. I commented on this recently in the context of MOOCs -- https://news.ycombinator.com/item?id=18511176 https://news.ycombinator.com/item?id=18511176

Y7ZCQtNo39 8y ago

We pushed very hard at my alma mater to publicize course/teacher evaluations. We eventually got the University to accept iff a minimum participation rate was met (on a per course basis-- so it wasn't all or nothing. We'd get a subset of the data when enough participation occurred. We considered this reasonable, since if very few students participate, the data isn't all that meaningful). Regarding professors wanting to keep review data tightly sealed: in my view, if you can't by public disclosure of your evaluation, then you either don't feel you're meeting expectations, or have no desire to improve in areas where students feel improvement could be made. Also, the biases pointed out in these reviews aren't unique to academia. Gender and age biases exist everywhere. This article sounds like it's just pushing the idea that students should have less influence in the hiring and promotion decisions of professors. You know, the very people that the teachers first and foremost serve at a university.

paulie_a 8y ago
If anything you would think professors would welcome reviews. They already get reviewed anyways. It's just word of mouth. At least a student could see a much wider range of opinions vs one person's experience.
- Y7ZCQtNo39 8y ago
  We made our own review system, to compete with the contracted-out system they used for their siloed data we didn't have access to. We agreed to not launch the system, in exchange for limited access to the data, as I described above. Professors en masse opposed opening up this data. That opposition alone made me feel we were doing good work. If they had to stand by their public reviews, hopefully they could stand by the quality of the instructional experience provided. I ended up making a startup to deal with issues like this, to sell SaaS solutions directly to student governance groups, rather than to institutions. Both control fairly significant budgets (student governance groups at mid-to-large sized institutions have 6/7 figure budgets to easily afford enterprise pricing). My platforms are student-first. It's more a passion project than trying to get me Zuckerberg levels of wealth.
- monksy 8y ago

tamaharbor 8y ago

I had taught two semesters of a basic pharmacy course at a local community college. Of the 20 or so student reviews I received, 80% of them mentioned my 'ta-tas'. Enough said.

dgellow 8y ago
What does “ta-tas” mean?
- travisjungroth 8y ago
  Breasts.
Fomite 8y ago
I've read the evaluations of a few of my female colleagues. Without fail, a good portion of them are toe-curlingly sexist.
- jhasse 8y ago
  Care to give an example? Are you male or female?
  - Fomite 8y ago
    I'm male. Lots of students under the impression that commenting on their dress is appropriate. A large chunk of those being suggesting they dress in ways more visually interesting. Some direct references to their appearance (usually "positive"). One particularly egregious one speculating on who she slept with to have her position. Tons of gendered expectations in language. Expectations to "be more nurturing" and things like that. Far more comments about them being "young", "inexperienced" and "new" than my own evaluations, despite being considerably more experienced than I am (and no, there was no way my class was just more polished - it was thrown together last minute).
    - belorn 8y ago

starbeast 8y ago

I would also assume that the students, taken as an average, would also be some of the worst qualified people to evaluate educators, pretty much by definition.

RcouF1uZ4gsC 8y ago

An interesting way to evaluate professors would be to contact alumni who have graduated and ask them about the professors that they remember. Chances are that the really good and the really bad will stand out. I still remember my calculus professor and statistics professor who both did a great job. I remember some other professors on the other hand who were bad at teaching undergraduates. My opinions on my college professors seems independent of the grade I received or how hard the class was. Often times, time and experience give a different perspective on how valuable something was.

Fomite 8y ago
> Often times, time and experience give a different perspective on how valuable something was. Indeed. I'm a professor in a STEM field. One of the most valuable courses I ever took was in medieval feminist art history.

epx 8y ago

I was a visiting professor in a local university for a while. The student evaluations were directly proportional to the grades - easier courses = better grades. I refused to see the evaluations and refused to discuss them with my supervisor, it was pointless. BTW my long-term relationship with ex-students is great, at least with the ones I care about.

ccleve 8y ago

If the school would force teachers to grade on a curve, the problem would go away. Teachers would have only a limited budget of high grades to give away. Grading on a curve is not without problems, particularly when class sizes are small, but it has a lot of benefits. It makes it easier to compare students across schools when the schools use the same curve.

azhenley 8y ago
That is a horrible idea. Then students will try to take courses with weak students. Oh, that smart group is taking CS340 next semester? Guess I'll wait until the following semester. Do not turn it into even more of a competition than it already is. Why should someone else's ability influence my grade???
- thaumasiotes 8y ago
  Because the meaning of your grade is that it gets compared to their grade.
  - azhenley 8y ago
    Says who? Why can't it be used to evaluate my individual learning/effort? Grades only make sense to compare within the same class at the same university during the same semester.
    - the_jeremy 8y ago
      Grades are relative to other people. That's the point of places asking for GPA. If you were just evaluating your effort / learning, your grades could just say "learned a lot" or "made a good effort". But GPA is a way to discriminate between potential candidates for schools/businesses, and thus needs to be quantifiable and comparable.
      - azhenley 8y ago
        Some schools have found how useless GPAs are and have adopted narrative evaluations [1]. The grading in my course is not relative. I set a rubric that I assess each individual on. Sure, within the class you can compare grades, but I wouldn't say it is particularly meaningful. Comparing between classes or universities is completely meaningless. People just want a simple number that can accurately represent someone's knowledge/learning/skill/effort, but it doesn't exist. [1] https://en.wikipedia.org/wiki/Narrative_evaluation https://en.wikipedia.org/wiki/Narrative_evaluation

waterpigcow 8y ago

It is my suspicion that the attitude of american students towards their professors is perhaps a function of the privatization of schools. since students are paying for their education they feel entitled to a good grade regardless of performance. I would like to know others thoughts on this. I don't know much about post secondary education in Europe but it seems different from the American collegiate system somehow.

sdinsn 8y ago
> privatization of schools Hmm? There has always been private schools. Nothing is new about them
- dragonwriter 8y ago
  Over the last several decades, US public colleges and universities have generally seen a declining share of funding from the chartering governments, relying more on business activity plus student fees. This is probably what is being referred to.
  - travisoneill1 8y ago
    The student fees and tuition are mostly paid for by government backed student loans. There really isn't any private education in the US past high school.
    - waterpigcow 8y ago
      I don't see the government paying for my college... In my experience a majority of my friends have or are on track to have student debt from having to pay for college.
      - Fomite 8y ago
        Technically you wouldn't see it even if it was there. For most public universities, it comes as a direct allocation from the state legislature to the university.

katsura 8y ago

To be honest, I'm not sure how teachers get and keep their jobs, but as students are graded on their studying skills, I feel like teachers should be graded every fews years on their teaching skills. My girlfriend is studying in a distance university, so she basically never meets any of her teachers. This year she has to do practices at a chosen company, and she got a teacher assigned to her who is supposed to "help" her. In her study guides it is written that the teacher has to give her classes every week. After first contact the teacher said that she will give every information by email (or other kind of online communication), and won't waste the students' precious time with classes. At first this sounded awesome, because my girlfriend has a lot to do in her last year. BUT... Since then 2 months have passed, the teacher has been repeatedly asked to keep at least a few classes, because things are not moving forward. She ignores her messages for days, answers in very short sentences, and although it is mandatory for her to keep the classes, she is always "busy" at those times and doesn't offer any other dates, so it is impossible to meet her. My girlfriend spends crazy hours combing through PDFs hoping to find information on how to do her practices, what she should prepare, how to write her work diary, etc. And the other day, when she tried to contact the teacher's higher-ups to do something about this, one guy basically shouted at her, on the phone, saying that she is making her teacher look bad, even though the said theacher works really hard, and she should just listen to the teacher and let her be. After some of this preaching he just put the phone down. It has been a really frustrating experience for us.

joe_the_user 8y ago
To be honest, I'm not sure how teachers get and keep their jobs, but as students are graded on their studying skills, I feel like teachers should be graded every fews years on their teaching skills. Well, to be honest, I really wish everyone who wants to one more hurdle in front of teachers should understand the correlation between bad teaching and teachers having a low-paid, unrewarding and effectively abusive job. Lousy teachers keep their job because no one highly competent wants these jobs. Lousy teachers keep their job because a huge array of bureaucratic song-and-dance exists and people good at that can be bad at teaching. I'm sorry you had that experience but you might consider looking at the system. Corporate customer support is terrible, let's all call for more test-of-competence for these low paid flunkies too. That will address the problem.

duchenne 8y ago

A possible solution would be to split the job, to avoid conficts of interest. One teacher teaches. Another one grades. When I was studying in France in "prépa", all students wanted to get a real evaluation of their level, because the really important exam to enter the best universities was organized nation-wide. But, this is hard to organize in private universities where students are also clients. Which parent would pay 40k$/y to be told that his kid does not study well? Maybe, here also, to avoid conflicts of interest, the job can be split. One university teaches. Another organization grades.

chmod775 8y ago
This is precisely what some private schools in Germany do (yes, there are a few of those). They might've started doing so to combat the notion that private education is worth less because it just might be easier.

nijave 8y ago

This article seems to suggest larger problem with grades than reviews. It seems like reviews are just aggravating the grade problem.

Overtonwindow 8y ago

Possible alternative method is to hire and train students to be observers, both covert and overt. Put them into classes as if they were students, and those students Would then Evaluate that professor. It could work especially cross colleges, where those students will never naturally take classes by that professor.

pas 8y ago
Around here the student associations have seats on the accreditation bodies and they do indeed sit on in classes as they tour the county. I don't think this is effective, but it happens.

avani 8y ago

Student reviews are also ridiculously sexist in the average case. In a recent study, the students rated identical teachers differently enough by gender that actual teaching performance was lost in the noise: https://academic.oup.com/jeea/advance-article-abstract/doi/10.1093/jeea/jvx057/4850534?redirectedFrom=fulltext https://academic.oup.com/jeea/advance-article-abstract/doi/1... Another pair of studies showed that identical courses where students thought instructors were male rated higher than those that had a female professor, and their qualitative answers were also very different: https://www.insidehighered.com/news/2018/03/14/study-says-students-rate-men-more-highly-women-even-when-theyre-teaching-identical https://www.insidehighered.com/news/2018/03/14/study-says-st... I'm grateful to be a computer scientist, where the consolation prize for not getting tenure is a cushy industry job, but I feel deeply for my pre-tenure colleagues in the life sciences who have to work twice as hard for their evaluations while still maintaining their research and service requirements.

craigzucchini 8y ago
From what I gathered from the mobile version (which may be paired down), the difference was not that significant, particularly in the analysis of evaluations and RMP submissions. Would have liked to find more on class composition, and would be curious how reproducible the relatively minor absolute differences would be, but "ridiculously sexist" seems a bit hyperbolic. Do you have a specific take on those studies that inform your opinion? I could be very well working with not enough information here.
YorkshireSeason 8y ago
students rated identical teachers Did you actually read critically the papers you cite? [1] claims to be comparing "identical teachers" but [1]'s sole claim to courses taught by male and female instructors is "neither students’ grades nor self-study hours are affected by the instructor’s gender". Clearly those are hardly the only factors relevant to teacher quality. Moreover [1] claims to use "objective measure of the instructors’ performance", and which includes -- I kid you not -- "self-reported number of hours students spent studying for the course". [2] is similarly vague, and claims that "the courses were identical: all lectures, assignments, and content were exactly the same in all sections" only to to state in the next sentence that the "only aspects of the course that varied between Dr. Mitchell’s and Dr. Martin’s sections were the course grader and contact with the instructor". Well, isn't "contact with the instructor" significant? Both [1, 2] use p-values [3], which doesn't increase confidence in the results. As an aside, neither paper discusses potential bias the authors might have, in particular their own social desirability bias [4]. [1] https://academic.oup.com/jeea/advance-article-abstract/doi/10.1093/jeea/jvx057/4850534?redirectedFrom=fulltext https://academic.oup.com/jeea/advance-article-abstract/doi/1... [2] https://www.insidehighered.com/news/2018/03/14/study-says-students-rate-men-more-highly-women-even-when-theyre-teaching-identical https://www.insidehighered.com/news/2018/03/14/study-says-st... [3] https://en.wikipedia.org/wiki/P-value https://en.wikipedia.org/wiki/P-value [4] https://en.wikipedia.org/wiki/Social_desirability_bias https://en.wikipedia.org/wiki/Social_desirability_bias

booleandilemma 8y ago

It’s been awhile since I was a student, but I remember the reviews on ratemyprofessors.com to be pretty accurate. There were some asshole teachers employed at my college and they were called out for it accordingly.

killjoywashere 8y ago

Teachers should be judged on the longitudinal performance of their students. Khan Academy and BlackBoard must have some utterly fascinating data in this regard.

gdsimoes 8y ago

One of the main problems with the educational system is that students are evaluated by the same person that teaches them. If the teacher wasn’t the one grading the mean grade of the students would be a decent evaluation of the performance of teacher.

gnicholas 8y ago
This is what “high stakes testing” is for. Not that it’s perfect, but the idea is to have objective third-party measurements.
khawkins 8y ago
If another professor wrote the exams for a class they didn't sit in, you'd have the students screaming in unison, "this wasn't discussed in class!" Not exactly disagreeing with the idea, but students have been normalized to the idea that, with enough studying and concentration, any exam can be aced. When they feel like they've been cheated by the system they get discouraged, making a harmful learning experience.
- pas 8y ago
  Pretty small price to get fair evaluations. Furthermore it forces teachers to communicate the requurements, the curriculum and sillabus well to the exam writers and graders.
- michaelt 8y ago
  If another professor wrote the exams for a class they didn't sit in, you'd have the students screaming in unison, "this wasn't discussed in class!" In standard testing at other levels, this is solved by a syllabus. For example, in electronic engineering I would expect a second-year class in switch mode power supply design at any university to look extremely similar. Admittedly there may be classes in cutting-edge research, opinion-based subjects or extremely niche topics, where you can't find two academics in the country who could agree on a year's syllabus and set of answers. In my degree I'd say less than 25% of courses were that way, though.
- tk75x 8y ago

tgsovlerkhgsel 8y ago

As I expected, this is an opinion piece from a teacher whining because they got bad ratings: > I got a terrible rating, and its publication humiliated me. While this doesn't mean that the article can't make valid points, large parts of it also complain about "today's youth". I suspect this got upvoted mostly based on the contrarian headline...

gaius 8y ago

It’s funny, a teacher’s whole job consists of marking work and assigning grades, but see how reluctant they are to have their own work graded! The entire teaching racket needs to be torn up and started again from scratch.

zeroname 8y ago

Grade inflation in "American Literature"? Shocking, indeed. Not only are these poor students paying ludicrous sums of money for worthless education, their meaningless grades are now even easier to obtain. Folks, let's just be honest and admit that this kind of education is for entertainment purposes only and therefore the consumers are the rightful arbitrators on who does the entertainment.

augbog 8y ago

As someone who had a teacher who literally was tenured, had the absolute worst reviews ever on the website and wherever, literally recorded his lectures once and now goes to lectures just to answer questions (which btw he barely answers em he sometimes just posts to the lecture video) I strongly feel evaluating teachers is a necessary thing.

wahern 8y ago
Is something preventing you from communicating your concerns directly to the administration (in person or anonymously) outside the context of a perfunctory, after-the-fact review? I would imagine a truly poor teacher would standout for the number of unsolicited complaints.
- augbog 8y ago
  Truthfully I have no idea what he even does with the required review papers that are given to us a the end of the class. Lots of people have reviewed him on Askmyprofessor.

wisty 8y ago

There's a much better way to hold educators to account - have other educators sit in on classes. But this is often very strongly resisted, apparently it's 'unprofessional'.

AndyMcConachie 8y ago
I'd be curious to hear who thinks this is unprofessional. Not any educators I know.
- kiriakasis 8y ago
  well, forcing another teacher in the class with the intent of supervision of the teacher is obviously a terrible idea. Having two teacher in every class collaborating during the lecture or as an aide to supervise the students can be a obviously good idea. Generally speaking, in my opinion, teaching is a very complex and personal interaction and micromanaging only makes it worse.
- jcagalawan 8y ago
  While I can't find any specific sources in the field of education, I was taught in engineering school that engineers reviewing the work of another engineer, unless their job specifically required them to review work, was against the code of ethics in Ontario. It looks like this is true as well in the USA which has this tidbit in their code of ethics.[1] "a. An Engineer in private practice will not review the work of another engineer for the same client, except with the knowledge of such engineer, or unless the connection of such engineer with the work has been terminated." [1]http://ethics.iit.edu/ecodes/node/4099 http://ethics.iit.edu/ecodes/node/4099
  - nickloewen 8y ago
    This is different though, because of the “except with knowledge of said engineer” line. This code is trying to prevent clients from, for example, getting a “second opinion” on a project if they are unhappy that the first engineer deemed it unsafe. This isn’t really an issue in teaching. I think that it would be impolite to just drop in on a colleagues class unannounced, especially if it is a bit lecture where you might go unnoticed, but I don’t see why a prearranged visit would be unprofessional per se. (Perhaps because it would cause students to doubt the quality of their instructor?) FWIW my understanding is that at my university the response to poor evals is a visit from a colleague, to assess whether the teacher really is weak or if the material is just difficult. Depending on that, the instructor may get additional coaching in teaching technique. This seems like a sensible approach to me.

sonnyblarney 8y ago

In my experience this has been good for weeding out really bad profs. At my Grad school they turfed a prof 1/2 way through that was bones bad. That said - it's perverse at the other end and I do believe that there is unconscious sexism. We had a really smart bunch of people, and some of the female teachers were a little more apprehensive in front of us which I think signals to people subconsciously. Another way: the feedback has to be interpreted. I suggest that the profs should not be 'graded' by students - rather there should just be an open ended opportunity for feedback.

enriquto 8y ago

As a teacher, I disagree strongly with the conclusions of this article. Anonymous student evaluations are an invaluable tool to improve my teaching. The students say so many fair and useful things!

kiriakasis 8y ago
I think the problem is when student evaluation have an affect (positive or negative) on the teacher career.
- enriquto 8y ago
  Yes, when that measure becomes a target it may have some unintended consequences. But the device is too useful to be discarded for that reason.
  - narag 8y ago
    The teachers can do it voluntarily and for their own eyes, without making it an evaluation form.

danmaz74 8y ago

There could be sexism at work or not, but how can you even define "identical" teachers? I never saw two teachers that were identical (even if they had the same gender, ethnicity, age and sexual orientation).

DanielBMarkham 8y ago

This is the age-old doctor question. You go to the doctor's office because you have a medical problem. Doctor 1 comes in on-time, she's smart, capable, friendly, and spends a good amount of time concerned about you and your situation. Doctor 2 comes in late, she's irascible, pedantic, hurried, and only listens long enough to tell you what to do. She also insults you and your family in various ways and doesn't seem to care. After the visit, you go to Yelp. Which one was the better doctor? You have no idea! But do you think that's going to stop somebody from forming and posting an opinion anyway? You can make the argument that these other qualities are good to have in a doctor. I agree. I want a doctor like the first one. But people go to doctors first and foremost to get better, not to make new social acquaintances. If the first doc is incompetent and hurting people, and the second doc a genius, and her attitude is actually causing more people to comply with what's best for them out of a sense anger at her style, perhaps? That's the thing. Who the hell cares if the outside looks cool/sexy/friendly or not? So when we ask for reviews from students who have never worked in the field they're studying for, they're like that Yelp doctor reviewer. I don't see how this situation is good for anybody. In fact, it'll probably lead to a bunch friendly, good-looking morons with great people skills entertaining bunch of kids who aren't learning anything. That's the only natural consequence here.

pas 8y ago
It's not like there can't be an accurate way to measure this. Pre-register patient problems, and see how fast doctors solve problems on average/median. Then it doesn't matter how jovial/amicable the doc is. With enough data you can fish out potential signals (and then test them properly), maybe bedside manner really doesn't matter, maybe being on time is more important. (Because then people spend less time in the clinic among other sick people.) And as others mentioned the teacher eval problem can be solved by splitting the lecturer, exam writer and grader roles. (Grading should be done by a book written as edge cases accumulate, edge cases should be handled without names to prevent bias, as much as possible.) And then aggregate stats can be published about each class and school.
- DanielBMarkham 8y ago
  Sorry no, it doesn't work like that. In either case. Doctors, like professors, are evaluated by former patients based on a mix of factors. In your doctor example, doctors don't "solve problems". They see patients. Many times these patients present with conflicting and hard-to-resolve symptoms. I imagine with the internet this has gotten worse. They use a mix of Occam's Razor, deductive reasoning, and social psychology for the patient's long-term health. You don't want a doctor that finds the answers to complex problems and gives out morphine. You want a doctor that over decades has patients with the best outcomes. In my example, the one doctor may have ticked you off so much you decide never to go back -- and quit smoking just to prove her wrong. That's a win for both you and the doc, although there's no problem being solved. You can speculate various ways to go at this. Perhaps the doctor is evaluated by other doctors working at that location. Or perhaps it's a matrix of various things. As long as you're just pulling stuff out of the air, you can invent any kind of system you'd like. And tech will deliver it for you. Human interventions are messy affairs, and thank goodness for that. We are not robots. Your proposal might do a great job of sorting out professors who teach students well enough to take the test, but heck if I see where that is the end of the analysis. That's just the beginning. We're just getting started talking about professors who can lead a class to mostly pass a test. That should be a baseline for any professor. This is the problem we see in software development. You can come up with all sorts of metrics to stand in place of evaluating whether a team/person is good or not, but whatever you come up with mostly seem to work -- and has very bad effects on those being measured. (With the possible exception of business-value-loop feedback time) I don't think this is tractable in the way you seem to want it. If I were picking a college or professor, I might look at five or six variables and weigh them in various ways depending on my personal needs and goals at the time -- which might change tomorrow. I might want to go interview them. Meet some current and former students. There is a severe and onerous "platform danger" in tech. It starts with the conceit that we can build platforms for anything. It ends up with people not-as-smart-as-they-used-to-be having Google tell them where to go for dinner. Or Wally in Dilbert coding himself up a minivan. [0] I'm not saying the metric definitions and accumulations aren't interesting. They deserve to be considered. I'm saying that by reducing human judgment to single vector, it makes us as a species dumber. That's one of those intuitive things that sound good but are in reality quite bad. 0. https://dilbert.com/strip/1995-11-13 https://dilbert.com/strip/1995-11-13

nocobot 8y ago

I'm a CS student in Germany and contrary to the authors claim we actually do evaluate teachers (to be fair that might not have been common place when she was teaching). This is obviously anecdotal but in my experience these ratings seem to reflect my perceived quality of the lectures quite well.

xtrapolate 8y ago

> "A teacher (also called a school teacher or, in some contexts, an educator) is a person who helps others to acquire knowledge, competences or values." The target audience is a crucial part of the feedback loop. Removing the students from the equation sounds counter intuitive. What are the proposed alternative metrics? Who delivers the feedback? And what is the feedback based on, if not on the direct opinions of the people on the receiving end of the service?

JacKTrocinskI 8y ago
I think the main idea here is that student evaluations only make sense if we assume that students are actually interested in acquiring knowledge, competences, or values, however if their main aim in a course is a good grade then their evaluations suddenly hold little value since they will rate "easy" instructors highest.
- dual_basis 8y ago
  When I was a college instructor I found that when I tried hard - putting a lot of thought into pedagogy, having weekly (open notes) quizzes, assigning challenging but fair homework - I got terrible evaluations, even though the average grade in my class was higher than other teachers who were teaching the same course. When I phoned it in, didn't really try, was very lax with the homework and often made the quizzes take-home, I got excellent reviews but the average grade in the class was worse.
  - thatcat 8y ago
    Did your reviews not ask for justification of the ratings?
    - dual_basis 8y ago
      There are a bunch of ratings 1-10, and then a place to optionally add comments. It's hard to consolidate information from 200+ comments, but, then again, most students do not fill out the comment section anyway. (In a sense, I guess the whole thing is optional - after all, it is anonymous, and we don't check that every student has filled the whole thing out. It also switched from in-class to an online form during the time I was teaching, and the response rate went down quite a bit as a result, but this was after the situation I mentioned above.)

skywhopper 8y ago

This strikes me as a very whiny article, not worthy of publishing in the Chronicle. A lifetime of hurt feelings and resentments bubbling up into a rant full of straw men and frankly irresponsible misinformation about the role student evaluations play in instructor assessment. It’s too bad, too, because there certainly are criticisms to be made, and this article hints at some of them. But a well designed evaluation instrument along with critical interpretation of the results and comments can be an invaluable tool for instructors who want to improve their teaching. But unfortunately this article also spreads untruths about the weight given to student evaluations, and dismisses common-sense security protocols like having a student deliver the results instead of the instructor (not perfect of course, but the student has much less incentive to tamper) as demeaning rather than a sensible precaution. If the author had presented any evidence whatsoever to support her claims linking student evals to grade inflation (possibly related, and I’m sure studies have been done, but evals are very far from the only obvious contributing factor), or even made a passing attempt to explain what percentage of weight is given to raw eval scores for actual tenure considerations (very little in the broad scheme of things), and cut the junk out about how universities seeing students as customers is a new thing (it’s not new at all as any skim of the history of universities as an institution will make clear, but someone at a US state university should realize that it’s skyrocketing tuition rates, and not whiny students, that give students a bigger sense of entitlement) the article would have been much better.

michaelmior 8y ago
While I think it's true that the article overemphasized the weight placed on student evaluations somewhat, I think this does get to the real problem. There's nothing wrong with students evaluating instructors. The problem comes when this is the only method used to evaluate teaching effectiveness. I'm a newly appointed professor, but I've heard many stories from colleagues in the past about students thanking them for being demanding in their course as what they learned was helpful in the future. Demanding classes tend to have negative impact on student evaluations, but that's obviously not capturing the whole picture.
wahern 8y ago
How does a sense of entitlement result in grade inflation? Well, the obvious way is if you pervasively require students evaluations, students systematically evaluate professors based on the grades they received, and these evaluations are used to make staffing and compensation decisions by the administration, incentivizing strategic grading behavior by professors. Solution: stop the evaluations. Even most entitled students won't make unsolicited complaints. And of the ones who do? As long as student complaints aren't used to rank every professor relative to each other, but rather only to weed out the most egregious instructors, there's significantly less pressure for grade inflation. It's when you provide the administration a regular, systematized, "objective" stream of data points that the temptation to assess professors this way becomes irresistible for the administration.

weliketocode 8y ago

Let me a add a wrinkle to the mix specifically in terms of grade inflation. The value of most coursework and any learnings gleaned from it, has dropped significantly. The best value from entering education institutions is instead from job opportunities after. Unfortunately, these opportunities are often based on grades. If the grades are more important than the coursework, then of course students will optimize for it. Students should choose institutions & teachers based on both quality of opportunities and ease of coursework. Going a step further, teachers that don't understand the changing interests of their students should be reprimanded. At my university, most major recruiting for internships and such happened in one semester. Now, I specifically remember the dichotomy of two of my professors' approach to this. Prof John - If you miss a class/deadline/exam due to recruiting, you get a 0. Prof George - I understand recruiting is happening, reach out to me ASAP if you feel overwhelmed or need a date changed. Now, the coursework has largely been forgotten. But, I will never forget John's idiocy.

sevensor 8y ago
I think you'll come to regret this attitude when you reach your mid-30s with no skills other than the concrete procedural skills you picked up on the job. I use my education constantly, and in ways I wouldn't have anticipated. I often see people doing things the hard way because they don't know there's any other way, or passing on opportunities to do something interesting because they don't realize it's even possible.
- quacked 8y ago
  The comment you're responding to isn't an attitude that the poster has, it's an explanation of how students are responding to changes in incentives. As the total number of college graduates grows, they're competing in job markets that are smaller than the total number of college graduates. This ratio is unbalanced and becoming more unbalanced with every year. The people hiring college graduates are developing stricter and stricter filters in order to decide what body of people they're even going to allow to compete in the job market; the biggest, easiest filter to sort out who's truly competitive for positions is grades, and so it makes sense that students compete to get the best grades. rather than competing to be the most competent. Most grads are leaving college in tremendous debt with the understanding that the longer they go without employment, the less likely it is they'll end up employed. This means that while they're in college, rather than focusing on the material they're learning, they're focusing on whatever will allow them to have the largest advantage when they enter the job market. In order for this trend to reverse, the cost of college needs to be dramatically lessened; I think this could be accomplished by reducing the total percentage of high school graduates that attend college immediately, increasing the prevalence of online schools, and reducing university overhead by cutting services to students and administrative positions.

chuckgreenman 8y ago

Sure, evalutations are flawed, but it appears that universities are unable or unwilling to develop other quality measures. For example, this semester I've had a networking professor that would get half way into explaining something interesting and then state that he didn't care about the topic so he didn't prepare to talk about it and we'd be moving on to something else. I've got an A in that class, but I wouldn't say that I learned much from it due to that attitude. I don't think that the CS department here has any way to tell that something like that is happening without evaluations.

bane 8y ago
Right. I think anybody reading comments in these review sites can see if students are ranking professors well for being "easy" or if they're ranking them well for being great educators. The universities I went to for my undergrad and graduate programs both had an informal rating system for professors and I definitely encountered some of both. One of the most highly rated professors I encountered was also one of the best educators at the school. He had a reputation as being a brutally hard grader. But most of the students who left his classes with some bruises and (what would have been in any other class) a mediocre grade, ached to take more from him because you felt the immense value of his teaching. I had other professors who taught poorly, tolerated cheating, or gave non-sequitur exams on material they never covered and so on. There was absolutely no recourse or way of providing feedback on these professors and even ones who received universally poor reviews at the end of the semester stayed on and even attained tenure in some cases. The only real recourse was for students to rank professors amongst themselves and simply starve out poor professors with lack of registrations. That's the only signal universities seem to respond to.
- chuckgreenman 8y ago
  I got the impression that the author was talking about department run evaluation sheets rather than review sites. Student evaluations shouldn't be seen as an evaluation metric but a smoke detector that indicates a staff member might need some more direction. Teaching is an art, just because someone isn't doing it well doesn't mean they can't do it well. In fact in some cases I imagine they are doing poorly because they would rather be working on their research area. Providing the opportunity to course correct rather than starve out is good.

cyberpanther 8y ago

This is also a problem with developer bootcamps since they get a lot of "sales" from reviews. They can often cater to students to the detriment of the learning experience. At times it is about catering to creature comforts, but can bleed over into lessening the learning experience. And especially in bootcamps since things are so fast paced and often times emotional, the reviews are not objective. Many are too positive or too negative.