9 ms·

Student Course Evaluations Get an 'F'

39 points by chrismealy 12y ago

pfooti 12y ago
At the end of the semester, students who enjoyed the course should just tip their professors. Adding a real free-market incentive would increase class size and teaching quality, right? Let's let the invisible hand of the market decide; it is certainly more motivating than course evaluations. I have taught at two higher-ed institutions. In one, each student was given a 20-question evaluation. Of those twenty questions, two were of any importance to the instructor: overall instructor and overall course. The other eighteen were frequently ignored by administration and had absolutely zero influence on tenure, promotion, raises or retention. Those two examined scores were averaged over all course over the entire year, but not weighted - you'd have a six-person course count as much toward your final average as a sixty-person course. The scores were from 1 (bad) to 5 (good). You needed an overall average of 3 on each scale to avoid being fired. Higher scores yielded no tangible (or intangible) rewards. Every faculty member taught enough grad-level courses (PhD in particular tend to yield all 5s) that nobody ever really worried about their evaluations from an enforcement perspective. Those of us who cared about actually improving our practice had better ways to get honest reflections from their students. Personally, I tended to get low-ish evaluations from students expecting an easy A ("Professor __ expects his students to work really hard and learn" "He thinks he's the King of Science" etc etc), and high evaluations from students who wanted to learn the stuff the course was about. Of course, I also got better over time and so did my course evaluations. I also got better at gaming the system. Add too much reading to the syllabus, and remove an assignment or two so you can be "responsive to student needs", for example. At the other institution, course evaluations were ignored entirely. They played no role whatsoever in anything. On the other hand, I had a better time teaching those students, and according to my evaluations that had no impact on my career, I was doing a better job of it to boot. Of course, anecdotes vs data, etc etc. Still, I think tips are the way to go. We already know mean course evaluations can be predicted by the grades students expect to get [1], so let's just flip that around a little. [1]: https://stat.duke.edu/~dalene/chance/chanceweb/153.johnson.pdf https://stat.duke.edu/~dalene/chance/chanceweb/153.johnson.p...
- naner 12y ago
  At the end of the semester, students who enjoyed the course should just tip their professors. Adding a real free-market incentive would increase class size and teaching quality, right? Let's let the invisible hand of the market decide First of all, broke college students certainly aren't going to 'tip' their professors. I guess perhaps you're thinking part of tuition will be used for this? Still, what students 'enjoy' is not necessarily what is most beneficial for them in a college course. I think you're setting up the incentives wrong.
  - userbinator 12y ago
    Still, what students 'enjoy' is not necessarily what is most beneficial for them in a college course. I think you're setting up the incentives wrong. Exactly. The majority of them just want the highest grades with the least effort, and the professors who will quite naturally want to get tipped more, will optimise toward that. It'll create a situation where teaching next-to-nothing and giving everyone perfect scores gets the highest rewards; i.e. not much of an education.
    - WalterBright 12y ago
      That may be true for some, but not for me. Not only was I paying most of the tuition bill, I selected classes based on what I though was knowledge I needed for the career I wanted. An "easy A" didn't figure into it. I wasn't interested in paying lots of money for nothing. I also wasn't interested in attending a university known for easy grading.
      - watwut 12y ago
        Quoting article: "There's an intriguing exception to the pattern: Classes full of highly skilled students do give highly skilled teachers high marks. Perhaps the smartest kids do see the benefit of being pushed." Maybe you are in that "highly skilled" group. Unfortunately, average student have different behavior.
Redoubts 12y ago
And of course, those with legitimate complaints with the Prof simply drop the class a few days or weeks in, and never get a chance to fill out an evaluation...
deleted 12y ago
[deleted]
bradleyjg 12y ago
In deteriming whether course evaluations are corrlelated with teacher quality, they compared them to grades received in a follow on course, and found that they had a negative correlation. So far so good. But if grades received in follow on courses are the gold standard for teacher effectiveness, then why not use that, instead of the other proxies the article goes on to recommend (peer evaluations primarily)? Certainly it won't be possible in every case as not every class has a follow on, but at the very least they should subject their new proposed methods to the same analysis.
- Alex3917 12y ago
  > if grades received in follow on courses are the gold standard for teacher effectiveness, then why not use that Because grades received in later courses isn't actually a meaningful metric. There was a theory that you could measure teacher quality this way using something called Value Added Metrics (VAM), but it's now been completely debunked. C.f. http://www.epi.org/publication/bp278/ http://www.epi.org/publication/bp278/ http://www.nber.org/papers/w14442 http://www.nber.org/papers/w14442 http://annenberginstitute.org/sites/default/files/product/211/files/valueAddedReport.pdf http://annenberginstitute.org/sites/default/files/product/21... http://ies.ed.gov/ncee/pubs/20104004/pdf/20104004.pdf http://ies.ed.gov/ncee/pubs/20104004/pdf/20104004.pdf http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1142237 http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1142237 http://educationnext.org/files/ednext20022_10.pdf http://educationnext.org/files/ednext20022_10.pdf
  - yummyfajitas 12y ago
    Your links do not remotely come close to debunking the concept of VAM. Your first (accurately) says "measurements are noisy" and the inaccurately says "and other things matter for learning too". Unless I missed something, it doesn't provide a reason to expect bias in the other things. The second link says that some VAM models assume student assignment is random when it is not, and also that measured educational outputs don't matter much for outcomes anyway. The logical conclusion of this is simply that VAM needs modification, and maybe we should reduce education. Claiming this debunks the idea of VAM is like saying "hah, this paper proves OLS doesn't always work well, therefore regression is debunked". The third is really long and I couldn't figure out it's point. The fourth one says VAM is noisy. If you believe VAM is unable to measure the effects of teachers, are you willing to follow through with the logical conclusions? Namely, the effect teachers have on students is so small it can't be measured and we should therefore focus on other things (e.g. reducing cost) instead?
tokenadult 12y ago
The methodology used in the study from Italy[1] mentioned in the article was quite interesting, because it had actual random assignment of comparable groups of students to different instructors. The students were then followed up over their further study in the same university. That's a good kind of data set to have for examining issues like this. The alternative means of evaluating teachers mentioned in the article are also quite reasonable. Both peer review and content analysis of instructor-prepared materials can identify better teachers with different sources of bias in the evaluation, allowing a triangulation with student ratings. Here on Hacker News, from another participant, I learned about a more rigorous method of student ratings of teachers[2] that ought to be applied (with appropriate adaptations) to higher education teaching. It appears to work will in K-12 teaching. [1] http://www.sciencedirect.com/science/article/pii/S0272775714000417 http://www.sciencedirect.com/science/article/pii/S0272775714... [2] https://news.ycombinator.com/item?id=4559682 https://news.ycombinator.com/item?id=4559682
leoh 12y ago
When I was at Columbia, we used an interesting website called CULPA ("Columbia Undergraduate Listing of Professor Ability"; http://culpa.info/ http://culpa.info/). The site is interesting is that unlike many other sites, students can only give written evaluations and there are no numerical scores. Reviews are generally thoughtful and, in some cases, were particularly useful in choosing a professor for a course.
- ics 12y ago
  Were evaluations mandatory? Did they have any required length? I've done plenty of evaluations in the past which only asked for written responses but was generally disappointed to hear from classmates that they either scribbled gibberish to make it look like they were filling it out or, if the teacher was good, just writing things like "Thanks" or "Great class" in each area.
- Alex3917 12y ago
  Non-numerical evaluations make much more sense. It's funny how most of the good universities in the US started out as being explicitly Christian, but over the last couple centuries the Christianity has been pretty much wholesale replaced with numerology -- student grades, teacher evaluations, impact factors, statistical significance, etc. The entire system is built upon the worship of numbers, and every phenomenon is assigned some sort of numerical value (to of course be aggregated and statistically 'analyzed') even if it's plainly non-quantitative in nature. You've got to wonder whether this is some sort of temporary societal blind spot, or whether this ideology is so toxic and so subtle and complex to refute that it will just be around forever as the endpoint of scientific history.
  - vdaniuk 12y ago
    Humans are prone to generalisation, reducing complex objects or states to several numeric parameters is extremely useful. Full of errors and generalization errors, but nevertheless usefull. Humanity will just have to build more complex models to quantify complex, dynamic objects, such as courses. There is nothing special in human interaction data that inherently prevents its quantification. We just need more data and more efficient algorithms.
JoeAltmaier 12y ago
Poor professors, getting arbitrary ratings from people they meet once or twice a week for a few weeks, which determine their professional future. Sounds like being a student or something. Go suck on a lemon, professors.
SynchrotronZ 12y ago
The way things work at my university, at least the science faculty, is that word of good or bad teachers just travels trough the grapevine. And it is usually quite accurate. Then we have (a active, and paid) representation in the faculties decision making body, which leads to, in my experience, the faculty actually dealing with bad professors. Not everything has to be boxed in by numerics, sometimes simply speaking up and listening is the easiest and best solution.
- blahedo 12y ago
  > And it is usually quite accurate. How do you know? How would you know?
mjhoy 12y ago
Back in college I used to check "rate my professor" when picking classes. I learned pretty quickly to avoid the highly-rated.
yummyfajitas 12y ago
So here is a fun fact about professor performance. When I was a grad student, various calculus classes had a shared final. We had some professors with great evaluations who devoted their life to teaching. We had some Germans and Chinese who students perceived to barely speak English [1] and hated. Student evals were fairly predictable - hard exams would reduce scores, jokes and sympathy would increase them. Everyone's students followed the exact same normal curve. I've been told that the only way one can make an observable difference in group final scores is to schedule a class at the same time as sports practice or remedial education. You need better students, not a better teacher. [1] I've noticed that either students are unable/unwilling to listen to a foreign accent, or perhaps I am unable to notice them. Recently a friend of mine said my secretary had a heavy accent - I barely notice it.
- dsr_ 12y ago
  If that was a true over an adequately large poulation of students, it would indicate that the professors are all equally bad at teaching.
  - revelation 12y ago
    Or they are all equally good, or they all have exactly the same wrinkle on their nose, as the very point of this article is that student evaluations don't measure what we think they do.
- ebiester 12y ago
  First, an 18 year old has less experience deciphering a thick accent than someone with years of experience in IT. I don't blink at accents that would have baffled me a decade ago. Second, if there is cognitive load devoted to understanding the words, that is energy not devoted to understanding the concepts. Friends in those classes in college spent extra time outside of class with tutors and study groups than those with good teachers because they knew the tests didn't change. Third, the lack of variance means the teachers don't make a difference in general. Perhaps there is a greater question to ask there.
- rtpg 12y ago
breen 12y ago
What about taking into account each student's performance in the class? In other words, partition the evaluations into sets based on the students' performance. Then you'd end up with a spectrum of instructor ratings, from F to A, not an average. If the institution wants to make a choice of instructors for an introductory course, they might want to select the instructor with the higher low-grade rating. The intuition here is that well-performing students in an introductory course might have prior knowledge, so their ratings add noise. If the institution wants to compare two instructors overall, they can compare their spectra after subtracting the skew that results from low-performers evaluating poorly and high-performers evaluating highly. What remains is some picture of how the instructor handles students of different strengths and weaknesses, which (I think) is the best measure of a teacher.
- watwut 12y ago
  "If the institution wants to make a choice of instructors for an introductory course, they might want to select the instructor with the higher low-grade rating. The intuition here is that well-performing students in an introductory course might have prior knowledge, so their ratings add noise." The complain in the article is that with exception of well-performing students, students rating does not measure how much they learned (measured be their performance in subsequent courses). Lower performing students tend to give lower marks to those teachers that pushed them to learn more. Your plan maximizes satisfaction of low performing students in introductory course. It does not maximizes how much they will learn, quite the opposite. University should strive for the best teaching, not for the highest immediate satisfaction. It has importance too, but should not be more important then actual learning.
dzink 12y ago
UChicago's rating system was quite useful, actually. The survey is given before the final, so exam difficulty doesn't matter. Professors who deliver insights you really learn from get the high ratings and those who don't (aka: they dwell in detail instead of creating understanding, can't communicate, read slides, make you fall asleep, or teach outdated material lose out). Over the years, professor ratings and course bidding points became a much better indicator of the quality of a course section than the description of a class.
j2kun 12y ago
From all of my experiences and everything every faculty member and career panel has ever told me (about research university positions in mathematics and computer science), teaching does not matter at all. Hiring boards don't look at your teaching statement even though they require one. Nobody reads a letter of recommendation if it is about teaching. Nobody reads your course evaluations. And in particular, the more you neglect teaching (and the more time you spend on research) the easier it will be to get tenure. As sad as it sounds, that's how my field is. So is this report specific to other fields?

39 points by chrismealy 12y ago

At the end of the semester, students who enjoyed the course should just tip their professors. Adding a real free-market incentive would increase class size and teaching quality, right? Let's let the invisible hand of the market decide; it is certainly more motivating than course evaluations. I have taught at two higher-ed institutions. In one, each student was given a 20-question evaluation. Of those twenty questions, two were of any importance to the instructor: overall instructor and overall course. The other eighteen were frequently ignored by administration and had absolutely zero influence on tenure, promotion, raises or retention. Those two examined scores were averaged over all course over the entire year, but not weighted - you'd have a six-person course count as much toward your final average as a sixty-person course. The scores were from 1 (bad) to 5 (good). You needed an overall average of 3 on each scale to avoid being fired. Higher scores yielded no tangible (or intangible) rewards. Every faculty member taught enough grad-level courses (PhD in particular tend to yield all 5s) that nobody ever really worried about their evaluations from an enforcement perspective. Those of us who cared about actually improving our practice had better ways to get honest reflections from their students. Personally, I tended to get low-ish evaluations from students expecting an easy A ("Professor __ expects his students to work really hard and learn" "He thinks he's the King of Science" etc etc), and high evaluations from students who wanted to learn the stuff the course was about. Of course, I also got better over time and so did my course evaluations. I also got better at gaming the system. Add too much reading to the syllabus, and remove an assignment or two so you can be "responsive to student needs", for example. At the other institution, course evaluations were ignored entirely. They played no role whatsoever in anything. On the other hand, I had a better time teaching those students, and according to my evaluations that had no impact on my career, I was doing a better job of it to boot. Of course, anecdotes vs data, etc etc. Still, I think tips are the way to go. We already know mean course evaluations can be predicted by the grades students expect to get [1], so let's just flip that around a little. [1]: https://stat.duke.edu/~dalene/chance/chanceweb/153.johnson.pdf https://stat.duke.edu/~dalene/chance/chanceweb/153.johnson.p...

naner 12y ago
At the end of the semester, students who enjoyed the course should just tip their professors. Adding a real free-market incentive would increase class size and teaching quality, right? Let's let the invisible hand of the market decide First of all, broke college students certainly aren't going to 'tip' their professors. I guess perhaps you're thinking part of tuition will be used for this? Still, what students 'enjoy' is not necessarily what is most beneficial for them in a college course. I think you're setting up the incentives wrong.
- userbinator 12y ago
  Still, what students 'enjoy' is not necessarily what is most beneficial for them in a college course. I think you're setting up the incentives wrong. Exactly. The majority of them just want the highest grades with the least effort, and the professors who will quite naturally want to get tipped more, will optimise toward that. It'll create a situation where teaching next-to-nothing and giving everyone perfect scores gets the highest rewards; i.e. not much of an education.
  - WalterBright 12y ago
    That may be true for some, but not for me. Not only was I paying most of the tuition bill, I selected classes based on what I though was knowledge I needed for the career I wanted. An "easy A" didn't figure into it. I wasn't interested in paying lots of money for nothing. I also wasn't interested in attending a university known for easy grading.
    - watwut 12y ago
      Quoting article: "There's an intriguing exception to the pattern: Classes full of highly skilled students do give highly skilled teachers high marks. Perhaps the smartest kids do see the benefit of being pushed." Maybe you are in that "highly skilled" group. Unfortunately, average student have different behavior.

Redoubts 12y ago

And of course, those with legitimate complaints with the Prof simply drop the class a few days or weeks in, and never get a chance to fill out an evaluation...

deleted 12y ago

[deleted]

bradleyjg 12y ago

In deteriming whether course evaluations are corrlelated with teacher quality, they compared them to grades received in a follow on course, and found that they had a negative correlation. So far so good. But if grades received in follow on courses are the gold standard for teacher effectiveness, then why not use that, instead of the other proxies the article goes on to recommend (peer evaluations primarily)? Certainly it won't be possible in every case as not every class has a follow on, but at the very least they should subject their new proposed methods to the same analysis.

Alex3917 12y ago
> if grades received in follow on courses are the gold standard for teacher effectiveness, then why not use that Because grades received in later courses isn't actually a meaningful metric. There was a theory that you could measure teacher quality this way using something called Value Added Metrics (VAM), but it's now been completely debunked. C.f. http://www.epi.org/publication/bp278/ http://www.epi.org/publication/bp278/ http://www.nber.org/papers/w14442 http://www.nber.org/papers/w14442 http://annenberginstitute.org/sites/default/files/product/211/files/valueAddedReport.pdf http://annenberginstitute.org/sites/default/files/product/21... http://ies.ed.gov/ncee/pubs/20104004/pdf/20104004.pdf http://ies.ed.gov/ncee/pubs/20104004/pdf/20104004.pdf http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1142237 http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1142237 http://educationnext.org/files/ednext20022_10.pdf http://educationnext.org/files/ednext20022_10.pdf
- yummyfajitas 12y ago
  Your links do not remotely come close to debunking the concept of VAM. Your first (accurately) says "measurements are noisy" and the inaccurately says "and other things matter for learning too". Unless I missed something, it doesn't provide a reason to expect bias in the other things. The second link says that some VAM models assume student assignment is random when it is not, and also that measured educational outputs don't matter much for outcomes anyway. The logical conclusion of this is simply that VAM needs modification, and maybe we should reduce education. Claiming this debunks the idea of VAM is like saying "hah, this paper proves OLS doesn't always work well, therefore regression is debunked". The third is really long and I couldn't figure out it's point. The fourth one says VAM is noisy. If you believe VAM is unable to measure the effects of teachers, are you willing to follow through with the logical conclusions? Namely, the effect teachers have on students is so small it can't be measured and we should therefore focus on other things (e.g. reducing cost) instead?

tokenadult 12y ago

The methodology used in the study from Italy[1] mentioned in the article was quite interesting, because it had actual random assignment of comparable groups of students to different instructors. The students were then followed up over their further study in the same university. That's a good kind of data set to have for examining issues like this. The alternative means of evaluating teachers mentioned in the article are also quite reasonable. Both peer review and content analysis of instructor-prepared materials can identify better teachers with different sources of bias in the evaluation, allowing a triangulation with student ratings. Here on Hacker News, from another participant, I learned about a more rigorous method of student ratings of teachers[2] that ought to be applied (with appropriate adaptations) to higher education teaching. It appears to work will in K-12 teaching. [1] http://www.sciencedirect.com/science/article/pii/S0272775714000417 http://www.sciencedirect.com/science/article/pii/S0272775714... [2] https://news.ycombinator.com/item?id=4559682 https://news.ycombinator.com/item?id=4559682

leoh 12y ago

When I was at Columbia, we used an interesting website called CULPA ("Columbia Undergraduate Listing of Professor Ability"; http://culpa.info/ http://culpa.info/). The site is interesting is that unlike many other sites, students can only give written evaluations and there are no numerical scores. Reviews are generally thoughtful and, in some cases, were particularly useful in choosing a professor for a course.

ics 12y ago
Were evaluations mandatory? Did they have any required length? I've done plenty of evaluations in the past which only asked for written responses but was generally disappointed to hear from classmates that they either scribbled gibberish to make it look like they were filling it out or, if the teacher was good, just writing things like "Thanks" or "Great class" in each area.
Alex3917 12y ago
Non-numerical evaluations make much more sense. It's funny how most of the good universities in the US started out as being explicitly Christian, but over the last couple centuries the Christianity has been pretty much wholesale replaced with numerology -- student grades, teacher evaluations, impact factors, statistical significance, etc. The entire system is built upon the worship of numbers, and every phenomenon is assigned some sort of numerical value (to of course be aggregated and statistically 'analyzed') even if it's plainly non-quantitative in nature. You've got to wonder whether this is some sort of temporary societal blind spot, or whether this ideology is so toxic and so subtle and complex to refute that it will just be around forever as the endpoint of scientific history.
- vdaniuk 12y ago
  Humans are prone to generalisation, reducing complex objects or states to several numeric parameters is extremely useful. Full of errors and generalization errors, but nevertheless usefull. Humanity will just have to build more complex models to quantify complex, dynamic objects, such as courses. There is nothing special in human interaction data that inherently prevents its quantification. We just need more data and more efficient algorithms.

JoeAltmaier 12y ago

Poor professors, getting arbitrary ratings from people they meet once or twice a week for a few weeks, which determine their professional future. Sounds like being a student or something. Go suck on a lemon, professors.

SynchrotronZ 12y ago

The way things work at my university, at least the science faculty, is that word of good or bad teachers just travels trough the grapevine. And it is usually quite accurate. Then we have (a active, and paid) representation in the faculties decision making body, which leads to, in my experience, the faculty actually dealing with bad professors. Not everything has to be boxed in by numerics, sometimes simply speaking up and listening is the easiest and best solution.

blahedo 12y ago
> And it is usually quite accurate. How do you know? How would you know?

mjhoy 12y ago

Back in college I used to check "rate my professor" when picking classes. I learned pretty quickly to avoid the highly-rated.

yummyfajitas 12y ago

So here is a fun fact about professor performance. When I was a grad student, various calculus classes had a shared final. We had some professors with great evaluations who devoted their life to teaching. We had some Germans and Chinese who students perceived to barely speak English [1] and hated. Student evals were fairly predictable - hard exams would reduce scores, jokes and sympathy would increase them. Everyone's students followed the exact same normal curve. I've been told that the only way one can make an observable difference in group final scores is to schedule a class at the same time as sports practice or remedial education. You need better students, not a better teacher. [1] I've noticed that either students are unable/unwilling to listen to a foreign accent, or perhaps I am unable to notice them. Recently a friend of mine said my secretary had a heavy accent - I barely notice it.

dsr_ 12y ago
If that was a true over an adequately large poulation of students, it would indicate that the professors are all equally bad at teaching.
- revelation 12y ago
  Or they are all equally good, or they all have exactly the same wrinkle on their nose, as the very point of this article is that student evaluations don't measure what we think they do.
ebiester 12y ago
First, an 18 year old has less experience deciphering a thick accent than someone with years of experience in IT. I don't blink at accents that would have baffled me a decade ago. Second, if there is cognitive load devoted to understanding the words, that is energy not devoted to understanding the concepts. Friends in those classes in college spent extra time outside of class with tutors and study groups than those with good teachers because they knew the tests didn't change. Third, the lack of variance means the teachers don't make a difference in general. Perhaps there is a greater question to ask there.
rtpg 12y ago

breen 12y ago

What about taking into account each student's performance in the class? In other words, partition the evaluations into sets based on the students' performance. Then you'd end up with a spectrum of instructor ratings, from F to A, not an average. If the institution wants to make a choice of instructors for an introductory course, they might want to select the instructor with the higher low-grade rating. The intuition here is that well-performing students in an introductory course might have prior knowledge, so their ratings add noise. If the institution wants to compare two instructors overall, they can compare their spectra after subtracting the skew that results from low-performers evaluating poorly and high-performers evaluating highly. What remains is some picture of how the instructor handles students of different strengths and weaknesses, which (I think) is the best measure of a teacher.

watwut 12y ago
"If the institution wants to make a choice of instructors for an introductory course, they might want to select the instructor with the higher low-grade rating. The intuition here is that well-performing students in an introductory course might have prior knowledge, so their ratings add noise." The complain in the article is that with exception of well-performing students, students rating does not measure how much they learned (measured be their performance in subsequent courses). Lower performing students tend to give lower marks to those teachers that pushed them to learn more. Your plan maximizes satisfaction of low performing students in introductory course. It does not maximizes how much they will learn, quite the opposite. University should strive for the best teaching, not for the highest immediate satisfaction. It has importance too, but should not be more important then actual learning.

dzink 12y ago

UChicago's rating system was quite useful, actually. The survey is given before the final, so exam difficulty doesn't matter. Professors who deliver insights you really learn from get the high ratings and those who don't (aka: they dwell in detail instead of creating understanding, can't communicate, read slides, make you fall asleep, or teach outdated material lose out). Over the years, professor ratings and course bidding points became a much better indicator of the quality of a course section than the description of a class.

j2kun 12y ago

From all of my experiences and everything every faculty member and career panel has ever told me (about research university positions in mathematics and computer science), teaching does not matter at all. Hiring boards don't look at your teaching statement even though they require one. Nobody reads a letter of recommendation if it is about teaching. Nobody reads your course evaluations. And in particular, the more you neglect teaching (and the more time you spend on research) the easier it will be to get tenure. As sad as it sounds, that's how my field is. So is this report specific to other fields?