7 ms·
Availability of cookies affects evaluation of teaching (2018)
- tekstar 6y agoThe margin of error in the results is greater than the difference between the groups.
- dr_orpheus 6y agoAre the +/- values the distribution of the data rather than the margin of error. So on average the cookie group was still better. Also I wonder if someone could draw the same conclusion that the environment being taught in (i.e. has cookies) is an important factor along with the actual material being taught.
- pacbard 6y agoI don't know what you mean by margin of error. The results in the paper are a little confusing because they report mean and standard deviation while t-tests use mean and standard error. In Table 3, they report the 95% confidence interval for the treatment condition (i.e., receiving cookies) as 0.67–11.62, so it seems that the difference between the two conditions is significant at the 5% level. Interestingly, the reported p-value here is "just" 0.02 instead of the <0.001 that they include in the abstract.
- laGrenouille 6y agoYes, I was confused about that as well. I think that the results are given as (mean +/- sd) rather than (mean +/- SE), though I do not understand why. Is that common in any field? I have not seen it in the computational social sciences or medicine fields, at least not right next to a p-value.
- azhenley 6y agoAs a professor, I sometimes feel as though teaching is more about customer service than education. It is unfortunate.
- dumbfounder 6y agoThere is a reason we don't just give people books and tell them to learn it themselves. Engaging, likable teachers help people learn. Swaying their sentiment with parlor tricks apparently helps gain favor, but the real question is, does this translate to them learning more?
- srtjstjsj 6y ago> Engaging, likable teachers help people learn. Maybe. Maybe it just gains favor and students don't worry about getting value from learning since they get value from the socializing.
- a_imho 6y agoThere are multiple reasons, but efficiency is far down the list. Providing daycare is by far the most important duty of teachers nowadays.
- shard 6y agoWell education is a performance art, as I'm sure you know. Reading the crowd, encouraging discussion, providing motivation -- these are more about the student and their frames of mind / moods / personalities than about the material itself. It's analogous to the difference between a professionally cooked meal versus a glass of Soylent: both provide nutrients, but the professionally cooked meal is more enjoyable to consume.
- mjburgess 6y agoI agree that this is essential. But the issue here is more about what you're not doing. ie., yes, every lesson must be a performance. but it must also be a challenging one. learning is no a michael-bay movie, it is a stanely-kubric: a wonderful light show, yes -- but challenging, stressful, opaque -- and requiring your active participation. What happens when you use student evals as guides to effectiveness is that learning becomes less challenging. (as evidenced here: http://faculty.econ.ucdavis.edu/faculty/scarrell/profqual2.pdf http://faculty.econ.ucdavis.edu/faculty/scarrell/profqual2.p...) I work in high-end commercial education, where this effect -- to me -- is quite clear. But its less obvious how to deal with it. Customer satisfaction is more important than education, to selling education.
- jimhefferon 6y agoThe best study that I know of on student evaluation of teaching was done at the Air Force Academy, where they assign students to teachers at random. Bottom line is that student performance in later classes is negatively correlated with student evaluation of teaching. http://faculty.econ.ucdavis.edu/faculty/scarrell/profqual2.pdf http://faculty.econ.ucdavis.edu/faculty/scarrell/profqual2.p...
- analog31 6y agoThere might be a confounding factor here. For a variety of reasons, I know a lot of academics. It's pretty widespread for teachers to prefer an early schedule for a variety of reasons. One is simply to help them organize their time better. Another is a rumor that earlier classes get better students. In any event it many mean that the teachers with better seniority get first crack at the better schedules. I've known teachers who bring cookies on evaluation day. Another rule of thumb is that evaluations tend to favor teachers who give out easy grades. The evaluations that my spouse received contained threats of violence.
- jimhefferon 6y agoThere are thousands of potential confounding factors. But this study seems to me to address many of the obvious ones, and provides reasonable data suggesting that, while there may be good ways to assess teaching, student evaluations are not one of them. > I've known ... That is common. I had a friend who left a picture of their family on the desk. I am sorry to hear about your spouse's evals. That stinks.
- karatinversion 6y ago> Bottom line is that student performance in later classes is negatively correlated with student evaluation of teaching. I at least read “later” here as “subsequent, following” classes, not classes which take place later in the day.
- HarryHirsch 6y agoThe later the class is in the day the lousier the students and the less inclined to learn. Can't expect a half-literate teenager to give you good ratings when the class is about writing coherent English. That's an observable fact. But the abstract in the paper is talking about follow-on classes, not a section of the same course that's given on a different hour of the day.
- snicker7 6y ago"Third-year medical students"
- terse_malvolio 6y agoAnd n=118 A better survey would measure sleep, nutrition, and exercise levels of students across many campuses against performance of students. It would also be a great thing to fund research of since ideally you'd want optimal schools.
- vmception 6y agoA similar study showed more favorable trial court results for the defendant in the hour after the judge’s lunch time
- throwaway2245 6y agoConclusion: "The provision of chocolate cookies had a significant effect on course evaluation. These findings question the validity of SETs and their use in making widespread decisions within a faculty." I disagree with this conclusion and note that the cookies were provided during the course, not during the evaluation. It seems obvious to me (and should be to anyone who has been in a class) that a course with cookies is a better experience that may indeed result in more knowledge transfer. So this would validate the SET.
- JackC 6y agoI agree. When I teach intro computer programming to law students I often provide snacks, because I think it makes the class more effective.
- srtjstjsj 6y agoDid you objectively test effectiveness?
- pacamara619 6y ago> I THINK it makes the class more effective.
- srtjstjsj 6y agoWhy would you assume cookies cause learning causes high ratings instead of occam's razor cookies cause high ratings?
- threatofrain 6y agoThe presence of cookies is known to affect the perception of time in a way that interacts with your BMI.
- spunker540 6y agoOr it affects perception of time such that time moves slower for the group with cookies. This gives them more time to chew on every word of the lecture and absorb more digestible facts.
- ggm 6y agoSubjectively, the presence of food or drink implies a concern for your physiological wellbeing. Since it was not a defined contractual element of the exchange, it also represents a bonus, an above-and-beyond. It would be interesting to have tested this during the time of fidget spinners or some other tchotchke, to remove the food element but that aside, I think a weakly confirming result would be confirming what we expect: signs of care and emotional engagement beyond the immediately neccessary improve mood, and so improve some outcomes. There is a saying that positive reinforcement beats negative reinforcement but it comes with an implicit corollary: negative reinforcement beats nothing: That negative pupil teacher interaction reports exist with higher scores should not be surprising, because it is possible SOME people achieved higher scores, despite hating (and negatively rating) their teacher.
- srtjstjsj 6y ago> corollary: negative reinforcement beats nothing That's a separate claim not a corollary. In fact, all else unknown, positive beats negative is suggestive that negative is worse than neutral, which might be better than positive.
- ggm 6y agogood point. I vote to be in the cookie set, not the electric jolt set testing this one.
- polm23 6y agoIt seems the full paper makes it clearer that 1. cookies were only provided in one session 2. there was no effect on grades / learning outcomes despite a change in evaluations https://twitter.com/emollick/status/1318737313497812996 https://twitter.com/emollick/status/1318737313497812996
- 1123581321 6y agoI used to work in retail banking, and part of our turnaround strategy for poorly performing branches, or newly acquired branches, was to increase the budget for cookies and hot drinks in the lobby. People really like them. Interestingly, we had to fight the operations department over this as they felt the snacks were interfering with their ability to monitor a more pure banking operation, perhaps similarly to evaluation administrators in a school setting.
- mjburgess 6y agoD'you think they were correct?
- 1123581321 6y agoI think the cookies were worth it. We funded them out of the marketing budget.
- contravariant 6y agoThis must be why websites constantly point out the availability of cookies. Though I'd be surprised if the correlation was the same way round there.
- userbinator 6y agoThis being HN, my mind immediately jumped to browser cookies and I thought "yes, if your course site has a cookie popup, that sure will affect the evaluation of teaching"... and that is actually something I've seen before on a course website, fortunately come across while searching the web and not while I was taking the course.
- scotty79 6y agoWeight of the notepad interviewer caries influences his/her estimation of how appropriate is given candidate for the job. Does this finding "question validity of" recruitment process? Or does it just (re)discover that recruitment process is imprecise as any other process that is based on human judgement?
- mjburgess 6y agoDepends on the effect size. The effect size here, 0.68, is pretty insane for a trivial change of this kind. (if accurate) I think it is pretty clear, from independent research, that student evals capture their eval-time happiness and little else. This is just another piece of evidence to that effect. An eval of this kind cannot really capture anything that useful. Esp. at a university, it would be better to ask people to eval their 101 prof.s at the end of their 201 courses -- ie., with sufficient distance to appreciate their impact on their learning.
- deleted 6y ago[deleted]
- sinab 6y agoNote the use of chocolate cookies; I suspect these results would not hold if oatmeal raisin cookies were used instead.
- tialaramex 6y agoI like a good oatmeal raisin cookie. The place where I buy groceries sells pretty good ones, I'd guess if I'm picking cookies I'll choose oatmeal raisin over chocolate chip about half the time depending on my mood.
- NeutronStar 6y agoI'm eating an oatmeal raisin cookie as I read your comment. Fear not that you are alone in think oatmeal raisin cookies are equivalent, if not better, than chocolate chip cookies.
- HarryHirsch 6y agoThey started with that student evaluation crap in Germany as well? Country's going to pot.
- asciimike 6y agoAnecdata: I taught a class on product management at my alma mater last year and thought it would be nice to bring my students snacks (cookies, fruit snacks, occasional grocery store baked goods, etc.). Sure as hell, I (first time teacher) got the highest average rating across the department. Definitely lots of confounding variables (for example, I was doing it for fun for a semester rather than for a career), but definitely something to be said for bribery...
- Animats 6y agoDoubletree Hotels gets this. They use cookies as food rewards. If you complain about something, you get a cookie.