At one of the keynotes for the Association for Learning Technology back in 2007, the speaker had a variation on this question. His point, if I recall, was a lit

At one of the keynotes for the Association for Learning Technology back in 2007, the speaker had a variation on this question. His point, if I recall, was a little different -- As normally asked, the question is code for "parrot the Archimedes principle back to me". But on a realistic "hot summers day" there's not just the Archimedes principle. There's evaporation that can remove water from the cup. There's wind, and occasional movement of the ice cubes as they melt and slip over each other in the glass, that could cause spillage if it's really that full that it's on the verge of overflowing. As could interruption of the surface tension (e.g., condensation in just the wrong spot at the edge of the glass, causing the water in the meniscus to spill over.) The initial temperature of the water being unstated, its density will also change a very small amount, potentially in either direction. Often when we ask our students scenario questions as teachers, we are essentially asking them to identify which taught principle we have flagged this question as being about, and base their answer on that, ignoring all other factors. If the student answers "A. It goes down", did they fail to grasp the Archimedes principle, or are they a pedant grumbling that over that length of time there will be evaporation so it won't be exactly the same? Take the question out of the context of a course, and suddenly the educational point of it can change. Is the person asking you this a physics teacher checking your understanding of Archimedes principle, or an ALT conference pedant checking whether you understand that in uncontrolled environments where bumps and spillages can occur you can't blindly expect the ideal result you'd get in a textbook? The peril comes when questions are used outside of a teaching context. For example in technical interviews, where the same question can sometimes be a signal for many different teaching points, each expecting subtly different answers, and the interviewee does not know which the interviewer has in mind.