7 ms·
On the topic of trusting surprising observations, I remember a quiz at school where almost everyone had the same failure mode. First question: "read all the qu
by ben_w 4y ago
On the topic of trusting surprising observations, I remember a quiz at school where almost everyone had the same failure mode.
First question: "read all the questions before answering"
Questions 2-99: Trivial but interestingly bizarre questions, like "draw a triangle with blue ink"
Question 100: "Now you have read all the questions, answer only questions 1 and 2"
I was very tempted to make the mistake everyone else made, and quite smug when I finished, sitting and watching as everyone else asked for ink and later on ink erasers.
- voz_ 4y agoI always found this kind of "test" infuriating.
- Zondartul 4y agoI just answered all the questions out of spite.
- Nition 4y agoI've noticed that most people don't seem to look at things like this from a meta level. I was given that same quiz at school when I was about 12, and I remember I skipped to the end and read the last question (which reveals the trick) because it was highly suspicious that we were randomly given a quiz with a whole series of questions that began by telling you to read the whole thing. Similarly, often quizzes/surveys/studies have an obvious answer when you include the meta-fact that the question is even in there. For example maybe you're shown an image like this[1] with the question "which glass do you think holds more liquid?" Well, it looks like the tall one, but that means it's probably the short one. But we had a good percent the class going for ages on that "read all questions" quiz. [1]https://i.imgur.com/6xkc813.jpg https://i.imgur.com/6xkc813.jpg
- KyeRussell 4y agoIn my day job, I deal with the development of, and verifying the usefulness of, tests. The lesson here is that most tests are designed poorly. Most people don’t realise the effort that goes into doing it properly. Every item needs to be proven to be useful. Items that either purposely or accidentally ‘trick’ the test taker often prove to not be very useful when you run the stats. If a test ‘breaks the fourth wall’ like this, the item probably won’t stand on its own to be useful.
- ben_w 4y agoAh, that explains something I have been confused by for a while now without really understanding why. I'm learning German, and one of the apps I'm using is Clozemaster, where a sentence is missing a word and I have to fill in the gap. The easy mode is to choose from a list of four, the hard mode is to type in the word. Easy mode I can do fairly reliably without even fully reading the sentences, hard mode I struggle with — while the latter is partly because it only accepts one single specific word (so if the sentence would be valid with another word, e.g. weiß/blau/gold/schwarz, keine Punkte für mich if I pick the wrong correct answer), that alone didn't feel sufficient given the subjective experience of the difference between them. I think, from what you've said, in the easy mode I'm probably picking up on things that aren't as useful as I need.
- Nition 4y agoThank you. I've seen surveys where every question is a 'trick' question! Where the correct answer is always the non-intuitive one. Usually when the surveyor is trying put push an agenda and prove that you're wrong about something, but it just ends up being patronising if you realise the trick.
- danaris 4y agoIn my experience, the actual trick when presented with a pair of glasses like that is to determine whether the answer is "the short one" or "neither, they hold exactly the same".
- Nition 4y agoHaha, very true.
- bsaul 4y agoThere’s a problem with this quiz. why would question 100 have the special privilege of discarding all the other questions ? From a purely linguistic pov, this question 100 isn’t a question, it’s a meta-information about how you should reply to the quiz itself. I hate it when teachers try to be too smart for their own good.
- junon 4y agoIt's a simple case of teaching the importance (according to the teacher) of reading the instructions before starting the work. It's not that deep.
- tremon 4y agoIt's a simple case of willfully hiding important information in the last place anyone will look. Awareness of these tactics is a useful skill to have in low-trust societies, and when used as a teaching tool it will guarantee the outcome to be a low-trust society.
- luma 4y agoThe world is messy and "the rules" aren't often written out for us in black and white. Teaching somebody to deal with uncertainty by way of studying the problem before jumping in without the full picture doesn't seem like it would create a "low trust society" as you claim. I would suggest it creates people who might look before they leap. We have enough people on this planet going through life without ever once thinking about what they are about to do.
- lelandfe 4y agoI had a teacher that did this but his whole MO was teaching us good test taking habits. You speed through the easy questions so you know how much time you have to devote to the hard ones. I.e., if that’s the last place you look, that’s an issue.
- bsaul 4y agoAs a student, i remember very vividly that any potential imprecision in the question of the exam was the source of great confusion, especially since you couldn't ask the examinator for explanation (the test was supposed to be perfect). I'm not being pedantic here, i am really furious against teachers that makes students think about the best way to disambiguate the question instead of thinking about the subject of the exam itself.
- bigbluesax 4y agoI would have answered wrong even if i read all the questions beforehand. Answering 2-99 gives you a score of 98/100, while doing it correctly gets you 3/100.
- phonebucket 4y agoYou can argue that you read the questions first too, so you should give yourself the answer to querstion 1 as correct as well and have 99/100 overall
- Winsaucerer 4y agoOne of my kids had the same sort of quiz/test given, but he and his friend had a concern with it. When I read it, I was inclined to agree with them that it doesn't work the way the writer intends. Presumably, each question contains an instruction to follow. Being instructed to read all questions first doesn't mean you then do what any of the other questions instruct. Why, after reading all other 99 questions first, do you do what question 100 instructs but not what the others instruct?
- MrPatan 4y agoBecause you get in the mind of the test-giver and correctly predict the point they're trying to make and act accordingly. If you can't or won't, you fail. That's what school is about, after all.
- reubenmorais 4y agoCan't let these natural intelligences loose in the world without first aligning them.
- Winsaucerer 4y agoSadly true, which leads to a conundrum when trying to get inside the test-giver's head: did the test-giver notice this intricacy of the question and therefore I should give the clever answer, or did they not realise and therefore I should give the naive answer? In this particular instance, my son and his friend were unable to convince their teacher that the test/quiz was flawed, so the naive answer was the "right" one to win the teacher's approval.
- madsbuch 4y agoSo you interpreted in a way, where you executed question 1, and then question 100? That seems counter-intuitive. Even with the knowledge of the questions, there is a case that your should still execute them in order.
- phonebucket 4y agoI think this test is pretty ambiguous. If you answer all the questions after reading all of them, you could only get only question 100 wrong and get the other 99 correct, so reach a score of 99/100. If you answer only questions 1 and 2, you get the answer to 100 right too, so you could score 3/100. In the latter case, you could score 3/3 too, depending on how the examiner decides to mark it. So which is it?
- ben_w 4y agoThis is part of why I think it's highly relevant to the AI/LLM discussion. Now sure, with the benefit of 25-ish years of hindsight, I think that while my actions were correct in the situation and the point of the exercise was to get the other kids to think more like I already did, the "strictly correct" response — and what we would demand from an AI for safety reasons right before complaining about how weirdly inhuman this is — would be to answer Q1-Q99, then for Q100 write down the answers to Q1/Q2 again (and as Q1 was just "read" that just meant Q2).
- peepeepoopoo3 4y ago> First question: "read all the questions before answering" But that's not a question.