6 ms·
Exam Credit for Knowing What You Do Not Know
- gnicholas 10y agoIn an economics class on Games and Strategies, the professor gave us a choice for one question: you can accept an 80% score for this question, or you can go to the front of the room and pick up the prompt. Once you have decided to do so you will get whatever score you deserve — there's no turning back and declining to answer the question after seeing it. I believe we were given the topic of the question in the instructions. IIRC, the class split about 50/50.
- epalm 10y agoI was a Computer Science student and teaching assistant at the University of Toronto for a number of years (about a decade ago), and the 20% rule was in full effect: "On a test, if you write 'I don't know', you get 20% of the marks for that question" As a student, this challenged you to know and understand what you know, and what you don't, and promoted the idea that knowing what you don't know is, in itself, valuable. As a teaching assistant marking tests, this virtually eliminated long, tedious, often vacuous answers where the student was clearly throwing darts and hoping for a mark or two.
- silverphoenix 10y agoI recall there was another professor at UofT during my time there that used a scheme of "Everyone starts with 30% on each question, every relevant and correct statement you write earns you marks, every irrelevant or incorrect statement loses you marks."
- derefr 10y agoIt'd be interesting to use that approach without having each question need to be limited by asymptotically approaching a 100% per-question score. You could do so well on a question (proving things the teacher didn't expect the students to have learned at that level) that it would help you out with your scores on other questions. Probably I'd just let the questions be scored with integers, start everyone off at 3, +1 for good points, -1 for bad points, and then take the Euclidean distance of all their question-scores to find out the total "length" of the reach the student has into good-answer space.
- neduma 10y agohttps://www.youtube.com/watch?v=GiPe1OiKQuk https://www.youtube.com/watch?v=GiPe1OiKQuk
- dnautics 10y ago> One problem in teaching economics--the course is on the application of economics to law, for law students--is that it is possible to think you understand something when you don't, when you are substituting some vague things you think you know already for much more precise and quite different things you are supposed to learn. Isn't this generally the problem with economics writ large? It's possible to think you understand something when you don't, when you are substituting some vague math you can do / can measure for the more detailed and difficult (and possibly intractable) phenomenology that is reality?
- Sniffnoy 10y agoNote that for multiple choice exams you can go further and have people assign a probability distribution over the answers, rather than just picking an answer. Then you can use a proper scoring rule: https://en.wikipedia.org/wiki/Scoring_rule#Proper_scoring_rules https://en.wikipedia.org/wiki/Scoring_rule#Proper_scoring_ru...
- gnicholas 10y agoWhen I was studying for the LSAT, I found it very helpful to mark a confidence indicator of 1-4 on every answer. Then, when I went back and graded my answers, I could see if I was missing questions that I knew I wasn't sure of, or if I was overconfident/sloppy on questions that I thought I knew. Not quite the same concept, but in the same vein of "know thyself".
- OisinMoran 10y agoThis is a great idea! I think there should definitely be more emphasis placed on knowing what you don't know in a good education system. Even something like knowing what areas of study exist can be quite valuable. I had a friend acting as a research assistant to a biology masters student who was tasked with removing duplicates from an Excel file. The masters student said it would take about four hours because that's how long it usually took her to do it. My friend made a simple script in about an hour that then did the work in effectively no time. I am by no means a proponent of the "everyone should learn to code" movement but I think there could have been a lot of time saved here if the biologist just new the shallowest fact that computers are good for doing tedious things. Obviously specialization is still very important but I feel introducing more shallow learning could really make a difference in both communication and innovation. If you like this general topic Terence Tao has an interesting piece in a similar vein on assigning partial credit for true-false questions [1] that was posted on HN a while back [2]. [1] https://terrytao.wordpress.com/2016/06/01/how-to-assign-partial-credit-on-an-exam-of-true-false-questions/ https://terrytao.wordpress.com/2016/06/01/how-to-assign-part... [2] https://news.ycombinator.com/item?id=11821903 https://news.ycombinator.com/item?id=11821903
- closed 10y agoI had a similar situation, where a colleague's research assistant estimated it would take her 300 hours to process some data. 15 mins of python later and I landed coauthorship on their paper! I've run into similar situations where people contributed < an hour of expertise that ended up saving weeks of frustration. (and I've probably squandered a lot of time too.. )
- nitrogen 10y agoThis reminds me of a claim made by a former spy on a cable comedy news show that Wikileaks must be run by an intelligence agency because their cable and email dumps in reaction to news events would have required 300 analysts working 24 hours in shifts. But in reality, a bit of 30-40 year old technology like grep, used by a handful of people, could accomplish the same thing.
- icebraining 10y agoThat sounds a little uncharitable to the student. I know that cars are good for carrying stuff, yet I don't rent one to carry my groceries for three miles from the store to my home. Finding a programmer willing to do it for free might alone take her more than four hours.
- andrewclunn 10y agoI'm upvoting all the comments people didn't make.
- pirocks 10y agoThis is already done by certain tests like the AMC and the old SAT.
- TillE 10y agoThis used to be the case on AP exams too, prior to 2011 according to Google. 0 points for a blank answer, but -0.25 for a wrong answer.
- tedmiston 10y agoI had several CS courses where writing something to the effect of "I don't quite know the answer this question (or don't have time to solve it), but the formulas / algorithms / bits of info I think are relevant are: x, y, z." On a difficult question this might get half credit.
- Asooka 10y agoI've seen it implemented in one regional mathematic competition. It was a multiple-choice test, where leaving an answer blank netted you 3 points, while answering wrong gave you 0 points.
- mikecsh 10y agoAs an electronic engineering student, our exams were negatively marked. If you got an answer wrong you incurred (-0.25 * $availablePoints). If you didn't answer the question you scored 0. We despised this at the time.. Now a medical student, we don't have negative marking. I feel it's actually very important in this discipline to know your limitations and not pass-off as though you know more than you do. I have however encountered resistance to this, having professors question my very reasons for doing medicine due to leaving questions unanswered..!
- datburg 10y agoDo not worry. We know the ocean of knowledge they throw/threw us in with full expectation of us drinking in all that new material while being surrounded with sharks and there is a heavy storm that keeps a person from taking a breath. Still Medical school kicks ass and life after it is colorless. Also, why are you complaining? With 4 Subjects tested every week year 2, 40 genes implicated in ovarian cancer and their frequencies are over the top. The only time I was disappointed was when a concept slipped because of the lack of waling minutes to master. Concepts are more important now as I see it in an age when any symptom or fact that slips can be double checked. Best years of my life!
- mikecsh 10y agoI wasn't complaining, but thanks for this!
- savanaly 10y agoIf you're interested in the topic of making students divulge what their confidence in their answer is, you might also like this classic post by Terence Tao. [0] It explores how you might handle and grade a true/false test where for each question the student has to fill in their best estimate of the probability they got the question right. [0] https://terrytao.wordpress.com/2016/06/01/how-to-assign-partial-credit-on-an-exam-of-true-false-questions/ https://terrytao.wordpress.com/2016/06/01/how-to-assign-part...
- whack 10y agoThere was a Decision Analysis class at Stanford that took this idea to an extreme. For each multiple choice option, you had to indicate the percentage confidence of that option being correct. The percentages for all 4 options had to add up to 100%. The score you got for each question = 1 + ln(confidence-assigned-for-correct-answer). So if you were completely sure and gave the correct answer 100%, you would get a score of 1. If you gave it 90%, you would get a slightly lower score. If you "guessed wrong" and gave the correct answer 0% probability, ln0 is negative infinity, so you would instantly fail the entire assignment. It was a pain, but it really made you reassess how well you know stuff, rewarded people who truly knew their stuff, and burnt anyone whose confidence exceeded their actual abilities.
- jacoblambda 10y agoI have heard of this before and while it horrifies me as a student, I would love to take a class with a policy like this.
- nickkoul 10y agoIn reality even in your most confident answers couldn't you put 97% on the one you believe is correct and 1% on all others ensuring that you will not receive an ln(0).
- Cyph0n 10y agoIt actually depends on how they take the confidence value. If it's percentage, I agree with you. But if it's in numerical form - i.e., 0.01 - you'd still lose that question and a few others.
- Cyph0n 10y agoWhy would anyone ever put 0% then? I would just redefine "impossible" to be an arbitrary percentage that guarantees I don't nuke the question. Quick question: is confidence taken in percent form (ln(90)) or decimal form (ln(0.9))? Edit: I am aware of the implication of having percent values, guys/gals. But if it's in decimal form, a 5% confidence level will end up losing you the current question AND three others.
- wrsh07 10y agoMy favorite thing about this is something I heard from Dr Bender, Stony Brook CS: if you get <20%, there's no way to argue you should have passed when you failed, because a student who didn't take the class would do better
- grecy 10y agoSomewhat related, during my studies I got into some monumental arguments over the "using your answer from the previous question" rubbish. Let's say I had no idea how to do question #10, but needed something to start with for #11. I would answer #10, and come up with an answer that made #11 as easy as possible. (I don't remember, something like x^2 instead of some integral junk). Then I would complete #11 using x^2 as input. Genius, I still think. My professors, not so much.
- mdholloway 10y agoI was hoping this would look something more like, given a scenario and a question about it, being able to clearly identify the pieces of information without which the question is unanswerable. That said, time saving and BS reduction are also valuable goals!
- redsn0w422 10y agoAt UIUC, Professor Erickson has a policy that rewards students with 25% credit for indicating they don't know the answer to a particular question, both on homework problems and exams. [1] https://courses.engr.illinois.edu/cs473/sp2017/exam-policy.html#other https://courses.engr.illinois.edu/cs473/sp2017/exam-policy.h...
- mathattack 10y agoSide note - if the article sounds uber-rational econ, it may be in part because the author is the son of Milton Friedman. Here is his first post on the topic. http://daviddfriedman.blogspot.com/2012/11/the-use-of-old-exams.html http://daviddfriedman.blogspot.com/2012/11/the-use-of-old-ex... There is a great quote at the end. And, for a last comment ... . I like to say that being a professor is better than working for a living, except when grading exams. One reason is that grading exams is a pain. Another is that it is when you find out that you have not done nearly as good a job of teaching as you thought you had.
- _yosefk 10y agoWhile it's hard to argue that it runs in the family to some extent in this case, David Friedman is an econ thinker in his own right and certainly goes way further into what most would call extreme libertarian/"uber-rational econ" territory than Milton Friedman ever did. In fact, I'd say that "The Machinery of Freedom" ought to entitle David Friedman to the simpler introduction "...because the author is David Friedman."
- gpawl 10y agoHis most famous book his named after his father's most famous book. His father clearly sets the context for his work.
- gpawl 10y agoThe author is apparently unaware that his "new" idea was standard practice for statistically-validates tests (like the SAT) for decades?