4 ms·

supposed to test for what? Your ability to synthesize knowledge out of thin air. Yes, actually. Synthesis is the penultimate level of Bloom's taxonomy [1]. Per

by chongli 5mo ago

supposed to test for what? Your ability to synthesize knowledge out of thin air. Yes, actually. Synthesis is the penultimate level of Bloom's taxonomy [1]. Perhaps not appropriate for a first quiz on new material, but absolutely in scope for a final exam. I had many exams in university that challenged me to synthesize a proof to a hypothesis I'd never seen before. I had to bring together my knowledge of the material I'd learned in class with the bit of new material presented in the problem statement, and then devise a proof with one or more steps I'd never done before. Many of my fellow students in first year struggled mightily with questions like this. There was much wailing and gnashing of teeth after the exams. In later years we'd all gotten used to it, and it was to be expected. That students in high school or younger years have been sheltered from synthesis-type problems on exams is a great disservice to them. Edit: I should also note that outside of upper-year pure mathematics courses, synthesis-type problems rarely accounted for more than a third of the grade on an exam. [1] https://en.wikipedia.org/wiki/Bloom's_taxonomy https://en.wikipedia.org/wiki/Bloom's_taxonomy

knollimar 5mo ago
Synthesis problems and time pressure don't feel fair together. I'd be furious to get some lofty theoretical synthesis problem on an engineering exam where the other problems are grindy analysis of "looks like concept you know but expand into a form you haven't seen and then will take 1/n time on your n question exam". I suppose these are already 1 extension in, so 1 more logical extension isn't that far off
- chongli 5mo ago
  They're absolutely fair if you're given plenty of practice at them. If the only synthesis problem you see all year is on the final exam then you have a point. However, if you're doing novel (to you) proofs on assignments every week throughout the course, you should be used to the novelty by the time you reach the exam. Expecting a final exam consisting only of calculation and memorization type questions is wholly inappropriate in a mathematics course full of proofs.
  - prerok 5mo ago
    I think you are both right and it really depends on the leap that has to be done during the time constraints of the exam and how much practise was included in the curriculum. Off-topic, I know, but I always disliked the time pressure of exams. All the bad scores I got were down to not feeling quite right at the time of the exam: maybe I didn't sleep well, the food I ate did not quite sit with me, had a fever, etc. None of these things actually impair too badly but if the time is short, there is an impact. I suppose it's similar to sports where you have to, at least somewhat, adapt on the go. Many teams botch a game here and there. But... your final exam? Botch it and you're screwed. It just does not feel right.
  - hashmap 5mo ago
    it depends on what you mean by fair, i think. is the purpose of the test to gauge how well a person performs on tests? the difficult problems i encounter in the real world involve collecting a bunch of context on it, sitting with it for a bit, then putting it down and going for a walk. when i come up with something to try it's usually when i am outside and looking at trees. maybe this is good for weeding out students who would not be a good fit for high stakes speed math.
    - chongli 5mo ago
      It's really not high speed math. You'll have 2.5 hours to solve maybe 5 problems on the exam. A math grad student who TAs the course can typically complete the exam in 45 minutes or less, without any preparation whatsoever. An undergraduate student who can't solve the exam in 2.5 hours (without external aid) is not going to solve it in 10 or even 20 hours. They simply do not know what they're doing at that point. It's really no different from a software dev interviewee who can't solve basic programming questions without being handheld and coached through the entire process.
      - knollimar 5mo ago
        I think if TA no prep completion time to total time ratio is above 2.5 it's not the time pressure based exams I'm referring to. I didn't get those exams in math courses. Engineering courses I did get those where some of the point of the test is the speed of your pattern matching ability (did you use design skills to reason about it or did you brute force a solution). Math you're already extrapolating theory and you need ample time for that.
- tekla 5mo ago
  If you can't do synthesis problems on a exam, you are not prepared for it. I quite fondly remember a situation with a two question three hour open everything exam where I stared at the 1st problem and relevant equation for 20 minutes because I had no idea what I was looking at and was wondering if I should just leave and drop out. Then I realized that a bunch of terms could cancel out and the problem became 3 lines trivial. The professor later admitted he wanted to give a "easy" question that you couldn't just copy out of your notes and had to think and to spend the rest of the exam on the actually difficult one.
  - knollimar 5mo ago
    This seems like you proved my point by staring at it for 20 minutes before starting. I don't think it's fair to grade with extreme time pressure and also expect deep thought. Being able to do deep thought quickly isn't a valuable skill, and you don't pick it up on the way ti mastery with lots of practice like you do the speed for rote problems
    - tekla 5mo ago
      No the literal oppposite. The exam was "no deep thought required. Need to know the subject and its trivial"
- majormajor 5mo ago
  > Synthesis problems and time pressure don't feel fair together. If you really want to grade the students its fair to put very hard problems on the exam to separate out mastery from merely very good understanding or recital.
  - knollimar 5mo ago
    Theres is more than ine dimension to mastery. What skill are you grading? Can you expect some students to be able to churn out the problems? Are they doing design work? Is it more inventive or more creative? Is there more than one solution? How rigorous is the solution? My point is that you can design an exam to test for speed and ability to do the day to day well. You prepare for this exam with lots of practice. There are a few problem types but you might not have seen every one. There's a basic level of extrapolation you're expecting. You can design an exam for abstracting theory, and you prep for this exam by doing the hardest problems you can find, reading more theory, etc. You might not have seen anything like this but the test is to use the tools you're given in a class well. The test is the extrapolation. I had theory courses where I expected the latter and practical courses where I expected the former. There's more than one type of mastery; I bet your math professors would get destroyed by gifted 14 year olds in algebra competitions that involved arithmetic and timing. Do those 14 year olds not have more mastery?
  - jltsiren 5mo ago
    Hard problems introduce noise to the grades. A student who gets a hard problem right could be good or lucky. In order to grade the students properly, you need multiple problems of similar difficulty. If you have both ordinary and difficult problems in the exam, it's probably long enough that it should take the entire day. Undergraduate exams tend to be short. Which means that a perfect grade should be interpreted as "meets the expectations".
    - chongli 5mo ago
      Which means that a perfect grade should be interpreted as "meets the expectations". None of the mathematics exams I wrote during undergrad at Waterloo (as a math major) would fit that description. Nearly every single one of them had midterm grades with unimodal distributions centred below 70%, tending toward 60%. Typically, only 1-5 people in the class (of 100-200) would score a perfect grade. In upper year pure mathematics courses it was common to not have any perfect grades (in a class of about 20-30).
- chermi 5mo ago
  Am I the only one that thinks a test in a technical discipline is poorly designed if more than one person gets a 100%? Exams are supposed to be hard. School is supposed to push you to learn the subject as best as possible.
chermi 5mo ago
I dread the day this collapse of thinking ability hits physics education. Maybe it already has.

by chongli 5mo ago

knollimar 5mo ago

Synthesis problems and time pressure don't feel fair together. I'd be furious to get some lofty theoretical synthesis problem on an engineering exam where the other problems are grindy analysis of "looks like concept you know but expand into a form you haven't seen and then will take 1/n time on your n question exam". I suppose these are already 1 extension in, so 1 more logical extension isn't that far off

chongli 5mo ago
They're absolutely fair if you're given plenty of practice at them. If the only synthesis problem you see all year is on the final exam then you have a point. However, if you're doing novel (to you) proofs on assignments every week throughout the course, you should be used to the novelty by the time you reach the exam. Expecting a final exam consisting only of calculation and memorization type questions is wholly inappropriate in a mathematics course full of proofs.
- prerok 5mo ago
  I think you are both right and it really depends on the leap that has to be done during the time constraints of the exam and how much practise was included in the curriculum. Off-topic, I know, but I always disliked the time pressure of exams. All the bad scores I got were down to not feeling quite right at the time of the exam: maybe I didn't sleep well, the food I ate did not quite sit with me, had a fever, etc. None of these things actually impair too badly but if the time is short, there is an impact. I suppose it's similar to sports where you have to, at least somewhat, adapt on the go. Many teams botch a game here and there. But... your final exam? Botch it and you're screwed. It just does not feel right.
- hashmap 5mo ago
  it depends on what you mean by fair, i think. is the purpose of the test to gauge how well a person performs on tests? the difficult problems i encounter in the real world involve collecting a bunch of context on it, sitting with it for a bit, then putting it down and going for a walk. when i come up with something to try it's usually when i am outside and looking at trees. maybe this is good for weeding out students who would not be a good fit for high stakes speed math.
  - chongli 5mo ago
    It's really not high speed math. You'll have 2.5 hours to solve maybe 5 problems on the exam. A math grad student who TAs the course can typically complete the exam in 45 minutes or less, without any preparation whatsoever. An undergraduate student who can't solve the exam in 2.5 hours (without external aid) is not going to solve it in 10 or even 20 hours. They simply do not know what they're doing at that point. It's really no different from a software dev interviewee who can't solve basic programming questions without being handheld and coached through the entire process.
    - knollimar 5mo ago
      I think if TA no prep completion time to total time ratio is above 2.5 it's not the time pressure based exams I'm referring to. I didn't get those exams in math courses. Engineering courses I did get those where some of the point of the test is the speed of your pattern matching ability (did you use design skills to reason about it or did you brute force a solution). Math you're already extrapolating theory and you need ample time for that.
tekla 5mo ago
If you can't do synthesis problems on a exam, you are not prepared for it. I quite fondly remember a situation with a two question three hour open everything exam where I stared at the 1st problem and relevant equation for 20 minutes because I had no idea what I was looking at and was wondering if I should just leave and drop out. Then I realized that a bunch of terms could cancel out and the problem became 3 lines trivial. The professor later admitted he wanted to give a "easy" question that you couldn't just copy out of your notes and had to think and to spend the rest of the exam on the actually difficult one.
- knollimar 5mo ago
  This seems like you proved my point by staring at it for 20 minutes before starting. I don't think it's fair to grade with extreme time pressure and also expect deep thought. Being able to do deep thought quickly isn't a valuable skill, and you don't pick it up on the way ti mastery with lots of practice like you do the speed for rote problems
  - tekla 5mo ago
    No the literal oppposite. The exam was "no deep thought required. Need to know the subject and its trivial"
majormajor 5mo ago
> Synthesis problems and time pressure don't feel fair together. If you really want to grade the students its fair to put very hard problems on the exam to separate out mastery from merely very good understanding or recital.
- knollimar 5mo ago
  Theres is more than ine dimension to mastery. What skill are you grading? Can you expect some students to be able to churn out the problems? Are they doing design work? Is it more inventive or more creative? Is there more than one solution? How rigorous is the solution? My point is that you can design an exam to test for speed and ability to do the day to day well. You prepare for this exam with lots of practice. There are a few problem types but you might not have seen every one. There's a basic level of extrapolation you're expecting. You can design an exam for abstracting theory, and you prep for this exam by doing the hardest problems you can find, reading more theory, etc. You might not have seen anything like this but the test is to use the tools you're given in a class well. The test is the extrapolation. I had theory courses where I expected the latter and practical courses where I expected the former. There's more than one type of mastery; I bet your math professors would get destroyed by gifted 14 year olds in algebra competitions that involved arithmetic and timing. Do those 14 year olds not have more mastery?
- jltsiren 5mo ago
  Hard problems introduce noise to the grades. A student who gets a hard problem right could be good or lucky. In order to grade the students properly, you need multiple problems of similar difficulty. If you have both ordinary and difficult problems in the exam, it's probably long enough that it should take the entire day. Undergraduate exams tend to be short. Which means that a perfect grade should be interpreted as "meets the expectations".
  - chongli 5mo ago
    Which means that a perfect grade should be interpreted as "meets the expectations". None of the mathematics exams I wrote during undergrad at Waterloo (as a math major) would fit that description. Nearly every single one of them had midterm grades with unimodal distributions centred below 70%, tending toward 60%. Typically, only 1-5 people in the class (of 100-200) would score a perfect grade. In upper year pure mathematics courses it was common to not have any perfect grades (in a class of about 20-30).
chermi 5mo ago
Am I the only one that thinks a test in a technical discipline is poorly designed if more than one person gets a 100%? Exams are supposed to be hard. School is supposed to push you to learn the subject as best as possible.

chermi 5mo ago

I dread the day this collapse of thinking ability hits physics education. Maybe it already has.