5 ms·

Lots of people think a test should measure one thing (often under the slightly "main character" assumption that they'll be really good at the one truly importan

by wisty 11mo ago

Lots of people think a test should measure one thing (often under the slightly "main character" assumption that they'll be really good at the one truly important thing). Tests usually measure lots of things, and speed and accuracy / fluency in the topic is one. It certainly shouldn't be entirely a race either though. Also if a test is time constrained it's easier to mark. Give a failing student 8 hours and they'll write 30 pages of nonsense.

jaredklewis 11mo ago
> Also if a test is time constrained it's easier to mark. Give a failing student 8 hours and they'll write 30 pages of nonsense. Sure that makes sense to me, but I don't see why this would not also apply to ADHD students or any other group. And of course, there needs to be some time limit. All I am saying is, instead of having a group that gets one hour and another group that gets two hours, just give everyone two hours. I meant "constrained" not in the sense of having a limit at all, but in the sense that often tests are designed in such a way that it is very common that takers are unable to finish in the allotted time. If this constraint serves some purpose (i.e. speed is considered to be desirable) then I don't see why that purpose doesn't apply to everyone.
- wisty 11mo ago
  There can be a genuine need to make it fair. Some students with anxiety can take 10 minutes to read the first question, then are fine. ASD could mean slower uptake as they figure out the exam format. So let's say you have a generally fair time bonus for mild (clinical) anxiety. The issue is that it's fair for the average mild anxiety, it's an advantage if a student has extremely mild anxiety. As you say, hopefully the test is not overly time focused, but it's still an advantage, and a lot of these students / parents will go for every advantage they can.
  - jaredklewis 11mo ago
    If thinking speed is determined to be important and made one of the evaluation criteria, then it's important whether or not you have clinical anxiety. If thinking speed is not important, why are we evaluating it at all?
  - koolba 11mo ago
    > So let's say you have a generally fair time bonus for mild (clinical) anxiety. The issue is that it's fair for the average mild anxiety, it's an advantage if a student has extremely mild anxiety. We might as well make races longer for athletes with longer legs. It’s unfair to the ones with shorter legs to have to move them more often.
    - LiKao 11mo ago
      Well, that is kind of what we do. We look at the range of lengths that is typical for legs. And all these get to compete under typical conditions. Now let's say someone has a leg length that is fairly outside of the typical range. Let's say someone has a leg length of zero. We let these athletes compete with each other as well with different conditions, but we don't really compare the results from the typical to the atypical group.
- ajsnigrutin 11mo ago
  > All I am saying is, instead of having a group that gets one hour and another group that gets two hours, just give everyone two hours. This means that someone fully abled can think about and solve problems for 1h and 50 minutes, and use 10 minutes to physically write/type the answers, and someone with a disability (eg. missing a hand, using a prosthetic) only gets eg. one hour to solve the problems and one hour to write/type the answers due to the disablity making them write/type more slowly. Same for eg. someone blind, while with proper eyesight, you might read a question in 30 seconds, someone blind reading braille might need multiple minutes to read the same text. With unlimited time this would not be a problem, but since speed is graded too (since it's important), this causes differences in grades.
  - jaredklewis 11mo ago
    Those examples seem like reasonable, narrowly tailored accomodations to me. But the article linked in the parent comment says: > The increase is driven by more young people getting diagnosed with conditions such as ADHD, anxiety, and depression, and by universities making the process of getting accommodations easier. I think these disabilities are more complex than the broken hand and blindness examples for reasons I commented on elsewhere in this thread. In your example, a student with depression or clinical anxiety presumably only needs the same 10 minutes to write/type the answers as all the other students. Which means the extra time is added for them to "think about and solve problems." That seems fundamentally different to me than the broken hand example.
    - kayodelycaon 11mo ago
      The accommodation process shouldn't be easier. I had to provide documentation to an employer per ADA rules. For real mental disabilities, extra time is actually necessary because a person's brain isn't able to work at the same rate as a healthy person under that situation. I'm bipolar and have personal experience with this. My brain can lock up on me and I'll need five minutes or so to get it back. Depressive episodes can also affect my memory retrieval. Things come to me slower than they usually do. I also can't keep track of time the way a healthy person does. I don't actually know how much time each problem takes, and sometimes I don't know how much time is left because can't remember when the test started. I can't read analog clocks; it takes me 10~20 seconds to read them. (1) Extra time isn't giving me any advantage, it just gives me a chance. 1: I'm not exaggerating here. I've have dyslexia when it comes to numbers. Here's what I need to do to figure out how much time is left: - Dig through my brain to find what time it's started. This could remember something was being heard, something I saw, or recalling everything I know about the class. - Hold onto that number and hope I don't flip the hour and minutes. - Find a clock anywhere in the classroom and try to remember if it's accurate or not. While I'm doing this I also have to continuously tell the start time to myself. - Find out the position of the hour hand. - Tell myself the start time. - Look at the dial, figure out the hour and try to hold on to it. - Tell myself the start time. - Tell myself the hour number. - Tell myself the start time. - Find out the position of the minute hand. - Tell myself the start time. - Hour forgotten, restart from the hour hand. - Hour remembered, start time forgotten, restart from the top. - Both remembered. - Look at the dial, figure out minute and try to hold to it. - Hour and start time need to be remembered. - Combined hour and minute from analog clock. - Figure out what order I should subtract them in. - Remember everything - Two math operations. Now that I have the time and I don't remember what I needed it for. - Realize I'm taking a test and try to estimate how much more time I need to complete it. I could probably use a stopwatch or countdown, but that causes extreme anxiety as I watch the numbers change. I don't have this kind of problem at my job because I'm not taking arbitrarily-timed tests that determine my worth to society. They don't, but that's what my brain tells me no matter how many times I try to correct it.
darth_avocado 11mo ago
Well that’s the core of the problem. Either you’re measuring speed on a test or you’re not. If you are, then people with disabilities unfortunately do not pass the test and that’s the way it is. If you are not, then testing some students but not others is unfair. At the end of the day setting up a system where different students have different criteria for succeeding, automatically incentivizes students to find the easiest criteria for themselves.
- LiKao 11mo ago
  Tests usually do measure the speed. And often they should. But the question here is "the speed of what?". And how do you measure the speed without also measuring the speed of something else as an error? If you just want to measure speed, we should clock the time the student gets up, until they get to the room where the test is, get's out his pen etc. So students get the same time to do all this. We are now measuring the speed at which the student is able to do the test material including all the preparatory steps. Students who live further away or have slower cars will get worse grade, but we are just measuring speed, aren't we? That is a deliberately stupid example, but it shows that is important to ask "speed of what?". When doing a physics exam, what do we want to include in our measurement? The time it takes the person to read an write? Or just the raw speed at which physics knowledge can be applied? What is error and what is measurement? You can see it as measuring based on different criteria. Or you can see it as trying to get rid of sources of errors that may be vastly different for different students. It would be great if we could reduce the sources of errors down to zero for everyone. But unfortunately humans are very stochastic in nature, so we cannot do this. But then there has to be an acceptable source of measurement error (typical distribution) and an unacceptable source of measurement error (atypical distribution) and to actually measure based on the same criteria, you need to measure differently based on what you believe the error to be.
bawolff 11mo ago
> Tests usually measure lots of things, and speed and accuracy / fluency in the topic is one. Why are you trying to measure speed though? I can't think of any situation where someone was like: you have exactly 1 minute to integrate this function, or else. Fluency yes, but speed is a poor proxy for fluency.
- schnable 11mo ago
  Why is it a poor proxy? Someone who really understands the concepts and has the aptitude for it will get answers more quickly than someone who is shakier on it. The person who groks it less may be able to get to the answer, but needs to spend more time working through the problem. They're less good at calculus and should get a lower grade! Maybe they shouldn't fail Calc 101, but may deserve a B or (the horror) a C. Maybe that person will never get an A is calculus and that should be ok. Joel Spolsky explained this well about what makes a good programmer[1]. "If the basic concepts aren’t so easy that you don’t even have to think about them, you’re not going to get the big concepts." [1] https://www.joelonsoftware.com/2006/10/25/the-guerrilla-guide-to-interviewing-version-30/ https://www.joelonsoftware.com/2006/10/25/the-guerrilla-guid...
  - LargeWu 11mo ago
    My middle school aged child was recently diagnosed with learning disorders around processing, specifically with written language and math, which means even though he might know the material well it will take him a long time to do things we take for granted like reading and writing. But, he does much much better with recall and speed when transmitting and testing his knowledge orally. He's awful with spelling and phonemes, but his vocabulary is above grade level. For kids like him, the time aspect is not necessarily correlated to subject mastery.
    - peterfirefly 11mo ago
      Can he build more advanced concepts on top of the ones he supposedly masters? Can he do that well? Is he likely to continue to be able to do that as he progresses to the stuff that is actually hard? (My guess is that the answers are yes (so far), no, and definitely not.) Take slow processing is a really good symptom of something that needs more practice time.

by wisty 11mo ago

jaredklewis 11mo ago

> Also if a test is time constrained it's easier to mark. Give a failing student 8 hours and they'll write 30 pages of nonsense. Sure that makes sense to me, but I don't see why this would not also apply to ADHD students or any other group. And of course, there needs to be some time limit. All I am saying is, instead of having a group that gets one hour and another group that gets two hours, just give everyone two hours. I meant "constrained" not in the sense of having a limit at all, but in the sense that often tests are designed in such a way that it is very common that takers are unable to finish in the allotted time. If this constraint serves some purpose (i.e. speed is considered to be desirable) then I don't see why that purpose doesn't apply to everyone.

wisty 11mo ago
There can be a genuine need to make it fair. Some students with anxiety can take 10 minutes to read the first question, then are fine. ASD could mean slower uptake as they figure out the exam format. So let's say you have a generally fair time bonus for mild (clinical) anxiety. The issue is that it's fair for the average mild anxiety, it's an advantage if a student has extremely mild anxiety. As you say, hopefully the test is not overly time focused, but it's still an advantage, and a lot of these students / parents will go for every advantage they can.
- jaredklewis 11mo ago
  If thinking speed is determined to be important and made one of the evaluation criteria, then it's important whether or not you have clinical anxiety. If thinking speed is not important, why are we evaluating it at all?
- koolba 11mo ago
  > So let's say you have a generally fair time bonus for mild (clinical) anxiety. The issue is that it's fair for the average mild anxiety, it's an advantage if a student has extremely mild anxiety. We might as well make races longer for athletes with longer legs. It’s unfair to the ones with shorter legs to have to move them more often.
  - LiKao 11mo ago
    Well, that is kind of what we do. We look at the range of lengths that is typical for legs. And all these get to compete under typical conditions. Now let's say someone has a leg length that is fairly outside of the typical range. Let's say someone has a leg length of zero. We let these athletes compete with each other as well with different conditions, but we don't really compare the results from the typical to the atypical group.
ajsnigrutin 11mo ago
> All I am saying is, instead of having a group that gets one hour and another group that gets two hours, just give everyone two hours. This means that someone fully abled can think about and solve problems for 1h and 50 minutes, and use 10 minutes to physically write/type the answers, and someone with a disability (eg. missing a hand, using a prosthetic) only gets eg. one hour to solve the problems and one hour to write/type the answers due to the disablity making them write/type more slowly. Same for eg. someone blind, while with proper eyesight, you might read a question in 30 seconds, someone blind reading braille might need multiple minutes to read the same text. With unlimited time this would not be a problem, but since speed is graded too (since it's important), this causes differences in grades.
- jaredklewis 11mo ago
  Those examples seem like reasonable, narrowly tailored accomodations to me. But the article linked in the parent comment says: > The increase is driven by more young people getting diagnosed with conditions such as ADHD, anxiety, and depression, and by universities making the process of getting accommodations easier. I think these disabilities are more complex than the broken hand and blindness examples for reasons I commented on elsewhere in this thread. In your example, a student with depression or clinical anxiety presumably only needs the same 10 minutes to write/type the answers as all the other students. Which means the extra time is added for them to "think about and solve problems." That seems fundamentally different to me than the broken hand example.
  - kayodelycaon 11mo ago
    The accommodation process shouldn't be easier. I had to provide documentation to an employer per ADA rules. For real mental disabilities, extra time is actually necessary because a person's brain isn't able to work at the same rate as a healthy person under that situation. I'm bipolar and have personal experience with this. My brain can lock up on me and I'll need five minutes or so to get it back. Depressive episodes can also affect my memory retrieval. Things come to me slower than they usually do. I also can't keep track of time the way a healthy person does. I don't actually know how much time each problem takes, and sometimes I don't know how much time is left because can't remember when the test started. I can't read analog clocks; it takes me 10~20 seconds to read them. (1) Extra time isn't giving me any advantage, it just gives me a chance. 1: I'm not exaggerating here. I've have dyslexia when it comes to numbers. Here's what I need to do to figure out how much time is left: - Dig through my brain to find what time it's started. This could remember something was being heard, something I saw, or recalling everything I know about the class. - Hold onto that number and hope I don't flip the hour and minutes. - Find a clock anywhere in the classroom and try to remember if it's accurate or not. While I'm doing this I also have to continuously tell the start time to myself. - Find out the position of the hour hand. - Tell myself the start time. - Look at the dial, figure out the hour and try to hold on to it. - Tell myself the start time. - Tell myself the hour number. - Tell myself the start time. - Find out the position of the minute hand. - Tell myself the start time. - Hour forgotten, restart from the hour hand. - Hour remembered, start time forgotten, restart from the top. - Both remembered. - Look at the dial, figure out minute and try to hold to it. - Hour and start time need to be remembered. - Combined hour and minute from analog clock. - Figure out what order I should subtract them in. - Remember everything - Two math operations. Now that I have the time and I don't remember what I needed it for. - Realize I'm taking a test and try to estimate how much more time I need to complete it. I could probably use a stopwatch or countdown, but that causes extreme anxiety as I watch the numbers change. I don't have this kind of problem at my job because I'm not taking arbitrarily-timed tests that determine my worth to society. They don't, but that's what my brain tells me no matter how many times I try to correct it.

darth_avocado 11mo ago

Well that’s the core of the problem. Either you’re measuring speed on a test or you’re not. If you are, then people with disabilities unfortunately do not pass the test and that’s the way it is. If you are not, then testing some students but not others is unfair. At the end of the day setting up a system where different students have different criteria for succeeding, automatically incentivizes students to find the easiest criteria for themselves.

LiKao 11mo ago
Tests usually do measure the speed. And often they should. But the question here is "the speed of what?". And how do you measure the speed without also measuring the speed of something else as an error? If you just want to measure speed, we should clock the time the student gets up, until they get to the room where the test is, get's out his pen etc. So students get the same time to do all this. We are now measuring the speed at which the student is able to do the test material including all the preparatory steps. Students who live further away or have slower cars will get worse grade, but we are just measuring speed, aren't we? That is a deliberately stupid example, but it shows that is important to ask "speed of what?". When doing a physics exam, what do we want to include in our measurement? The time it takes the person to read an write? Or just the raw speed at which physics knowledge can be applied? What is error and what is measurement? You can see it as measuring based on different criteria. Or you can see it as trying to get rid of sources of errors that may be vastly different for different students. It would be great if we could reduce the sources of errors down to zero for everyone. But unfortunately humans are very stochastic in nature, so we cannot do this. But then there has to be an acceptable source of measurement error (typical distribution) and an unacceptable source of measurement error (atypical distribution) and to actually measure based on the same criteria, you need to measure differently based on what you believe the error to be.

bawolff 11mo ago

> Tests usually measure lots of things, and speed and accuracy / fluency in the topic is one. Why are you trying to measure speed though? I can't think of any situation where someone was like: you have exactly 1 minute to integrate this function, or else. Fluency yes, but speed is a poor proxy for fluency.

schnable 11mo ago
Why is it a poor proxy? Someone who really understands the concepts and has the aptitude for it will get answers more quickly than someone who is shakier on it. The person who groks it less may be able to get to the answer, but needs to spend more time working through the problem. They're less good at calculus and should get a lower grade! Maybe they shouldn't fail Calc 101, but may deserve a B or (the horror) a C. Maybe that person will never get an A is calculus and that should be ok. Joel Spolsky explained this well about what makes a good programmer[1]. "If the basic concepts aren’t so easy that you don’t even have to think about them, you’re not going to get the big concepts." [1] https://www.joelonsoftware.com/2006/10/25/the-guerrilla-guide-to-interviewing-version-30/ https://www.joelonsoftware.com/2006/10/25/the-guerrilla-guid...
- LargeWu 11mo ago
  My middle school aged child was recently diagnosed with learning disorders around processing, specifically with written language and math, which means even though he might know the material well it will take him a long time to do things we take for granted like reading and writing. But, he does much much better with recall and speed when transmitting and testing his knowledge orally. He's awful with spelling and phonemes, but his vocabulary is above grade level. For kids like him, the time aspect is not necessarily correlated to subject mastery.
  - peterfirefly 11mo ago
    Can he build more advanced concepts on top of the ones he supposedly masters? Can he do that well? Is he likely to continue to be able to do that as he progresses to the stuff that is actually hard? (My guess is that the answers are yes (so far), no, and definitely not.) Take slow processing is a really good symptom of something that needs more practice time.