5 ms·

How do we know that scores between different years are comparable? Couldn't it be that tests after the pandemic became harder?

mwint 4y ago
They’re normalized against previous tests: when was taking ACTs, they would always say “one section is not counted, but you don’t know which one it is, so do your best on everything.” Weird thing to say, but anyway, I’d been told that one section is used to test upcoming questions, and includes previous test questions to correlate performance cross-tests. (Remember, there’s multiple test dates per year, with multiple versions per date. They have to be consistent within the year as well as across time) I don’t recall the SAT doing this, but I’d assume they’re doing something similar.
- arjvik 4y ago
  These days they count all four sections and add a fifth calibration one. I'm sure some people screw around on it, but they probably developed some heuristic to select only the students who tried on that section.
- staticman2 4y ago
  The SATS at some point had an experimental section that was undisclosed: in other words students had to take it seriously because they didn't know what section was experimental and which one counted. I don't know how long this practice ran. Since the ACT tests reuse test questions, I'm guessing they aren't able to be transparent and you ultimately have to trust that the test makers know what they are doing.
  - lupire 4y ago
    It was always obvious to people whow practiced, because the experimental question was too easy, too hard, or both.
arjvik 4y ago
Supposedly, the SAT and ACT use statistical testing to ensure that their scores are consistent across years. Among other measures of difficulty, they have students preview the questions of future tests in a fifth, ungraded section, to make a correlation between their scores.
nomel 4y ago
That’s a great question that definitely doesn’t deserve downvotes. It appears the test method did start to change in 2020 [1], but it doesn’t seem that these changes would obviously affect performance. Perhaps many/most have moved to the computer based testing, and there could be some disadvantage there? 1. https://www.winwardacademy.com/act-announces-big-changes-retesting-computer-based-testing-superscoring/ https://www.winwardacademy.com/act-announces-big-changes-ret...

mwint 4y ago

They’re normalized against previous tests: when was taking ACTs, they would always say “one section is not counted, but you don’t know which one it is, so do your best on everything.” Weird thing to say, but anyway, I’d been told that one section is used to test upcoming questions, and includes previous test questions to correlate performance cross-tests. (Remember, there’s multiple test dates per year, with multiple versions per date. They have to be consistent within the year as well as across time) I don’t recall the SAT doing this, but I’d assume they’re doing something similar.

arjvik 4y ago
These days they count all four sections and add a fifth calibration one. I'm sure some people screw around on it, but they probably developed some heuristic to select only the students who tried on that section.
staticman2 4y ago
The SATS at some point had an experimental section that was undisclosed: in other words students had to take it seriously because they didn't know what section was experimental and which one counted. I don't know how long this practice ran. Since the ACT tests reuse test questions, I'm guessing they aren't able to be transparent and you ultimately have to trust that the test makers know what they are doing.
- lupire 4y ago
  It was always obvious to people whow practiced, because the experimental question was too easy, too hard, or both.

arjvik 4y ago

Supposedly, the SAT and ACT use statistical testing to ensure that their scores are consistent across years. Among other measures of difficulty, they have students preview the questions of future tests in a fifth, ungraded section, to make a correlation between their scores.

nomel 4y ago

That’s a great question that definitely doesn’t deserve downvotes. It appears the test method did start to change in 2020 [1], but it doesn’t seem that these changes would obviously affect performance. Perhaps many/most have moved to the computer based testing, and there could be some disadvantage there? 1. https://www.winwardacademy.com/act-announces-big-changes-retesting-computer-based-testing-superscoring/ https://www.winwardacademy.com/act-announces-big-changes-ret...