3 ms·
There are limits to what you can assess on timed assessments, and there are students whose performance on such assessments is not a good signal of their intelle
by gchallen 3mo ago
There are limits to what you can assess on timed assessments, and there are students whose performance on such assessments is not a good signal of their intellectual ability.
In addition, at many institutions such tests are given infrequently and can be worth a significant component of a student's overall grade, increasing both student stress levels and the tendency for such assessments to measure short term knowledge students have obtained by cramming, not more meaningful longer-term knowledge gains.
I see you're giving quizzes every three weeks, which is better than twice a semester, but still not what I would consider an ideal cadence. In my course weekly computer-based quizzes comprise 70% of a student's grade, but that's supported by a significant institutional investment in high-frequency computer-based testing: https://cbtf.illinois.edu/ https://cbtf.illinois.edu/.
- sourcecodeplz 3mo agoso let say you give the students a pop-quiz. is that not acceptable anymore because some students don't do well when surprised?
- rfergie 3mo ago> there are students whose performance on such assessments is not a good signal of their intellectual ability Is there a form of assessment that is a good signal of the intellectual ability of all students?
- deleted 3mo ago[deleted]
- warumdarum 3mo agoFruit of labor? Good students of science produce novel science..as in testable ideas without precedent.. a thing llms struggle with, as they at best can interpolate and mash up previous ideas
- hansvm 3mo agoMany good students don't produce such artifacts. That's one of the big problems in academia -- a smart person with good ideas can inadvertently pursue a less promising path and not produce ground-breaking research, so how do you tease that apart from somebody scamming the system? The current approach is to measure unrelated bullshit like citation metrics. I'd hate to see those sorts of perverse incentives pushed down into the student body as well.
- warumdarum 3mo agoOr maybe you can be smart but have no ideas in quantity and quality, thus you rig the systems in your favour to not be dethroned by those who can. But that would show by number of innovations per yearafter a systemic shakeup like a war.
- hansvm 3mo agoIt shows up easily at a macro level of you look at total "innovations" for a society (and have a good way to measure that -- currently still in an "I know it when I see it" stage). It won't usually show up in individual researchers. A few will absolutely be insanely productive with real innovations, but unless you're proposing only hiring that small cast of individuals then you're back to the problem of distinguishing good scientific ideas and methods which just didn't pan out from other forms of actual laziness and fraud given that in both cases you see no meaningful output from the researcher in question.
- recursivedoubts 3mo agoI agree and would like to move towards a customized computer setup like you mention. A friend at Berkeley manages a similar setup. Unfortunately Montana State is too small to have set one up yet.
- nneonneo 3mo agoSpeed is an asset, and I think it's an underrated one. Timed assessments are, in part, a speed challenge; students who understand the material more thoroughly can apply it faster and more accurately, giving them more time to complete the exam and to review it. Yes, students can raise their score by cramming, but won't be as fast or as fluent as a student who has learned and internalized the material over the term.