11 ms·
In the AI era, tests are going to have to be in person and hand written. I have written an article on how I have adjusted my classes to the situation: https:/
by recursivedoubts 4mo ago
In the AI era, tests are going to have to be in person and hand written.
I have written an article on how I have adjusted my classes to the situation:
https://htmx.org/essays/universities-and-ai/ https://htmx.org/essays/universities-and-ai/
Ironically, I think the AI era may make university degrees a better signal of the intellectual abilities of students due to the presence of pre-computer infrastructure like large lecture halls, industrial-scale copiers, etc.
- gchallen 4mo agoThere are limits to what you can assess on timed assessments, and there are students whose performance on such assessments is not a good signal of their intellectual ability. In addition, at many institutions such tests are given infrequently and can be worth a significant component of a student's overall grade, increasing both student stress levels and the tendency for such assessments to measure short term knowledge students have obtained by cramming, not more meaningful longer-term knowledge gains. I see you're giving quizzes every three weeks, which is better than twice a semester, but still not what I would consider an ideal cadence. In my course weekly computer-based quizzes comprise 70% of a student's grade, but that's supported by a significant institutional investment in high-frequency computer-based testing: https://cbtf.illinois.edu/ https://cbtf.illinois.edu/.
- sourcecodeplz 4mo agoso let say you give the students a pop-quiz. is that not acceptable anymore because some students don't do well when surprised?
- rfergie 4mo ago> there are students whose performance on such assessments is not a good signal of their intellectual ability Is there a form of assessment that is a good signal of the intellectual ability of all students?
- deleted 4mo ago[deleted]
- warumdarum 4mo agoFruit of labor? Good students of science produce novel science..as in testable ideas without precedent.. a thing llms struggle with, as they at best can interpolate and mash up previous ideas
- hansvm 4mo agoMany good students don't produce such artifacts. That's one of the big problems in academia -- a smart person with good ideas can inadvertently pursue a less promising path and not produce ground-breaking research, so how do you tease that apart from somebody scamming the system? The current approach is to measure unrelated bullshit like citation metrics. I'd hate to see those sorts of perverse incentives pushed down into the student body as well.
- warumdarum 4mo agoOr maybe you can be smart but have no ideas in quantity and quality, thus you rig the systems in your favour to not be dethroned by those who can. But that would show by number of innovations per yearafter a systemic shakeup like a war.
- hansvm 4mo agoIt shows up easily at a macro level of you look at total "innovations" for a society (and have a good way to measure that -- currently still in an "I know it when I see it" stage). It won't usually show up in individual researchers. A few will absolutely be insanely productive with real innovations, but unless you're proposing only hiring that small cast of individuals then you're back to the problem of distinguishing good scientific ideas and methods which just didn't pan out from other forms of actual laziness and fraud given that in both cases you see no meaningful output from the researcher in question.
- recursivedoubts 4mo agoI agree and would like to move towards a customized computer setup like you mention. A friend at Berkeley manages a similar setup. Unfortunately Montana State is too small to have set one up yet.
- nneonneo 4mo agoSpeed is an asset, and I think it's an underrated one. Timed assessments are, in part, a speed challenge; students who understand the material more thoroughly can apply it faster and more accurately, giving them more time to complete the exam and to review it. Yes, students can raise their score by cramming, but won't be as fast or as fluent as a student who has learned and internalized the material over the term.
- walrus01 4mo ago> In the AI era, tests are going to have to be in person and hand written. Forcing hand written should really not be necessary. It would be very cheap in terms of old computing hardware to set up a test room with old desktop PCs that have wired only NICs (with a network connection that goes to a switch in the same room with no uplink, connected to a decent size laser printer only), running something like lubuntu and libreoffice writer as a basic word processor. Let people at least type their essay with the standard features of a word processor as usable as MS Word 2000 or better.
- BaculumMeumEst 4mo agoNobody is going to do that.
- walrus01 4mo agoWhy not?
- sokoloff 4mo agoDo you let students bring their own keyboards? If not, does that disadvantage a Dvorak user? Or a Kinesis user? Or a non-US layout? Or a Mac user? If you do let them bring their own keyboards, how sure are you that those are just dumb keyboards?
- walrus01 4mo agoI would bet good money, statistically, that forcing hand written tests with pen and paper is much more likely to disadvantage students with a medically documented physical disability than you are to encounter someone who can't type at all on a standard 101 key qwerty layout keyboard.
- vcf 4mo agoAt my university, students with verified disabilities are allowed to use a university-provided laptop (properly locked-down, with allowed tools such as screen readers). But these are special cases. Computers for everyone would be costly and impractical given exams are punctual but all roughly over the same week or so.
- Waterluvian 4mo agoBeing hand written would be silly theatre that excludes perfectly capable people (like me) who cannot sustain writing more than a few minutes. Edit: I think we’re actually all agreeing. That a typed exam as an option is perfectly cromulent. I’m just saying the idea that it must be written is silly.
- glimshe 4mo agoI don't want to dismiss your limitation, but it's better to create an acommodation for your particular case than to prevent the system from being implemented for everybody else.
- Waterluvian 4mo agoYeah, that’s the way it worked in undergrad for me. People would have hand written exams and the ones that had essay portions I’d do under more careful scrutiny in a different location using their hardware.
- notreallyauser 4mo agoGen Xer here: we coped with hand written exams just fine. Accommodations (extra time and/or a scribe) were available for those who needed them.
- deleted 4mo ago[deleted]
- recursivedoubts 4mo agoWe have a testing center set up for people who require accommodations, another piece of legacy functionality I never expected to use back when I gave my tests online.
- woodruffw 4mo agoI don't think it's theatre, in the sense that it is effective for the advertised purpose (preempting AI-based cheating). But it seems to me like there are also plenty of ways to make reasonable accommodations for people who can't do a pen-and-paper exam, such as an offline computer.
- redwall_hp 4mo agoIt's news to me that they weren't already. My exams were all in person and on paper in the early 2020s, and even my physics homework was a "do it on engineering paper and drop it in a mail slot" affair. The professors would forbid computers and phones during lectures and would stop to shame anyone who thought they were being sneaky. Computer Science classes were all on paper for exams, and low level ones did the old "here's a Javadoc, write some code with a pencil." The only online exams I had were for 100 level electives.
- markus_zhang 4mo agoIMO many CS classes should be lab based already, except the theory heavy ones. I still wonder why MIT needs to test students of OS courses on paper when the labs cover a lot of the ground. If I can do well in the labs I wouldn’t bother to memorize stuffs for the exams.
- fn-mote 4mo agoYou have to test to make sure the student who you are giving the grade to is the student who did the work. Without certifying someone’s abilities, the degree doesn’t mean much.
- nothercastle 4mo agoThe degree doesn’t mean much and never did. Minimum bar that’s all
- kevin_thibedeau 4mo agoCS is a science of computation curriculum, not programming. You can have automation do all the work for you in the labs.
- crazylogger 4mo agoStudents will just one-shot whatever lab problems professor comes up with using Claude Code.
- 4mo ago
- Escapado 4mo agoI agree that that is probably the lowest stake solution. Alternatively there are solutions like the safe exam browser which locks down the device quite well during the exam session. —- Disclosure: I run a small start up which offers teachers a platform to create and conduct digital exams and interfaces with the safe exam browser precisely because cheating is the number one complaint teachers I have spoken to have when it comes to digital exams.
- Atotalnoob 4mo agoSure, and those require kernel level access, strip privacy, and don’t run on all OS
- Escapado 4mo agoYup! That’s true and I wish it was different! In Germany there are quite a few schools which hand out locked down iPads to their students, with the expectation that they are used for school purposes and not for private usage or schools that have fixed installation school computers which can be used in exam settings.
- softwaredoug 4mo agoI think most teachers have adapted pretty well. I'm really surprised to find teachers that haven't reckoned with possibility of AI cheating in 2026. In person writing, etc. But also for anything take home students have to verbally discuss their work in some fashion. Seems to mostly work. Students still us ChatGPT as a search engine, which seems fine. (Source: married to a professor. And my son is in high school)
- SecretDreams 4mo agoAll my stem exams were hand written, it's how it should be. The best part is coming out with everyone else already disheveled and then grabbing a drink (or many) after the last one is over. That's some solidarity drinking right there.
- Lerc 4mo agoWhy hand written? Wouldn't a one-on-one interview be a better way to establish a broad evaluation of competency.
- JoshTriplett 4mo ago> Wouldn't a one-on-one interview be a better way to establish a broad evaluation of competency. 1) Possibly, depending on the material, but most classes aren't looking for a "broad evaluation of competency", they're looking for the specific material taught. It'd make more sense as a graduation requirement: the equivalent of a dissertation but for undergraduate work. 2) Even in small classes (e.g. 20-30 students) that wouldn't scale, let alone the massive courses earlier in a curriculum or that are shared by many degrees (e.g. hundreds of students).
- panicinducer 4mo agoMaybe not everything needs to "scale"? You're asking each student to spend well over 100 hours in lectures and study for each class. Surely you can find 20 or 30 minutes per student for evaluation.
- JoshTriplett 4mo agoPlease by all means direct more funding at universities, hire several times as many professors, and have them each teach half as many classes with a fraction of the students. The result would absolutely be an improvement. (That said, any kind of subjective assessment has its own pile of hazards compared to objective assessments.)
- account42 4mo agoA problem with not scaling here is that just adding multiple examiners means that now there will be differences in how students are being graded depending on which examiner they get.
- userbinator 4mo agoI think suspected cheaters should be subjected to an interview.
- Yossarrian22 4mo agoI recommend reading this article, there is a very good reason that's not an option for this professor.
- bArray 4mo agoEven then, AI smart glasses are now making an appearance in classrooms [1]. The situation is getting really quite ridiculous. [1] https://www.evenrealities.com/en-GB/blogs/buyers-guide/ai-models-in-glasses https://www.evenrealities.com/en-GB/blogs/buyers-guide/ai-mo...
- mohamedkoubaa 4mo agoThat should result in an immediate expulsion
- deleted 4mo ago[deleted]
- cm2187 4mo agoSame with job interviews. Right now Hr insists on us doing them over zoom so we get this absurd result that we eliminate candidates that perform too well to be true, at the risk of eliminating a genuine excellent candidate. You have to look a bit messy!
- danielvaughn 4mo agooddly enough, since AI can be used to read and interpret written text, it could itself usher in a return to physical pen and paper.
- bottled_poe 4mo ago> In the AI era, tests are going to have to be in person… So are jobs.
- a-dub 4mo agoit will be interesting to see if any new formats emerge. if ai kills the take home, what can replace it for similar "let's see how far you can go if you have resources available" style examinations? maybe we'll see specialty chatbots that give the equivalent of oral examinations and/or are willing to provide reference material but not make suggestions, connections, solutions or generate prose on behalf of the user?
- snickerbockers 4mo agoWait, is this not how things are now? Do students do exams at home now? I guess it has been 14 years since I graduated but I didnt expect it to change that much.
- annzabelle 4mo agoI was in college during COVID and in upper level classes we typically had take home exams with a few days or a week to do it that were open book and open internet. Think challenging proofs that had to be typeset in LaTeX or a series of short responses that weren't expected to be as polished as our 10+ page term paper. I went to a liberal arts college that prided itself on small class sizes and the honor code, and where professors typically had a good enough read on every student that they could tell when work wasn't your own or you'd collaborated with classmates. Our in person exams were not proctored except by your peers, the professors would sit outside the room. My understanding is that this sort of arrangement has been common at liberal arts colleges and similar universities for a long time - I know UVA has long prided itself on unproctored exams. There's been a massive culture shift in the US where cheating is more and more accepted, beyond just the AI stuff, over the last few decades, and these sorts of peer based honor codes are no longer sustainable.
- madaxe_again 4mo agoStudents do exams at home with an answer key. It isn’t possible to get a mark lower than 80, as this is emotionally damaging. If you require assistance a professor will do the exam for you. Back in my day we’d sit down ‘t muddy corpse pit having questions screamed at us by ‘t dragon who were also ‘t drill instructor. Get one wrong, and that was that, fricassé.
- maCDzP 4mo agoDuring uni I only took a Java class. During the exam I had to turn in hand written code. I guess that would work today.
- TulliusCicero 4mo agoHand written is unnecessary, just ban/confiscate phones and restrict internet access if the test needs a computer.
- xavortm 4mo agogood, this will make using local llms impossible on said computer
- kome 4mo agoItalian university graduates are going to shine in this world. They face oral and written exams held in person, where it is very easy and perfectly normal to fail. This culture is completely alien to Anglo academia, which needs to find other solutions. And given the comments below/above, they are really not ready yet.
- dingdongditchme 4mo agoI did my masters in the UK and PhD in Germany. The difference in quality graduates (in engineering) is staggering. The first two semesters are more challenging than the 4 years Meng.
- anshumankmr 4mo agoFunny thing is in India we were ahead of the curve in this one cause most of our CS undergrad classes involved writing code by hand. And students were ahead of the curve too, with chits, hidden iPads, phones and the classic psst-psst in the middle of exams.
- sureglymop 4mo agoAs a student, I have no issue doing oral exams or written exams without notes. I mean I'm there to learn and out of curiosity so I like that challenge... I truly don't understand how people can sign up for a degree and then have to cheat? Must feel torturous to endure a class you're not even interested in.
- kaladin-jasnah 4mo agoSome of this is sort of a tangent, but: When I was a student (and a student TA), what I heard or saw was that students were in CS for money, or their parents forced them to study it. Both of these things created some sort of extrinsic motivation that leads to cheating. In some cases (eg. in my high school) I heard parents would threaten to beat their children if they did not do well in their classes. So maybe that pressure continues in college. And for some, they just want an easy 6 figure job and are willing to take shortcuts. Students I know (some honest, some not) have mentioned they cheat on CS interviews or lie on their resume. Additionally, I heard that multitides of parents would threaten to withhold tuition if their child failed a class. since the university is not well off, they acquiesce and make classes easier for students who aren't interested.
- bocytron 4mo agoSome are chasing a diploma, career opportunities, family expectations, social status, or simply feel like they have no other path...
- danparsonson 4mo agoFor many people and in many places, having a degree is a differentiator that increases someone's chance of being able to earn money. Not everyone is studying because they're interested, but rather because if they don't, their opportunities will be reduced, possibly significantly.
- red75prime 4mo agoI wonder if messages like "there's no such thing as gifted children"[1], create a situation where if a student struggles to keep up with their peers, they might conclude that their peers have unfair advantages. [1] https://www.brunel.ac.uk/news-and-events/news/articles/Theres-no-such-thing-as-gifted-Leading-professors-message-to-parents-in-25000-UK-schools https://www.brunel.ac.uk/news-and-events/news/articles/There...
- make3 4mo agoyou can give people laptops with Internet disabled, that's a good solution
- piokoch 4mo agoThanks for the very interesting write up, I am teaching management studies programming Python (people who want to be quants, dana analysts, etc.) and I am struggling with the same problem - how the hell do grading to make it reliable. LLMs has become good enough so whatever homework task I give, it can be solved easily by any LLM (and, in fact, this becomes unfair to those who are not able to pay for good model). I was asking to add comments & interpretation of results, this slightly helped, but LLMs are increasingly good in all this as well. So, quite seriously, I am considering some on paper tests & quizes, because what else can be done? Making people aware about importance of writing code is indeed a good hint to convince, at least part of the people to do this. Another thing is: they will leave university one day, they will search for employment and the employer might be much more hostile towards cheating during job interview and can easily make cheating with AI impossible... Question is, will it still matter?
- internet_points 4mo ago> I stress to them that I admire the heroism of their generation in resisting these temptations. > (I think older generations would do well to recognize this fact.) That is so, so much better than the default mode of blaming kids for getting addicted to the systems their older generations developed. https://ploum.net/2026-01-19-exam-with-chatbots.html https://ploum.net/2026-01-19-exam-with-chatbots.html is another interesting perspective
- nneonneo 4mo agoUsing a special computer works too. I do my exams with our institution's Computer-Based Testing Facility, a bank of computers with fixed software and firewall rules that only permit connections to the exam site. As a result, I've been able to challenge students to solve interactive software security challenges on the midterm and final with automatic grading - something that would have been impossible with pen and paper. Scalability is really the major challenge. We're rolling out more CBTF rooms and rolling out access to other departments due to demand, but it's definitely more resource-intensive than pen-and-paper. One possibility is to treat CBTFs as computer labs when not actively administering exams (or maybe even vice-versa), something we're looking into doing.
- ElProlactin 4mo agoThis is just...depressing. BSL-4-like rooms just to test university students.
- nneonneo 4mo agoI'm sure you're exaggerating, but the exam process in a CBTF is pretty lightweight. Students arrive, drop off their bags and phone etc., check in (swipe an ID card, get their picture matched), log into the computer and the exam website. When the exam starts, they refresh the site and do the exam; we've got proctors in the room as usual to watch for any conventional cheating (using a phone, consulting a friend). If you find that onerous, I guess a paper exam would probably also feel pretty depressing to you.
- goosejuice 4mo agoThe loss of trust would be the depressing part (to me). That said the amount of cheating I witnessed in undergrad was also depressing.
- nneonneo 4mo agoTBH - for me, it’s an opportunity to do a different kind of exam, with a level of interactivity and realism that isn’t possible with paper exams. It is possible but much more annoying to run such exams in a BYOD setting: for example, the lack of consistency between people’s machines, and the risk of device failures, are just two reasons why BYOD is hard even before you get into the cheating aspects.
- negergreger 4mo ago[dead]
- deleted 4mo ago[deleted]
- pixlmint 4mo agoI'm currently getting my Bachelor's in Computer Science, while in-person is absolutely necessary, it doesn't necessarily need to be hand-written. We now have the exam questions printed out, and we respond on the online exam platform, just like we would before. It is explicitly stated that no aids, including AI, are permitted. There's usually the prof and at least one more TA walking around the room, if anyone is seen with something other than the exam platform open they fail the exam. Some profs also started requiring special software for doing the exams, which works fine but is pretty annoying to use since it requires Windows, but that application basically hijacks the OS, making it impossible to navigate to any different webpage/ application. In 2024, we had some coding exams, and AI was explicitly NOT forbidden, which lead to them making the exam more difficult (because with AI, you can solve it so much more quickly, so gotta make it harder), which was really annoying, because it made it practically impossible to solve it yourself because of the time crunch. I hope they realized how wrong that was.
- dmitrygr 4mo agoHow would that software stop you from Just googling things on your phone at the same time?
- nextaccountic 4mo agoYou are supposed to not use your phone during an exam
- pixlmint 4mo agoall our exams are in-person.
- JR1427 4mo agoAll my university (Oxford, UK) exams were in-person and had written. I had 6 exams over 6 days in 2012, in which my entire undergraduate degree was examined. It's not easy, but I think that method is just as good (or bad) now as it was then.
- spydr 4mo agoI graduated from the University of Washington about 8 years ago. Even then we had to write our code on paper for exams. Every missed semi colon was marked. This isn’t a new practice and I’m surprised more colleges haven’t already been doing this regardless of AI.