10 ms·

Bring Back the Blue-Book Exam

85 points by diodorus 1y ago

sarchertech 1y ago
This is the obvious solution to chatgpt essays. You could also just add more lab sections to CS classes, and force students to do assignments with no access to AI. When I took physics we had weekly 3.5 hour lab sections. That should be enough for most CS assignments.
- SoftTalker 1y ago
  The lab sections were enough time to run the experiments. Writing up the lab report took additional time outside of that, at least that's how my lab sections went in school.
  - girvo 1y ago
    Yep, absolutely the case for my labs.
xqcgrek2 1y ago
How do blue book exams translate to real world problems the students will encounter? Why not open book + AI use exams, because that's what students will have in their careers?
- parpfish 1y ago
  most exams aren't about solving a problem, it's about demonstrating that you've learned something.
  - nxobject 1y ago
    Sadly, I think that's what's going to be lost in the switch back to exams: the ability to assess someone's ability to iteratively solve a problem or craft a thesis.
- acbart 1y ago
  There are many subskills that you must be proficient in without tools, before you can learn more interesting skills. You need to know how to do multiplication by hand before you rely on a calculator. If you can't do multiplication with a calculator, you're not going to be able to make sense of the concepts in Algebra.
  - petra303 1y ago
    I doubt they’re talking about entry level maths.
    - bootsmann 1y ago
      Why should other subjects be any different?
  - mlloyd 1y ago
    There are also many subskills not worth learning to some people. Sometimes traversal is what's needed and not understanding. (Though I'm never going to knock gaining more understanding) Tools allow traversal of poorly understood, but recognized, subskills in a way that will make one effective in their job. An understanding of the entire stack of knowledge for every skill needed is an academic requirement born out of a lack of real world employment experience. For example, I don't need to know how LLMs work to use them effectively in my job or hobby. We should stop spending so much time teaching kids crap that will ONLY satisfy tests and teachers but has a much reduced usefulness once they leave school.
  - superposeur 1y ago
    Yes, totally. Relevant study: https://www.media.mit.edu/publications/your-brain-on-chatgpt/ https://www.media.mit.edu/publications/your-brain-on-chatgpt...
  - sdwr 1y ago
    Are multiplication and long division by hand really necessary skills? I never need to "fall back" to the principles of multiplication. Multiplying by the 1s column, then the 10s, then the 100s feels more like a mental math trick (like the digits of multiples of 9 adding to 9) than a real foundational concept.
- add-sub-mul-div 1y ago
  Because a tradeoff of relying on calculators means the average person can't do simple math on their own. It would be bad for society if a crutch for general thinking dulls those skills the same way calculators dulled home economics skills. In theory people could pull out a calculator at the grocery store but I've never seen it happen.
  - repiret 1y ago
    I don't know what you mean by "home economics", but to me, that encompasses things like balancing a checkbook and making a budget and taxes and understanding how savings and debt and compound interest works how to choose when to save and when to go into debt. The sort of money matters that apply to any schmuck trying to live in the world. The reason so many people lack those skills is that for the most part we don't teach them in high school. Calculators have nothing to do with it. Thank god we still teach quadratic equations, complex numbers, hyperbolic trig functions, and geometric constructions though. I don't know what would become of the world if most people didn't understand those things when we set them loose in the world.
- benbreen 1y ago
  Exactly, this is the reason why I struggle with this sort of solution to the problems we are all facing in education currently. On the surface it seems to make sense, but the blue book exam is entirely artificial and has basically no relationship to real world skills (subconsciously, even the quality of student handwriting handwriting could influence how graders assess blue books). Even leaving AI aside, anyone writing anything nowadays is using Google and Wikipedia and word processors, so why constrain those? Oxford and Cambridge have a "tutorial" system that is a lot closer to what I would choose in an ideal world. You write an essay at home, over the course of a week, but then you have to read it to your professor, one on one, and they interrupt you as you go, asking clarifying questions, giving suggestions, etc. (This at least is how it worked for history tutorials when I was a visiting student at an Oxford college back in 2004-5 - not sure if it's still like that). It was by far the best education I ever had because you could get realtime expert feedback on your writing in an iterative process. And it is basically AI proof, because the moment they start getting quizzed on their thinking behind a sentence or claim in an essay, anyone who used ChatGPT to write it for them will be outed.
  - zero_shift 1y ago
    Not sure how it is these days, but at Cambridge, supervisions (what Oxford calls tutorials) did not contribute to our examination / tripos scores. They were just a learning aid.
  - SoftTalker 1y ago
    It really boils down to: are universities trying to teach abstract knowledge, or are they trade schools? If they are trade schools, yes teach React and Node using LLMs (or whatever the enabling tools of the day are) and get on with it.
  - nxobject 1y ago
    > Even leaving AI aside, anyone writing anything nowadays is using Google and Wikipedia and word processors, so why constrain those? And the library, and inter-library loan (in my case), and talking to a professor with a draft...
  - lokar 1y ago
- nine_k 1y ago
  The problem is not that a student can ask AI for answers. The problem is that the student has to have an idea what questions to ask. For that, the student must have internalized certain concepts, ideas, connections. This is what has to be tested in a connectivity-free environment.
- furyofantares 1y ago
  In fact, why teach the material at all?
- altairprime 1y ago
  Can’t use AI on a date, or at a dinner party, or during a board meeting. Faking intelligence with AI only works in an online-exclusive modality, and there’s a lot of real world circumstances where being able to speak, reason, and interpret on the fly without resorting to a handheld teleprompter is necessary if you want to be viewed positively. I think a lot of people are going to be enraged when they discover that dependency on AI is unattractive once AI is universally accessible. “But I benefited from that advantage! How dare they hold that against me!”
  - noosphr 1y ago
    >Can’t use AI on a date, or at a dinner party, or during a board meeting. I get the same "you won't always have a calculator with you" vibes from 90s teachers chiding you to show your work when I hear people say stuff like this.
    - altairprime 1y ago
      I wouldn’t equate trigonometry, which underpins the classic parabolic example you’re referring to, with critical reasoning in human conversation. One is situationally useful at best; the other is mandatory to prevent exploitation by malicious people. Mental quadratics may be appealing, but the ability to reason is the bare minimum. Besides: if you’re using a calculator or AI in a board meeting, you’re likely unprepared for the board meeting.
      - noosphr 1y ago
        I'm not sure how you did proofs in trigonometry with a calculator.
        deleted 1y ago
        [deleted]
        altairprime 1y ago
        I ran my school district out of math to teach me in the 80s and ended up focusing on sysadmin in the 90s rather than repeating trigonometry or pursuing formal math at the local college. Sorry I can’t be of more use to your argument! Perhaps someone else will have applicable experiences.
- slipperydippery 1y ago
  They’re proof you learned the topic well enough to coherently write a very-few pages about it. That’s what they’re for. Making them open book + AI would just mean you need “larger” questions to be as effective a test, so you’re adding work for the graders for basically no reason.
- philwelch 1y ago
  If there weren’t differences between the educational environment and the workplace, that would be the same as if you just dumped people straight into the workplace with no education. For education to be useful at all it has to be distinct from the “real world” and optimized for learning. One way of optimizing for learning is to make the student solve the problem the hard way, so the student understands the entire process and not just the parts that are harder to optimize.
- viccis 1y ago
  Would you let a second grader take their spelling test using a word processor with a spell checker?
- lokar 1y ago
  So we just give up on direct reading comprehension and critical thinking?
- girvo 1y ago
  For the same reason I wasn’t allowed my extremely powerful SAT solver calculator in my maths exams (for high school and first year uni where it would’ve helped): Because I was demonstrating that I understood the material intrinsically, not just knew how to use tools to answer it.
nxobject 1y ago
As someone whose college papers always went through multiple drafts, I genuinely hope that the "joy" the author feels doesn't get in the way of teaching writing skills. And, as someone who got paid minimum wage to proctor tests in college, I couldn't keep a straight face at this: > The most cutting-edge educational technology of the future might very well be a stripped-down computer lab, located in a welcoming campus library, where students can complete assignments, by hand or on machines free of access to AI and the internet, in the presence of caring human proctors. I think the author's leaning heavily on vibes to do the convincing here.
- sarchertech 1y ago
  We had a math lab where you could go do your math homework and the majority of the student assistants were great. Same with the undergrad lab assistants in physics labs.
  - op00to 1y ago
    Writing labs exist. What’s the existence of tutoring and assistance have to do with anything?
    - sarchertech 1y ago
      The OP was saying that paid student proctors couldn’t possibly be caring. I pointed out that most of the student assistants in similar positions were in fact very caring.
      - nxobject 1y ago
        I've done both – in my undergrad I staffed a drop-in help night for a required freshman comp class, and later on in grad school I made extra cash proctoring a "alternate testing center" for students needing retakes and ADA accommodations. I can see the comparison to drop-in homework help centers. But as a test proctor my job was to get out of the way of students as much as possible while keeping things quiet – I'm not sure it's worth idealizing as "caring".
mlpoknbji 1y ago
The commenters lamenting this trend presumably have not given a takehome assignment to college students in recent years. The problem is huge and in class tests are basically the only way to test if students are learning. Unfortunately this doesn't solve the problem of AI assisted cheating on in-class exams, which is shockingly prevalent these days at least in STEM settings.
- refulgentis 1y ago
  I'm curious, how is AI assistance on an in class exam even possible? I can't picture how AI changed anything from, say, post-iPhone. i.e. I expect there to be holes in security re: bathroom breaks, but even in 2006 they confiscated cell phones during exams. I guess what I'm asking is, how did AI shift the status quo for in class exams over, say, Safari?
  - i_am_proteus 1y ago
    People bring in second phones: one to have confiscated, one to use on the exam. A common mode I have seen is phone in lap, front-facing camera ingests an exam page hung over the edge of the desk. Student then flips the page and looks down for the answer.
    - pragma_x 1y ago
      Seems easy to thwart with two exam proctors, one at each end of the exam room. Hypothetically, the whole affair is going for maximum manual effort (grading), so why not double down on test-taking labor too?
      - i_am_proteus 1y ago
        It's thwartable with adequate proctoring. I know about this mode because I found students doing it, and the students in question endured substantial academic consequences. Large lectures have hundreds of students, and to properly proctor an exam for one of these classes, one needs dozens of proctors. It can be done, but observe: for most faculty, maintaining the integrity of their course does not aid them on the path to tenure/advancement.
i_dont_know_ 1y ago
I feel like these kind of things push us as a society to decide what exactly the purpose of school should be. Currently, it's been a place for acquiring skills but also a sorting mechanism for us to know who the "best" are... I think we've put too much focus on the sorting mechanism aspect, enticing many to cheat without thinking about the fact that in doing so they shortchange themselves of actual skills. I feel like some of the language here ("securing assessments in response to AI") really feels like they're worried more about sorting than the fact that the kids won't be developing critical thinking skills if they skip that step. Maybe we can have
- belinder 1y ago
  Sure school is for acquiring skills, but it's also day care. A place to keep children during the day so their parents can work, especially in a society where more and more the expectation is that both parents work.
  - bdowling 1y ago
    The article is about college-level education, which is primarily about ranking students in order of who should get the best entry-level jobs. If technology is disrupting the effectiveness of that ordering function, then something needs to change.
    - Merrill 1y ago
      There is evidence that the ranking of students in order of who should get the best entry-level jobs is done mainly by the college admissions process which bins students into more or less selective colleges.
- viccis 1y ago
  The current system "sorts" the students who "developed critical thinking skills" out from the ones who didn't put in effort. If there's no expectation that they'll be sorted thus, then the vast majority won't (and right now don't) bother with developing or exercising those skills. Usually they'll just put all their effort into the one or two classes they have that actually make them demonstrate mastery of the material.
neonate 1y ago
https://archive.ph/Gn0sO https://archive.ph/Gn0sO
wrp 1y ago
Survey question: To what extent did blue book exams go away? When I started out (and the original Van Halen was still together), blue book exams were the norm in humanities classes. I've had narrow experience with American undergrad classes the past 25 years, so I don't have a feeling for how things have evolved.
- ipcress_file 1y ago
  I've never stopped using them, with the exception of the pandemic year, when we were forced to run online exams. Why replace a system that generally works well with one that introduces additional potential problems?
  - MikeTheGreat 1y ago
    Same here. Online instruction / learning can work for some people, and that's good. I don't understand how anyone ever thought that an online exam could be made secure. There's just no way to ensure that the person who registered for the course is the one taking the exam when you don't control anything about the hardware or location, when students have a wide variety of hardware that you must support, and any attempt at remove video monitoring of the exam immediately runs into scalability and privacy issues. Like, even if you're watching a video of the person taking the online exam, how do they prove that they didn't just hook up an extra keyboard, mouse and (mirrored) monitor for person #2 to take the exam for them while they do their best to type and/or mouse in a convincing way? It also doesn't help that you periodically get students who will try to wheedle, whinge, and weasel their way into an online exam, but then bomb the in-person exam (it's so strange and totally unrelated that they really, really wanted to take an online exam instead of in-person!). Ok, I'll stop ranting now :)
    - ipcress_file 1y ago
      I get it. My major concern was that students were cheating online (nearly 50% if I was detecting them all) who I didn't think would have cheated in the classroom. I didn't like the idea that we were creating a situation that enticed students to cheat. That being said, the whole experience had an impact on my generally optimistic view of human nature.
CompoundEyes 1y ago
There are schemes happening in interviews too. I've been doing many technical interviews for a senior role with off shore candidates lately. They come to me after passing a challenging online test that has controls to check for browser focus lost, opening new tabs etc. I think it may require a camera to be on too. Even with that more than half of the candidates can't code. Our interview usually starts with them breathlessly reading from a script out of the corner of their eye. I'm ok with notes to make sure you hit some high points about yourself even in person. Nervousness shouldn't disqualify a talented person. But with the coding part I've gotten exasperated and started asking these senior candidates to share their screen and do a fizz buzz exercise live in a text editor in the first few minutes. If they struggle I politely end the interview on the 15. One candidate cheated and it was interesting to watch. In the time I sent the message in Zoom and them sharing their screen, just a few seconds, they had either queried or LLM-ed it on their phone or another computer, had someone off screen or in the same room listening and sharing the answer on another monitor or something else. Whatever it was they turned their head slightly to the side, squinted a bit and typed the answer in Java. A few syncopated characters at a time. When asked what modulo was they didn't know and couldn't make any changes to it. It was wacky. In retrospect I think it was them reading the question out loud to an LLM. I'm waiting for the candidate who has someone behind them with the same shirt on pretending to be their arms.
- OptionOfT 1y ago
  > They come to me after passing a challenging online test that has controls to check for browser focus lost, opening new tabs etc. These are the absolute worst. You're taking people out of their comfort zone (highly customized IDE like JetBrains / VSCode / Vim) which cause them to lose shortcuts and decently working intellisense. Yes, my TypeScript in my projects is configured in such a way that I get way more information from the compiler than the standard config. After all, you're testing my ability as a software engineer, not a code monkey, right? In this very uncomfortable place there is no way of asking questions. Yes, sometimes stuff is ambiguous. I rather have someone who asks questions vs someone who guesses and gets it right. The testing setup is horrible too. No feedback as to what part of the tests fail, just... fail. No debugger. No way of adding log messages. When was the last time you've been in that situation at your workplace? All under the pressure of time, and additional stress from the person that they really NEED a new job. Oh, and when you use compiled languages, they're way slower than say TypeScript due to the compilation phase. And then even when your score (comprised of x passed tests and y failed tests) is of passing grade there is a manager out there looking at how many times someone tabbed outside of the window/tab? Where am I supposed to look up stuff? Do you know all of this information by heart: https://doc.rust-lang.org/std/collections/struct.BTreeMap.html#method.lower_bound https://doc.rust-lang.org/std/collections/struct.BTreeMap.ht... Which reminded me that one time I used a function recently stabilized, but the Rust version used was about 8 versions behind. With that slow compilation cycle. /sigh.
Syzygies 1y ago
One can't have an informed opinion on this without witnessing how grading a college exam takes place, first hand. Grading a stack of blue books is a "just kill me now" brutal experience. A majority of cognitive effort is just finding the page for the next problem to grade, with finding the answer another delay; any programming language with this kind of access latency would stay a "one bus" language. So of course professors rely on disinterested grad students to help grade. They'll make one pass through the stack, getting the hang of the problem and refining the point system about twenty blue books in, but never going back. With stapled exams one problem per page one can instead sort into piles for scores 0-6 (if you think your workplace is petty, try to imagine a sincere conversation about whether an answer is worth 14 or 15 points out of 20), and it's easy to review piles. When I had 200 linear algebra exams to grade at once, I'd scan everything, and use my own software to mark and bin one question at a time, making review passes a pleasure. I could grade a 10 question final in one intense sitting, with far more confidence in the results than team grading ever gave me.
- ipcress_file 1y ago
  I think problem number one is that you had 200 exams to grade.
  - downrightmike 1y ago
    The children yearn to learn. And the admin cuts costs.
    - 8bitsrule 1y ago
      How else can they hire more admins?
      - deleted 1y ago
        [deleted]
  - kkylin 1y ago
    This is not an uncomon situation in e.g., math departments across the US. Luckily I've not had to grade more then 60-70 in a single term, or ~200 but a select number of problems in a large course. Also, second stapled exams with prescribed spaces for answers. So much time wasted just looking for where the answer is.
deleted 1y ago
[deleted]
siliconc0w 1y ago
The other side is with AI tutoring, you don't even really need exams. You have a constant real time map of where the student is strong and where they're weak.
- xphos 1y ago
  But like you don't have that because AI will lie to you but kindly. People who are learning don't have the tools to point out those types of mistakes yet either so they will learn things the wrong way. Not to say teachers dont teach the wrong thing most of math is learning the wrong way only to later distill the right lesson but I think the automation of it is much worse
  - umbra07 1y ago
    You can literally just system-prompt-fix that kind of problem though.
    - xphos 1y ago
      Then why do chat bots still make stuff up? If it is so easily fixable. I think knowing what is actually true is very difficult and chat bots are struggling against a hard problem
      - umbra07 1y ago
        Cuato
      - umbra07 1y ago
        I mean, if you use SOTA models, they just about never hallucinate on these sorts of topics (elementary through lower-division college math), school-level history, science, etc. I only ever see hallucinations when using non-SOTA models, when using an intermediary product, asking about obscure information, or occasionally coding. I don't think any of that applies to being a elementary-high-school tutor. Writing is probably the most difficult subject, because even now it's difficult to prompt an LLM to write in a human intonation - but they will do a perfectly good job at explaining how to write (here's a basic essay structure, here are words we avoid in formal writing, etc). You can even augment a writing tutor chatbot by making it use specific, human-written excerpts from English textbooks, instead of allowing it to generate example paragraphs and essays.
apparent 1y ago
When I was in school, some older students complained that tests were hybrid: you could handwrite or you could type. They complained that they had grown up before typing was as common and that it wasn't fair to let students type, since they could write much more. They argued for a word limit, which was denied. Now we'll see the reverse, with students arguing that they can't handwrite effectively and need more time or whatever in order for exams to be fair. Hopefully handwritten exams will become the norm from grade school onward, so that this complaint will be rendered moot.
djoldman 1y ago
It's hard to have any substantial sympathy for the troubles of teaching private US college and university students. Given the absolutely wild increases in tuition, administrations should have massive resources to bring to bear to solving this and other problems. The reasons they don't feel more like excuses than legitimate explanations.
- bradleyjg 1y ago
  These institutions are not about teaching students. That’s a fundraising exercise for their actual purpose. And even the actual purpose is under heavy stress from cancerous administrative growth.
- secabeen 1y ago
  The wild increases in tuition are for the nameplate price, not the actual price paid by median students net of grants. See pg. 12 here: https://research.collegeboard.org/media/pdf/Trends-College-Pricing-2024-presentation.pdf https://research.collegeboard.org/media/pdf/Trends-College-P... Published tuition has gone up, but grant aid has matched it, making the net cost at 4-year public institutions to be flat, or even down slightly over the 10 year period. The same applies at private 4-year institutions, large increase in nameplate price, matched by large increase in grant aid, actual net tuition flat. Expenditure data also show that they are not spending significantly more. See the chart at the end of this page, which gives expenditures in real, per-student dollars. They are up, a little less than 20% over 10 years, but half of that is increases in hospital costs of their associated health care systems, which have nothing to do with tuition. The rest is a mix of various activities of universities, much of which are not tuition-funded. https://nces.ed.gov/programs/digest/d23/tables/dt23_334.10.asp https://nces.ed.gov/programs/digest/d23/tables/dt23_334.10.a...
  - djoldman 1y ago
    Interesting. I'll have to dig into this more. Thanks for the links.
- acbart 1y ago
  No no, that money goes to the administration. They are not involved in the teaching. That is left to the faculty, who are paid inverse to the amount of teaching they handle. Teaching is an afterthought at universities, the primary activities instead are research, building football stadiums, and paying for the revolving door of administrators.
calmbonsai 1y ago
I'm all for written exams, but not on the traditional physical Blue-Books. Just allow students to use traditional paper and then staple the exam to their answers as they're turning it in. On the few occasions (3) I took one, I ended up "punching through" the paper every single time. I tend to write with high pressure and the paper quality is atrocious. Twice, the tares were so bad the book was partially de-bound. On both occasions, when presented with a "torn/destroyed book" I had to show the proctor the "issues" and then, very carefully, hand-copy over everything into a new book in their presence--absolute PITA.
8bitsrule 1y ago
Tests are not about technology or blue books. The question is: what does an exam measure better: the aptitude or hard work of the student, or the creative effectiveness of the teacher? I experienced this question firsthand one year. I taught a branch of math in the way I (remembered being) taught to a class of students who were not receptive to that approach. When I tested them, I was very disappointed. After a few weeks of mulling, I went back to that branch with some new ideas about how to approach the topic. This time with a graphic rather than an abstract approach. More grounded in their likely life experiences. Almost immediately I started hearing "oh, now I get it!" and "well that's easy". Same test, but -much- better results. It wasn't their fault. What they taught me was invaluable. Yes, exams measure the effectiveness of teacher presentations as much as they measure what students have learned. Good teaching is not a part-time job ... many students are ill-served by this approach. A person who resents teaching as a part-time burden is unlikely to shine at it. And students sense it. Nor is good teaching a gift from the divine - any more than great lab technique, or crisp programming. Many teachers don't recite the same notes year-after-year, because they're 'good enough'. Their exam results help them to learn from their mistakes. If the exams don't measure teacher effectiveness as well, then what does? What their paying students walk away with. Is it a treasure, or a wheelbarrow of dirt?

85 points by diodorus 1y ago

This is the obvious solution to chatgpt essays. You could also just add more lab sections to CS classes, and force students to do assignments with no access to AI. When I took physics we had weekly 3.5 hour lab sections. That should be enough for most CS assignments.

SoftTalker 1y ago
The lab sections were enough time to run the experiments. Writing up the lab report took additional time outside of that, at least that's how my lab sections went in school.
- girvo 1y ago
  Yep, absolutely the case for my labs.

xqcgrek2 1y ago

How do blue book exams translate to real world problems the students will encounter? Why not open book + AI use exams, because that's what students will have in their careers?

parpfish 1y ago
most exams aren't about solving a problem, it's about demonstrating that you've learned something.
- nxobject 1y ago
  Sadly, I think that's what's going to be lost in the switch back to exams: the ability to assess someone's ability to iteratively solve a problem or craft a thesis.
acbart 1y ago
There are many subskills that you must be proficient in without tools, before you can learn more interesting skills. You need to know how to do multiplication by hand before you rely on a calculator. If you can't do multiplication with a calculator, you're not going to be able to make sense of the concepts in Algebra.
- petra303 1y ago
  I doubt they’re talking about entry level maths.
  - bootsmann 1y ago
    Why should other subjects be any different?
- mlloyd 1y ago
  There are also many subskills not worth learning to some people. Sometimes traversal is what's needed and not understanding. (Though I'm never going to knock gaining more understanding) Tools allow traversal of poorly understood, but recognized, subskills in a way that will make one effective in their job. An understanding of the entire stack of knowledge for every skill needed is an academic requirement born out of a lack of real world employment experience. For example, I don't need to know how LLMs work to use them effectively in my job or hobby. We should stop spending so much time teaching kids crap that will ONLY satisfy tests and teachers but has a much reduced usefulness once they leave school.
- superposeur 1y ago
  Yes, totally. Relevant study: https://www.media.mit.edu/publications/your-brain-on-chatgpt/ https://www.media.mit.edu/publications/your-brain-on-chatgpt...
- sdwr 1y ago
  Are multiplication and long division by hand really necessary skills? I never need to "fall back" to the principles of multiplication. Multiplying by the 1s column, then the 10s, then the 100s feels more like a mental math trick (like the digits of multiples of 9 adding to 9) than a real foundational concept.
add-sub-mul-div 1y ago
Because a tradeoff of relying on calculators means the average person can't do simple math on their own. It would be bad for society if a crutch for general thinking dulls those skills the same way calculators dulled home economics skills. In theory people could pull out a calculator at the grocery store but I've never seen it happen.
- repiret 1y ago
  I don't know what you mean by "home economics", but to me, that encompasses things like balancing a checkbook and making a budget and taxes and understanding how savings and debt and compound interest works how to choose when to save and when to go into debt. The sort of money matters that apply to any schmuck trying to live in the world. The reason so many people lack those skills is that for the most part we don't teach them in high school. Calculators have nothing to do with it. Thank god we still teach quadratic equations, complex numbers, hyperbolic trig functions, and geometric constructions though. I don't know what would become of the world if most people didn't understand those things when we set them loose in the world.
benbreen 1y ago
Exactly, this is the reason why I struggle with this sort of solution to the problems we are all facing in education currently. On the surface it seems to make sense, but the blue book exam is entirely artificial and has basically no relationship to real world skills (subconsciously, even the quality of student handwriting handwriting could influence how graders assess blue books). Even leaving AI aside, anyone writing anything nowadays is using Google and Wikipedia and word processors, so why constrain those? Oxford and Cambridge have a "tutorial" system that is a lot closer to what I would choose in an ideal world. You write an essay at home, over the course of a week, but then you have to read it to your professor, one on one, and they interrupt you as you go, asking clarifying questions, giving suggestions, etc. (This at least is how it worked for history tutorials when I was a visiting student at an Oxford college back in 2004-5 - not sure if it's still like that). It was by far the best education I ever had because you could get realtime expert feedback on your writing in an iterative process. And it is basically AI proof, because the moment they start getting quizzed on their thinking behind a sentence or claim in an essay, anyone who used ChatGPT to write it for them will be outed.
- zero_shift 1y ago
  Not sure how it is these days, but at Cambridge, supervisions (what Oxford calls tutorials) did not contribute to our examination / tripos scores. They were just a learning aid.
- SoftTalker 1y ago
  It really boils down to: are universities trying to teach abstract knowledge, or are they trade schools? If they are trade schools, yes teach React and Node using LLMs (or whatever the enabling tools of the day are) and get on with it.
- nxobject 1y ago
  > Even leaving AI aside, anyone writing anything nowadays is using Google and Wikipedia and word processors, so why constrain those? And the library, and inter-library loan (in my case), and talking to a professor with a draft...
- lokar 1y ago
nine_k 1y ago
The problem is not that a student can ask AI for answers. The problem is that the student has to have an idea what questions to ask. For that, the student must have internalized certain concepts, ideas, connections. This is what has to be tested in a connectivity-free environment.
furyofantares 1y ago
In fact, why teach the material at all?
altairprime 1y ago
Can’t use AI on a date, or at a dinner party, or during a board meeting. Faking intelligence with AI only works in an online-exclusive modality, and there’s a lot of real world circumstances where being able to speak, reason, and interpret on the fly without resorting to a handheld teleprompter is necessary if you want to be viewed positively. I think a lot of people are going to be enraged when they discover that dependency on AI is unattractive once AI is universally accessible. “But I benefited from that advantage! How dare they hold that against me!”
- noosphr 1y ago
  >Can’t use AI on a date, or at a dinner party, or during a board meeting. I get the same "you won't always have a calculator with you" vibes from 90s teachers chiding you to show your work when I hear people say stuff like this.
  - altairprime 1y ago
    I wouldn’t equate trigonometry, which underpins the classic parabolic example you’re referring to, with critical reasoning in human conversation. One is situationally useful at best; the other is mandatory to prevent exploitation by malicious people. Mental quadratics may be appealing, but the ability to reason is the bare minimum. Besides: if you’re using a calculator or AI in a board meeting, you’re likely unprepared for the board meeting.
    - noosphr 1y ago
      I'm not sure how you did proofs in trigonometry with a calculator.
      - deleted 1y ago
        [deleted]
      - altairprime 1y ago
        I ran my school district out of math to teach me in the 80s and ended up focusing on sysadmin in the 90s rather than repeating trigonometry or pursuing formal math at the local college. Sorry I can’t be of more use to your argument! Perhaps someone else will have applicable experiences.
slipperydippery 1y ago
They’re proof you learned the topic well enough to coherently write a very-few pages about it. That’s what they’re for. Making them open book + AI would just mean you need “larger” questions to be as effective a test, so you’re adding work for the graders for basically no reason.
philwelch 1y ago
If there weren’t differences between the educational environment and the workplace, that would be the same as if you just dumped people straight into the workplace with no education. For education to be useful at all it has to be distinct from the “real world” and optimized for learning. One way of optimizing for learning is to make the student solve the problem the hard way, so the student understands the entire process and not just the parts that are harder to optimize.
viccis 1y ago
Would you let a second grader take their spelling test using a word processor with a spell checker?
lokar 1y ago
So we just give up on direct reading comprehension and critical thinking?
girvo 1y ago
For the same reason I wasn’t allowed my extremely powerful SAT solver calculator in my maths exams (for high school and first year uni where it would’ve helped): Because I was demonstrating that I understood the material intrinsically, not just knew how to use tools to answer it.

nxobject 1y ago

As someone whose college papers always went through multiple drafts, I genuinely hope that the "joy" the author feels doesn't get in the way of teaching writing skills. And, as someone who got paid minimum wage to proctor tests in college, I couldn't keep a straight face at this: > The most cutting-edge educational technology of the future might very well be a stripped-down computer lab, located in a welcoming campus library, where students can complete assignments, by hand or on machines free of access to AI and the internet, in the presence of caring human proctors. I think the author's leaning heavily on vibes to do the convincing here.

sarchertech 1y ago
We had a math lab where you could go do your math homework and the majority of the student assistants were great. Same with the undergrad lab assistants in physics labs.
- op00to 1y ago
  Writing labs exist. What’s the existence of tutoring and assistance have to do with anything?
  - sarchertech 1y ago
    The OP was saying that paid student proctors couldn’t possibly be caring. I pointed out that most of the student assistants in similar positions were in fact very caring.
    - nxobject 1y ago
      I've done both – in my undergrad I staffed a drop-in help night for a required freshman comp class, and later on in grad school I made extra cash proctoring a "alternate testing center" for students needing retakes and ADA accommodations. I can see the comparison to drop-in homework help centers. But as a test proctor my job was to get out of the way of students as much as possible while keeping things quiet – I'm not sure it's worth idealizing as "caring".

mlpoknbji 1y ago

The commenters lamenting this trend presumably have not given a takehome assignment to college students in recent years. The problem is huge and in class tests are basically the only way to test if students are learning. Unfortunately this doesn't solve the problem of AI assisted cheating on in-class exams, which is shockingly prevalent these days at least in STEM settings.

refulgentis 1y ago
I'm curious, how is AI assistance on an in class exam even possible? I can't picture how AI changed anything from, say, post-iPhone. i.e. I expect there to be holes in security re: bathroom breaks, but even in 2006 they confiscated cell phones during exams. I guess what I'm asking is, how did AI shift the status quo for in class exams over, say, Safari?
- i_am_proteus 1y ago
  People bring in second phones: one to have confiscated, one to use on the exam. A common mode I have seen is phone in lap, front-facing camera ingests an exam page hung over the edge of the desk. Student then flips the page and looks down for the answer.
  - pragma_x 1y ago
    Seems easy to thwart with two exam proctors, one at each end of the exam room. Hypothetically, the whole affair is going for maximum manual effort (grading), so why not double down on test-taking labor too?
    - i_am_proteus 1y ago
      It's thwartable with adequate proctoring. I know about this mode because I found students doing it, and the students in question endured substantial academic consequences. Large lectures have hundreds of students, and to properly proctor an exam for one of these classes, one needs dozens of proctors. It can be done, but observe: for most faculty, maintaining the integrity of their course does not aid them on the path to tenure/advancement.

i_dont_know_ 1y ago

I feel like these kind of things push us as a society to decide what exactly the purpose of school should be. Currently, it's been a place for acquiring skills but also a sorting mechanism for us to know who the "best" are... I think we've put too much focus on the sorting mechanism aspect, enticing many to cheat without thinking about the fact that in doing so they shortchange themselves of actual skills. I feel like some of the language here ("securing assessments in response to AI") really feels like they're worried more about sorting than the fact that the kids won't be developing critical thinking skills if they skip that step. Maybe we can have

belinder 1y ago
Sure school is for acquiring skills, but it's also day care. A place to keep children during the day so their parents can work, especially in a society where more and more the expectation is that both parents work.
- bdowling 1y ago
  The article is about college-level education, which is primarily about ranking students in order of who should get the best entry-level jobs. If technology is disrupting the effectiveness of that ordering function, then something needs to change.
  - Merrill 1y ago
    There is evidence that the ranking of students in order of who should get the best entry-level jobs is done mainly by the college admissions process which bins students into more or less selective colleges.
viccis 1y ago
The current system "sorts" the students who "developed critical thinking skills" out from the ones who didn't put in effort. If there's no expectation that they'll be sorted thus, then the vast majority won't (and right now don't) bother with developing or exercising those skills. Usually they'll just put all their effort into the one or two classes they have that actually make them demonstrate mastery of the material.

neonate 1y ago

https://archive.ph/Gn0sO https://archive.ph/Gn0sO

wrp 1y ago

Survey question: To what extent did blue book exams go away? When I started out (and the original Van Halen was still together), blue book exams were the norm in humanities classes. I've had narrow experience with American undergrad classes the past 25 years, so I don't have a feeling for how things have evolved.

ipcress_file 1y ago
I've never stopped using them, with the exception of the pandemic year, when we were forced to run online exams. Why replace a system that generally works well with one that introduces additional potential problems?
- MikeTheGreat 1y ago
  Same here. Online instruction / learning can work for some people, and that's good. I don't understand how anyone ever thought that an online exam could be made secure. There's just no way to ensure that the person who registered for the course is the one taking the exam when you don't control anything about the hardware or location, when students have a wide variety of hardware that you must support, and any attempt at remove video monitoring of the exam immediately runs into scalability and privacy issues. Like, even if you're watching a video of the person taking the online exam, how do they prove that they didn't just hook up an extra keyboard, mouse and (mirrored) monitor for person #2 to take the exam for them while they do their best to type and/or mouse in a convincing way? It also doesn't help that you periodically get students who will try to wheedle, whinge, and weasel their way into an online exam, but then bomb the in-person exam (it's so strange and totally unrelated that they really, really wanted to take an online exam instead of in-person!). Ok, I'll stop ranting now :)
  - ipcress_file 1y ago
    I get it. My major concern was that students were cheating online (nearly 50% if I was detecting them all) who I didn't think would have cheated in the classroom. I didn't like the idea that we were creating a situation that enticed students to cheat. That being said, the whole experience had an impact on my generally optimistic view of human nature.

CompoundEyes 1y ago

There are schemes happening in interviews too. I've been doing many technical interviews for a senior role with off shore candidates lately. They come to me after passing a challenging online test that has controls to check for browser focus lost, opening new tabs etc. I think it may require a camera to be on too. Even with that more than half of the candidates can't code. Our interview usually starts with them breathlessly reading from a script out of the corner of their eye. I'm ok with notes to make sure you hit some high points about yourself even in person. Nervousness shouldn't disqualify a talented person. But with the coding part I've gotten exasperated and started asking these senior candidates to share their screen and do a fizz buzz exercise live in a text editor in the first few minutes. If they struggle I politely end the interview on the 15. One candidate cheated and it was interesting to watch. In the time I sent the message in Zoom and them sharing their screen, just a few seconds, they had either queried or LLM-ed it on their phone or another computer, had someone off screen or in the same room listening and sharing the answer on another monitor or something else. Whatever it was they turned their head slightly to the side, squinted a bit and typed the answer in Java. A few syncopated characters at a time. When asked what modulo was they didn't know and couldn't make any changes to it. It was wacky. In retrospect I think it was them reading the question out loud to an LLM. I'm waiting for the candidate who has someone behind them with the same shirt on pretending to be their arms.

OptionOfT 1y ago
> They come to me after passing a challenging online test that has controls to check for browser focus lost, opening new tabs etc. These are the absolute worst. You're taking people out of their comfort zone (highly customized IDE like JetBrains / VSCode / Vim) which cause them to lose shortcuts and decently working intellisense. Yes, my TypeScript in my projects is configured in such a way that I get way more information from the compiler than the standard config. After all, you're testing my ability as a software engineer, not a code monkey, right? In this very uncomfortable place there is no way of asking questions. Yes, sometimes stuff is ambiguous. I rather have someone who asks questions vs someone who guesses and gets it right. The testing setup is horrible too. No feedback as to what part of the tests fail, just... fail. No debugger. No way of adding log messages. When was the last time you've been in that situation at your workplace? All under the pressure of time, and additional stress from the person that they really NEED a new job. Oh, and when you use compiled languages, they're way slower than say TypeScript due to the compilation phase. And then even when your score (comprised of x passed tests and y failed tests) is of passing grade there is a manager out there looking at how many times someone tabbed outside of the window/tab? Where am I supposed to look up stuff? Do you know all of this information by heart: https://doc.rust-lang.org/std/collections/struct.BTreeMap.html#method.lower_bound https://doc.rust-lang.org/std/collections/struct.BTreeMap.ht... Which reminded me that one time I used a function recently stabilized, but the Rust version used was about 8 versions behind. With that slow compilation cycle. /sigh.

Syzygies 1y ago

One can't have an informed opinion on this without witnessing how grading a college exam takes place, first hand. Grading a stack of blue books is a "just kill me now" brutal experience. A majority of cognitive effort is just finding the page for the next problem to grade, with finding the answer another delay; any programming language with this kind of access latency would stay a "one bus" language. So of course professors rely on disinterested grad students to help grade. They'll make one pass through the stack, getting the hang of the problem and refining the point system about twenty blue books in, but never going back. With stapled exams one problem per page one can instead sort into piles for scores 0-6 (if you think your workplace is petty, try to imagine a sincere conversation about whether an answer is worth 14 or 15 points out of 20), and it's easy to review piles. When I had 200 linear algebra exams to grade at once, I'd scan everything, and use my own software to mark and bin one question at a time, making review passes a pleasure. I could grade a 10 question final in one intense sitting, with far more confidence in the results than team grading ever gave me.

ipcress_file 1y ago
I think problem number one is that you had 200 exams to grade.
- downrightmike 1y ago
  The children yearn to learn. And the admin cuts costs.
  - 8bitsrule 1y ago
    How else can they hire more admins?
    - deleted 1y ago
      [deleted]
- kkylin 1y ago
  This is not an uncomon situation in e.g., math departments across the US. Luckily I've not had to grade more then 60-70 in a single term, or ~200 but a select number of problems in a large course. Also, second stapled exams with prescribed spaces for answers. So much time wasted just looking for where the answer is.

deleted 1y ago

[deleted]

siliconc0w 1y ago

The other side is with AI tutoring, you don't even really need exams. You have a constant real time map of where the student is strong and where they're weak.

xphos 1y ago
But like you don't have that because AI will lie to you but kindly. People who are learning don't have the tools to point out those types of mistakes yet either so they will learn things the wrong way. Not to say teachers dont teach the wrong thing most of math is learning the wrong way only to later distill the right lesson but I think the automation of it is much worse
- umbra07 1y ago
  You can literally just system-prompt-fix that kind of problem though.
  - xphos 1y ago
    Then why do chat bots still make stuff up? If it is so easily fixable. I think knowing what is actually true is very difficult and chat bots are struggling against a hard problem
    - umbra07 1y ago
      Cuato
    - umbra07 1y ago
      I mean, if you use SOTA models, they just about never hallucinate on these sorts of topics (elementary through lower-division college math), school-level history, science, etc. I only ever see hallucinations when using non-SOTA models, when using an intermediary product, asking about obscure information, or occasionally coding. I don't think any of that applies to being a elementary-high-school tutor. Writing is probably the most difficult subject, because even now it's difficult to prompt an LLM to write in a human intonation - but they will do a perfectly good job at explaining how to write (here's a basic essay structure, here are words we avoid in formal writing, etc). You can even augment a writing tutor chatbot by making it use specific, human-written excerpts from English textbooks, instead of allowing it to generate example paragraphs and essays.

apparent 1y ago

When I was in school, some older students complained that tests were hybrid: you could handwrite or you could type. They complained that they had grown up before typing was as common and that it wasn't fair to let students type, since they could write much more. They argued for a word limit, which was denied. Now we'll see the reverse, with students arguing that they can't handwrite effectively and need more time or whatever in order for exams to be fair. Hopefully handwritten exams will become the norm from grade school onward, so that this complaint will be rendered moot.

djoldman 1y ago

It's hard to have any substantial sympathy for the troubles of teaching private US college and university students. Given the absolutely wild increases in tuition, administrations should have massive resources to bring to bear to solving this and other problems. The reasons they don't feel more like excuses than legitimate explanations.

bradleyjg 1y ago
These institutions are not about teaching students. That’s a fundraising exercise for their actual purpose. And even the actual purpose is under heavy stress from cancerous administrative growth.
secabeen 1y ago
The wild increases in tuition are for the nameplate price, not the actual price paid by median students net of grants. See pg. 12 here: https://research.collegeboard.org/media/pdf/Trends-College-Pricing-2024-presentation.pdf https://research.collegeboard.org/media/pdf/Trends-College-P... Published tuition has gone up, but grant aid has matched it, making the net cost at 4-year public institutions to be flat, or even down slightly over the 10 year period. The same applies at private 4-year institutions, large increase in nameplate price, matched by large increase in grant aid, actual net tuition flat. Expenditure data also show that they are not spending significantly more. See the chart at the end of this page, which gives expenditures in real, per-student dollars. They are up, a little less than 20% over 10 years, but half of that is increases in hospital costs of their associated health care systems, which have nothing to do with tuition. The rest is a mix of various activities of universities, much of which are not tuition-funded. https://nces.ed.gov/programs/digest/d23/tables/dt23_334.10.asp https://nces.ed.gov/programs/digest/d23/tables/dt23_334.10.a...
- djoldman 1y ago
  Interesting. I'll have to dig into this more. Thanks for the links.
acbart 1y ago
No no, that money goes to the administration. They are not involved in the teaching. That is left to the faculty, who are paid inverse to the amount of teaching they handle. Teaching is an afterthought at universities, the primary activities instead are research, building football stadiums, and paying for the revolving door of administrators.

calmbonsai 1y ago

I'm all for written exams, but not on the traditional physical Blue-Books. Just allow students to use traditional paper and then staple the exam to their answers as they're turning it in. On the few occasions (3) I took one, I ended up "punching through" the paper every single time. I tend to write with high pressure and the paper quality is atrocious. Twice, the tares were so bad the book was partially de-bound. On both occasions, when presented with a "torn/destroyed book" I had to show the proctor the "issues" and then, very carefully, hand-copy over everything into a new book in their presence--absolute PITA.

8bitsrule 1y ago

Tests are not about technology or blue books. The question is: what does an exam measure better: the aptitude or hard work of the student, or the creative effectiveness of the teacher? I experienced this question firsthand one year. I taught a branch of math in the way I (remembered being) taught to a class of students who were not receptive to that approach. When I tested them, I was very disappointed. After a few weeks of mulling, I went back to that branch with some new ideas about how to approach the topic. This time with a graphic rather than an abstract approach. More grounded in their likely life experiences. Almost immediately I started hearing "oh, now I get it!" and "well that's easy". Same test, but -much- better results. It wasn't their fault. What they taught me was invaluable. Yes, exams measure the effectiveness of teacher presentations as much as they measure what students have learned. Good teaching is not a part-time job ... many students are ill-served by this approach. A person who resents teaching as a part-time burden is unlikely to shine at it. And students sense it. Nor is good teaching a gift from the divine - any more than great lab technique, or crisp programming. Many teachers don't recite the same notes year-after-year, because they're 'good enough'. Their exam results help them to learn from their mistakes. If the exams don't measure teacher effectiveness as well, then what does? What their paying students walk away with. Is it a treasure, or a wheelbarrow of dirt?