5 ms·
I doubt oral examinations come back in general. The undergrad model relys on massive classes, I don’t see how you can do oral exams in those 300 person classes
by HDThoreaun 3y ago
I doubt oral examinations come back in general. The undergrad model relys on massive classes, I don’t see how you can do oral exams in those 300 person classes without massive cheating as the first students tell everyone else what to expect.
- nsxwolf 3y agoUse ChatGPT to write and administer 300 personalized oral exams.
- HDThoreaun 3y agoThe issue with different exams for each students is that it’s next to impossible to make them equally difficult which ruins the curve.
- btilly 3y agoThe suggestion was personalized, not different. ChatGPT could have the exact same set of questions to ask. All students are independently taking the same exam at the same time, responding conversationally. ChatGPT asks all of them the same question, and then probes based on weaknesses in the responses. Grades the answer to that question, then moves on to the next. Everyone got equal questions. Is there going to be inconsistency in how minor wording trips different followups? Sure. But it isn't clear that this randomness is worse than the subjective differences in how humans grade an essay response to a question.
- HDThoreaun 3y agoAh yea I could see something like that propping up
- OkayPhysicist 3y agoYou've missed the point of curves. Curves exist as a different solution to the problem of "we don't know whether everybody understands the material or not, and we don't know how to evaluate whether they do". It's like saying "The problem with electric cars is that they don't fit the equipment for our emissions test" If you give a pre-written test, you basically need to curve it because you don't really know how well the test aligns with your expectations of the class's understanding of the material. So you say "typically, about 20% of the class doesn't understand the course at all, 20% have a very good grasp..." etc, and then fit the written test results to the expected curve. A one-on-one communication with a student allows you to directly gauge their understanding, allowing you to assign a grade. It doesn't matter how well or poorly the rest of the class understands the material, this student, sitting in front of me, has a "B" level understanding.
- HDThoreaun 3y agoI disagree. The point of curves is to differentiate the students. Students who don’t understand do and the material should be failed, it that doesn’t tell us who should get A vs B vs C
- pierat 3y ago"Do they proficiently understand the topic?" Should be met with a trinary answer: YES, NO, WITHDRAW. We should not encourage stack ranking (or a curve) as a way to stratify students because they joined the class with too many smarter people. Go read up about stack ranking, and why it's a horrific practice. In a "grade on a curve" situation, it benefits you to drop the class if there's too many people you think smarter or more experienced than you. And it benefits you if you're smarter than them. And, thats a REALLY UGLY way to approach education.
- HDThoreaun 3y agoIn the end I guess it comes down to what the point of undergrad is. Imo we have completely left the "learn stuff" era and are in the "differentiate yourself to employers" era. That means that stack ranking is largely the point, even if it has many downsides.
- arcticfox 3y agoI feel like a 5-minute conversation, not a pre-determined list of oral questions, on most non-hard science topics would be enough to evaluate a student. You could even have an LLM grade it as a second opinion.
- mrweasel 3y agoWe had an exam that was either pass or fail, no grades. The professor and censor would kick you out with a "PASS" in five minutes if they felt that you had sufficient grasp of the topic. Some found it a bit offensive, I just loved how efficient it was. I happened to hang around on the first day of their planned three days examination and the professor just popped his head out the door and asked if I was ready now (a day and a half early), sure... I guess. Five minutes later "PASS. NEXT!"
- Negitivefrags 3y agoI am starting a private high school and this is exactly how we plan to assess students. Not the LLM part though. Not only do you get a good sense of what they know, but you get a chance to give real individual feedback about what they need to focus on next. Of course towards the end in order to get a qualification that a university will accept as entrance we will need to teach the kids to pass the standardised tests too.
- petemir 3y agoFurthermore oral examinations do not scale. We had to do them during covid times and it was gruesome, spending full days in meetings and losing a full week with only 70-80 students.
- brucethemoose2 3y agoWhat about a voiced AI chatbot to handle the simpler questions? Fight fire with fire.
- wfvr 3y agoAn AI could generate relevant questions, follow-up questions during the oral, and evaluate the student at the end. The teacher could then review the whole conversarion at a glance and check/adjust the AI evaluation, which shouldn't take more time than grading an exam and be equally scalable. I've actually tested this with a VR app (with support for mobile also), and it actually works quite well.
- brucethemoose2 3y ago> An AI could generate relevant questions, follow-up questions during the oral, and evaluate the student at the end. I dunno about that level of automation. Maybe the proctors can use it to generate a big bag of questions beforehand, that they manually review, but generating them on the fly is just asking for trouble. Administering the preset questions and responding to the student is a much better fit.
- Brian_K_White 3y agoSo as a student, I'm forced to converse with an ai and take it seriously, when in reality I reject them all? The whole original post is an example of this being terrible and not a solution for anything. It's bad enough that these things don't do a good job, don't basically function for the task, but what's as bad or worse is being forced to pretend you buy into it all like having to pretend to believe in some religion just to not get killed by everyone else. I'm not a student and so far I have been free to judge all these ais as bad on a variety of fronts, and not use them (directly anyway, of course they are used behind the scenes in ways that affect me which I can't control). This would be hell.
- ineptech 3y agoIs there any value to those lectures, nowadays? I can see a PhD lecture on the fundamentals of virtually any academic topic, for free and at my leisure, on youtube and similar sites. Waking up at 8am to attend an in-person Geology 117 lecture with 300 other freshmen is probably as outdated as a fax machine. Perhaps the next step is, education that isn't based on personal instruction isn't rare enough to be worth paying for.
- mdorazio 3y agoThe value is the degree you get at the end of your coursework.
- cooper_ganglia 3y agoAh, I see, so no value then.
- goatlover 3y agoVery much depends on the field you're in. It's a requirement for some careers.
- kjkjadksj 3y agoThe value of lecture is to see what you are expected to know for the exam. Khan academy isn’t necessarily covering the same things in the same order or level of detail as your professor.
- godelski 3y agoAs a PhD candidate (post classes) I honestly use those lectures a lot. No, I do not find a massive classroom better than an online lecture. Actually worse. The advantage of a classroom over a video lecture is the ability to interrupt and ask questions. To have the expert in front of you. In this day and age, sitting in a classroom and not participating is an entire waste of money and you're doing yourself a disservice by not participating. I actively remind my students this when I teach, who honestly are less active post pandemic. It is something I worry about. And though my account is "anonymous", if you ever show up to my lectures, not as a student but just sitting in (I don't take attendance, idk if you walk off the street and want to sit in), you too are welcome to participate. (Fwiw, there aren't really "PhD level lectures" so much as graduate lectures. Generally you stop taking classes at the end of year 2 in your PhD, which is equivalent to a masters. The distinction of the PhD is that you are learning to learn on your own. This does add extra context to why grad students ask for higher pay, especially at the non-conditional and candidate levels, as they're actually taking less resources from the school. E.g. I pay the school $10k/term (big round) for research credits, which requires no involvement from anyone. There no longer is the "student" part to my "51% student, 49% employee" split)
- specproc 3y agoI'm assuming this is America? 300 person classes is insane, particular how much debt's involved. I've studied in the UK and EU, and I don't think I've ever been in a class larger than 40.
- amalcon 3y agoThe bigger schools like Harvard often have that for the first year classes, especially classes that are required for all or many majors. The class itself, and the lectures, will have many students -- but there's usually ample availability of grad student TAs for more individualized interaction. A grad student is usually perfectly capable of teaching a freshman class. Smaller schools and upper level classes generally just have fewer people. Edit: Keep in mind that teaching is something the professor usually just puts up with, to get the resources and credibility to do the thing they are really passionate about (research or writing).
- bluGill 3y agoOne someone is standing in front of the room talking it doesn't really matter if there is one person in the room or 1 billion. sure you need a PA system and other technology to support a large crowd, but otherwise it doesn't matter. Most students would prefer to be in a large room with someone who is a great speaker vs a small room with less. If you do have questions - that is what the TA is for and you get to see the TA at least weekly. Note that the above only applies if the class is supposed to be a lecture. If it is supposed to be interactive then small classes are better. However many of the basic undergrad 1st year courses have been done so often the professor should already answer all possible questions if you keep listening. As you advance that becomes less and less true of course.
- sumtechguy 3y agoAlso they are the 'weeder' classes as the staff put it to me. Basically everyone wants to go to college. Many can get in. But not everyone is cut out for it. Those classes are usually decently challenging but mostly lectures. They cut out the people who should not be there or are not ready for it. They are also decently big because almost all of the curriculum paths require these core classes. Your second year will be 20-30 people. By the time you are in your 4th year most classes are 10-15 people usually mixed graduate/undergraduate people.
- pmarreck 3y ago> I don’t see how you can do oral exams in those 300 person classes Well that's easy: An AI will administer the oral exam and transcribe the oral answers from each person simultaneously, and possibly even grade it!
- nicbou 3y agoOne thing I don't understand is that despite automating more and more of the work away, we seem to have fewer and fewer humans available for anything that's left. I have to scan my own groceries, I can't ever talk to an actual human at any business, I order my food from a machine. Where is everyone?
- mistercheph 3y agoDidn't you know? Everyone is engaged. What did you think the point of all those engagement metrics were? To get everybody engaged. I like it better like this, before, almost no one was engaged.
- b112 3y agoThe boomers are retiring. Yes, really, that's what's happening. And it is going to get far, far worse.
- letrowekwel 3y agoThe boomers are retiring, and the US economy is strong. Here in EU boomers are also retiring, but many countries still have high youth unemployment, and there is no real worker shortage even if businesses like to complain about one.
- _jal 3y ago> Where is everyone? Get used to it. The emerging model is a virtual vending machine - if what you want isn't there, tough. If the machine breaks, well, sorry about your dollar, maybe it'll work next week. If you want to talk to a human, start by getting rich. That costs extra.
- Meekro 3y agoI've wondered about this myself. It's not just grocery scanning-- my parents used to get a bunch of bills in the mail every month, and send out a bunch of checks to pay them. It's stunning to think about how many people were needed at every step of that pipeline to make the whole thing work. All of that's automated now, and you can find similar things in many industries. That should have freed up lots of people to handle the special cases that fall through the cracks of the automation, but they're nowhere to be found.
- beej71 3y agoScaling is an issue for sure--just having the time in the week is impossible without distributing the work to TAs, and administering oral exams is tough for TAs. But you can do an oral exam along the lines of "explain to me how your project works" and that'll catch people who don't understand it regardless of whether or not they know the question in advance.
- slaymaker1907 3y agoYou're right about oral examinations, but written _proctored_ exams would also solve a lot of these problems, particularly if you require it to be handwritten. Having take home essays be handwritten would also help deter using AI by making it much more inconvenient to use compared to just doing the assignment in the first place.
- sho_hn 3y agoAnd these days, we can even rely on AI to parse the illegible handwriting!
- amalcon 3y agoI went to high school (secondary school) back when it was pretty normal to forbid students from typing assignments because "not everyone has a computer". I happen to be personally responsible, via my terrible handwriting, for at least two teachers dropping that policy. At least one other just gave me a specific exception in the name of "accessibility".
- devilbunny 3y agoA student a year ahead of me had atrocious handwriting. They allowed him to type up his AP tests after the answer period and included the typewritten sheets along with his handwritten booklet (so the graders could confirm he hadn’t altered his answers).
- faeriechangling 3y agoThey did for me as well (dysgraphia is the technical word). They made me practice handwriting after class and I think all the times I resultingly burst into tears because my wrist hurt made them take pity on me. For how much schools talk about how useful handwriting is at memorisation and such - I seldom see anybody in the working world pulling out some paper and handwriting notes. Schools seem to care more about teaching you to pass closed-book tests than they do about teaching you useful skills like how to create high quality, organised, and searchable notes.
- xena 3y ago
- fiftyfifty 3y agoUnfortunately the tried-and-true multiple choice exam complete with Scantron answer sheets remains fairly resistant to cheating with AI and is fully scalable to classes of any size. The only hope for these poor students is that AI can help their professors write better test questions.
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- madmax96 3y agoIt doesn’t matter if students know what to expect. An oral exam isn’t the same as reading a written exam out loud. There are a set of learning outcomes defined in the syllabus. The examiner uses the learning outcomes to ask probing questions that start a conversation - emphasis on conversation. A conversation can’t be faked. A simple follow up question can reveal a student only has a shallow understanding of material. Or, the student could remember everything they’ve been told, but fail to make connections between learning outcomes. You can’t cram for an oral exam. You have to digest the course material and think about what things mean in context. After all, students know what to expect on standardized tests. Some still do better than others :-).
- deleted 3y ago[deleted]
- entropinflictor 3y ago>"a conversation can't be faked" It can. An examiner questions are gonna be parsed via covert Voice recognition,processed by ai and replies played back via a e. g. small headset(a magnet inside an ear cavity, accompanied by an induction loop wire around the neck(works fine,google "микронаушник магнит" in russian, it's popular here) , and it's not as hard as you may think.
- wodenokoto 3y agoMost oral exams I've been to have been either a) talk about a report you've written, or b) draw 1 of x (say 8) subjects that have been covered throughout the year, and do a presentation on it, usually with 20-30 minutes preparation. All subjects have of course been published at least 2 weeks prior to the exam. So I don't see a problem with telling "what to expect", but I do see a problem with 1 professor and one censor examining 300 students for for 30 minutes each.