8 ms·

AI Detection services are the oiliest of snake oils. Seems like education is going to need to change drastically in the next few years. Maybe we'll go all the w

AI Detection services are the oiliest of snake oils. Seems like education is going to need to change drastically in the next few years. Maybe we'll go all the way back to oral examinations again.

HDThoreaun 3y ago
I doubt oral examinations come back in general. The undergrad model relys on massive classes, I don’t see how you can do oral exams in those 300 person classes without massive cheating as the first students tell everyone else what to expect.
- nsxwolf 3y ago
  Use ChatGPT to write and administer 300 personalized oral exams.
  - HDThoreaun 3y ago
    The issue with different exams for each students is that it’s next to impossible to make them equally difficult which ruins the curve.
    - btilly 3y ago
      The suggestion was personalized, not different. ChatGPT could have the exact same set of questions to ask. All students are independently taking the same exam at the same time, responding conversationally. ChatGPT asks all of them the same question, and then probes based on weaknesses in the responses. Grades the answer to that question, then moves on to the next. Everyone got equal questions. Is there going to be inconsistency in how minor wording trips different followups? Sure. But it isn't clear that this randomness is worse than the subjective differences in how humans grade an essay response to a question.
      - HDThoreaun 3y ago
        Ah yea I could see something like that propping up
    - OkayPhysicist 3y ago
      You've missed the point of curves. Curves exist as a different solution to the problem of "we don't know whether everybody understands the material or not, and we don't know how to evaluate whether they do". It's like saying "The problem with electric cars is that they don't fit the equipment for our emissions test" If you give a pre-written test, you basically need to curve it because you don't really know how well the test aligns with your expectations of the class's understanding of the material. So you say "typically, about 20% of the class doesn't understand the course at all, 20% have a very good grasp..." etc, and then fit the written test results to the expected curve. A one-on-one communication with a student allows you to directly gauge their understanding, allowing you to assign a grade. It doesn't matter how well or poorly the rest of the class understands the material, this student, sitting in front of me, has a "B" level understanding.
      - HDThoreaun 3y ago
        I disagree. The point of curves is to differentiate the students. Students who don’t understand do and the material should be failed, it that doesn’t tell us who should get A vs B vs C
        pierat 3y ago
        "Do they proficiently understand the topic?" Should be met with a trinary answer: YES, NO, WITHDRAW. We should not encourage stack ranking (or a curve) as a way to stratify students because they joined the class with too many smarter people. Go read up about stack ranking, and why it's a horrific practice. In a "grade on a curve" situation, it benefits you to drop the class if there's too many people you think smarter or more experienced than you. And it benefits you if you're smarter than them. And, thats a REALLY UGLY way to approach education.
        HDThoreaun 3y ago
        In the end I guess it comes down to what the point of undergrad is. Imo we have completely left the "learn stuff" era and are in the "differentiate yourself to employers" era. That means that stack ranking is largely the point, even if it has many downsides.
- arcticfox 3y ago
  I feel like a 5-minute conversation, not a pre-determined list of oral questions, on most non-hard science topics would be enough to evaluate a student. You could even have an LLM grade it as a second opinion.
  - mrweasel 3y ago
    We had an exam that was either pass or fail, no grades. The professor and censor would kick you out with a "PASS" in five minutes if they felt that you had sufficient grasp of the topic. Some found it a bit offensive, I just loved how efficient it was. I happened to hang around on the first day of their planned three days examination and the professor just popped his head out the door and asked if I was ready now (a day and a half early), sure... I guess. Five minutes later "PASS. NEXT!"
  - Negitivefrags 3y ago
    I am starting a private high school and this is exactly how we plan to assess students. Not the LLM part though. Not only do you get a good sense of what they know, but you get a chance to give real individual feedback about what they need to focus on next. Of course towards the end in order to get a qualification that a university will accept as entrance we will need to teach the kids to pass the standardised tests too.
- petemir 3y ago
  Furthermore oral examinations do not scale. We had to do them during covid times and it was gruesome, spending full days in meetings and losing a full week with only 70-80 students.
  - brucethemoose2 3y ago
    What about a voiced AI chatbot to handle the simpler questions? Fight fire with fire.
    - wfvr 3y ago
      An AI could generate relevant questions, follow-up questions during the oral, and evaluate the student at the end. The teacher could then review the whole conversarion at a glance and check/adjust the AI evaluation, which shouldn't take more time than grading an exam and be equally scalable. I've actually tested this with a VR app (with support for mobile also), and it actually works quite well.
      - brucethemoose2 3y ago
        > An AI could generate relevant questions, follow-up questions during the oral, and evaluate the student at the end. I dunno about that level of automation. Maybe the proctors can use it to generate a big bag of questions beforehand, that they manually review, but generating them on the fly is just asking for trouble. Administering the preset questions and responding to the student is a much better fit.
      - Brian_K_White 3y ago
        So as a student, I'm forced to converse with an ai and take it seriously, when in reality I reject them all? The whole original post is an example of this being terrible and not a solution for anything. It's bad enough that these things don't do a good job, don't basically function for the task, but what's as bad or worse is being forced to pretend you buy into it all like having to pretend to believe in some religion just to not get killed by everyone else. I'm not a student and so far I have been free to judge all these ais as bad on a variety of fronts, and not use them (directly anyway, of course they are used behind the scenes in ways that affect me which I can't control). This would be hell.
- ineptech 3y ago
  Is there any value to those lectures, nowadays? I can see a PhD lecture on the fundamentals of virtually any academic topic, for free and at my leisure, on youtube and similar sites. Waking up at 8am to attend an in-person Geology 117 lecture with 300 other freshmen is probably as outdated as a fax machine. Perhaps the next step is, education that isn't based on personal instruction isn't rare enough to be worth paying for.
  - mdorazio 3y ago
    The value is the degree you get at the end of your coursework.
    - cooper_ganglia 3y ago
      Ah, I see, so no value then.
      - goatlover 3y ago
        Very much depends on the field you're in. It's a requirement for some careers.
  - kjkjadksj 3y ago
    The value of lecture is to see what you are expected to know for the exam. Khan academy isn’t necessarily covering the same things in the same order or level of detail as your professor.
  - godelski 3y ago
    As a PhD candidate (post classes) I honestly use those lectures a lot. No, I do not find a massive classroom better than an online lecture. Actually worse. The advantage of a classroom over a video lecture is the ability to interrupt and ask questions. To have the expert in front of you. In this day and age, sitting in a classroom and not participating is an entire waste of money and you're doing yourself a disservice by not participating. I actively remind my students this when I teach, who honestly are less active post pandemic. It is something I worry about. And though my account is "anonymous", if you ever show up to my lectures, not as a student but just sitting in (I don't take attendance, idk if you walk off the street and want to sit in), you too are welcome to participate. (Fwiw, there aren't really "PhD level lectures" so much as graduate lectures. Generally you stop taking classes at the end of year 2 in your PhD, which is equivalent to a masters. The distinction of the PhD is that you are learning to learn on your own. This does add extra context to why grad students ask for higher pay, especially at the non-conditional and candidate levels, as they're actually taking less resources from the school. E.g. I pay the school $10k/term (big round) for research credits, which requires no involvement from anyone. There no longer is the "student" part to my "51% student, 49% employee" split)
- specproc 3y ago
  I'm assuming this is America? 300 person classes is insane, particular how much debt's involved. I've studied in the UK and EU, and I don't think I've ever been in a class larger than 40.
  - amalcon 3y ago
    The bigger schools like Harvard often have that for the first year classes, especially classes that are required for all or many majors. The class itself, and the lectures, will have many students -- but there's usually ample availability of grad student TAs for more individualized interaction. A grad student is usually perfectly capable of teaching a freshman class. Smaller schools and upper level classes generally just have fewer people. Edit: Keep in mind that teaching is something the professor usually just puts up with, to get the resources and credibility to do the thing they are really passionate about (research or writing).
  - bluGill 3y ago
    One someone is standing in front of the room talking it doesn't really matter if there is one person in the room or 1 billion. sure you need a PA system and other technology to support a large crowd, but otherwise it doesn't matter. Most students would prefer to be in a large room with someone who is a great speaker vs a small room with less. If you do have questions - that is what the TA is for and you get to see the TA at least weekly. Note that the above only applies if the class is supposed to be a lecture. If it is supposed to be interactive then small classes are better. However many of the basic undergrad 1st year courses have been done so often the professor should already answer all possible questions if you keep listening. As you advance that becomes less and less true of course.
    - sumtechguy 3y ago
      Also they are the 'weeder' classes as the staff put it to me. Basically everyone wants to go to college. Many can get in. But not everyone is cut out for it. Those classes are usually decently challenging but mostly lectures. They cut out the people who should not be there or are not ready for it. They are also decently big because almost all of the curriculum paths require these core classes. Your second year will be 20-30 people. By the time you are in your 4th year most classes are 10-15 people usually mixed graduate/undergraduate people.
- pmarreck 3y ago
  > I don’t see how you can do oral exams in those 300 person classes Well that's easy: An AI will administer the oral exam and transcribe the oral answers from each person simultaneously, and possibly even grade it!
- nicbou 3y ago
  One thing I don't understand is that despite automating more and more of the work away, we seem to have fewer and fewer humans available for anything that's left. I have to scan my own groceries, I can't ever talk to an actual human at any business, I order my food from a machine. Where is everyone?
  - mistercheph 3y ago
    Didn't you know? Everyone is engaged. What did you think the point of all those engagement metrics were? To get everybody engaged. I like it better like this, before, almost no one was engaged.
  - b112 3y ago
    The boomers are retiring. Yes, really, that's what's happening. And it is going to get far, far worse.
    - letrowekwel 3y ago
      The boomers are retiring, and the US economy is strong. Here in EU boomers are also retiring, but many countries still have high youth unemployment, and there is no real worker shortage even if businesses like to complain about one.
  - _jal 3y ago
    > Where is everyone? Get used to it. The emerging model is a virtual vending machine - if what you want isn't there, tough. If the machine breaks, well, sorry about your dollar, maybe it'll work next week. If you want to talk to a human, start by getting rich. That costs extra.
  - Meekro 3y ago
    I've wondered about this myself. It's not just grocery scanning-- my parents used to get a bunch of bills in the mail every month, and send out a bunch of checks to pay them. It's stunning to think about how many people were needed at every step of that pipeline to make the whole thing work. All of that's automated now, and you can find similar things in many industries. That should have freed up lots of people to handle the special cases that fall through the cracks of the automation, but they're nowhere to be found.
- beej71 3y ago
  Scaling is an issue for sure--just having the time in the week is impossible without distributing the work to TAs, and administering oral exams is tough for TAs. But you can do an oral exam along the lines of "explain to me how your project works" and that'll catch people who don't understand it regardless of whether or not they know the question in advance.
- slaymaker1907 3y ago
  You're right about oral examinations, but written _proctored_ exams would also solve a lot of these problems, particularly if you require it to be handwritten. Having take home essays be handwritten would also help deter using AI by making it much more inconvenient to use compared to just doing the assignment in the first place.
  - sho_hn 3y ago
    And these days, we can even rely on AI to parse the illegible handwriting!
  - amalcon 3y ago
    I went to high school (secondary school) back when it was pretty normal to forbid students from typing assignments because "not everyone has a computer". I happen to be personally responsible, via my terrible handwriting, for at least two teachers dropping that policy. At least one other just gave me a specific exception in the name of "accessibility".
    - devilbunny 3y ago
      A student a year ahead of me had atrocious handwriting. They allowed him to type up his AP tests after the answer period and included the typewritten sheets along with his handwritten booklet (so the graders could confirm he hadn’t altered his answers).
    - faeriechangling 3y ago
      They did for me as well (dysgraphia is the technical word). They made me practice handwriting after class and I think all the times I resultingly burst into tears because my wrist hurt made them take pity on me. For how much schools talk about how useful handwriting is at memorisation and such - I seldom see anybody in the working world pulling out some paper and handwriting notes. Schools seem to care more about teaching you to pass closed-book tests than they do about teaching you useful skills like how to create high quality, organised, and searchable notes.
  - xena 3y ago
- fiftyfifty 3y ago
  Unfortunately the tried-and-true multiple choice exam complete with Scantron answer sheets remains fairly resistant to cheating with AI and is fully scalable to classes of any size. The only hope for these poor students is that AI can help their professors write better test questions.
- deleted 3y ago
  [deleted]
- deleted 3y ago
  [deleted]
- madmax96 3y ago
  It doesn’t matter if students know what to expect. An oral exam isn’t the same as reading a written exam out loud. There are a set of learning outcomes defined in the syllabus. The examiner uses the learning outcomes to ask probing questions that start a conversation - emphasis on conversation. A conversation can’t be faked. A simple follow up question can reveal a student only has a shallow understanding of material. Or, the student could remember everything they’ve been told, but fail to make connections between learning outcomes. You can’t cram for an oral exam. You have to digest the course material and think about what things mean in context. After all, students know what to expect on standardized tests. Some still do better than others :-).
  - deleted 3y ago
    [deleted]
  - entropinflictor 3y ago
    >"a conversation can't be faked" It can. An examiner questions are gonna be parsed via covert Voice recognition,processed by ai and replies played back via a e. g. small headset(a magnet inside an ear cavity, accompanied by an induction loop wire around the neck(works fine,google "микронаушник магнит" in russian, it's popular here) , and it's not as hard as you may think.
- wodenokoto 3y ago
  Most oral exams I've been to have been either a) talk about a report you've written, or b) draw 1 of x (say 8) subjects that have been covered throughout the year, and do a presentation on it, usually with 20-30 minutes preparation. All subjects have of course been published at least 2 weeks prior to the exam. So I don't see a problem with telling "what to expect", but I do see a problem with 1 professor and one censor examining 300 students for for 30 minutes each.
mrweasel 3y ago
Students should have the right to be shown exactly where and why texts are flagged. Much of the snake oil will disappear pretty quickly if the solutions are required to provide any form of proof. It's would be funny, if it also wasn't so sad, that the point is that the students need to prove that they did the work, while shitty software can flag their work as being plagiarism or AI generated while providing no proof.
- onemoresoop 3y ago
  Sounds like a nightmare is about to start for kids, parents, schools, teachers and so on. Would be easily solved by going back to oral examinations, paper and pen in class essays...
  - pierat 3y ago
    Or better yet, get rid of all HOME-work. Do it all inclass. We train children and up to adulthood that encroaching on time for self is acceptable, and its absolutely not. I remember asshole teachers giving out essay homeworks during christmas breaks and similar, because "we needed to be busy". No wonder why wage theft is the biggest retail theft we have - those utter lack of boundaries are eroded the moment we set foot in schools. https://www.cbsnews.com/news/owed-employers-face-little-accountability-for-wage-theft/ https://www.cbsnews.com/news/owed-employers-face-little-acco...
    - em-bee 3y ago
      at university level that's not really going to work. you have lectures and assignments and maybe meetings to discuss assignments or present work. there are no more classes like in school. at least there weren't in my university. you are also missing one important aspect: work is usually 40 hours a week. full time study should be 40 hours too. but at least where i come from, you generally have less than 20 hours of lectures and TA meetings, etc. so you are absolutely expected to spend the other 20 hours in your room or in the library working on assignments. that is not encroaching on your time. there is only a problem if you spend more time on assignments causing you to work/study more than 40 hours in a week
RandomLensman 3y ago
Wouldn't just things were people sit in a room without or limited tools to write the exam work, too?
- SoftTalker 3y ago
  Yep the old "blue book" exams I had to sit. You get the exam questions on a piece of paoer, and a small booklet of blank paper in which to write your answers. Called a "blue book" because the cover sheet on the booklet was a sky blue color.
  - spondylosaurus 3y ago
    Blue book exams were super common when I graduated undergrad less than a decade ago—they may be old, but they're enduring :)
- OkayPhysicist 3y ago
  Or you can just trust your students. Most upper-division courses I took at university gave take-home exams. "Here's the test, come back with it in a couple days, don't talk about your answers with other students".
  - Gigachad 3y ago
    lmao, and then find that the whole class has a discord group for exchanging answers.
    - OkayPhysicist 3y ago
      Being kicked out of the science school and into the business school is a pretty huge punishment that late in the game, for negligible reward: "Oooh, I aced my Particle Physics final". It was just easier to do the test honestly, and lick your wounds later.
      - Ekaros 3y ago
dheera 3y ago
The entire idea of "spit out knowledge in 1 hour without references" is such a terrible test of one's capability in the real world that it should not be done at all.
GloriousKoji 3y ago
"The more things change, the more they stay the same." When I was a student decades ago it was when they first introduced anti-plagiarism software. We all had to submit a printed out copy of our book reports along with a floppy disk or cd-rom of our report in the latest Microsoft Word format. Almost the whole class was flagged for plagiarism. Of course the school takes plagiarism very seriously so we all had to have a one-on-one with the principal. The real truth is no one copied each others work and it was the anti-plagiarism software flagging essays because students quoted parts of the book and expanded on that. In hindsight as an adult I think the excessive principal one-on-ones was the teachers way of shoving it to him and his new software.
- natpalmer1776 3y ago
  The best form of protest is often (malicious) compliance.
- fennecfoxy 3y ago
  Yeah I remember TurnItIn. It sucked hard. And of course all Uni staff put 100% trust in it to be completely infallible and not be a piece of crap bit of software.
koito17 3y ago
Mathematics programs in Europe still use oral examinations, though they are increasingly rare in the United States. I'm not European, but I have always preferred oral examinations over written examinations because I can actually communicate my understanding of the problem to the professor and outline my thought process towards a solution, something that is much harder to do in writing, especially since you cannot get any hints or feedback on your attempts during a written examination -- you're on your own.
- deleted 3y ago
  [deleted]
beached_whale 3y ago
They need to be sued into pain. They are part of the problem and are hurting people. People are going to cheat, we know this, it is an incentive problem. But there is no excuse for false positives
fennecfoxy 3y ago
As per usual the problem isn't the technology or the tools, it's the dumb meatbag using them. This teacher should have thought "hmm, well if I'm going to pull up students on this then it's probably important I do my own research to make sure that I know that the score means and how reliable it is" but nope. "LLMs are bad cause people will use them to generate spam and fake news!" LLMs aren't the bad actor there...it's the human doing the spamming. There is a fine line, though. For example the "guns don't kill people argument" is pretty moot, gun control saves lives. If someone grabs an LLM, tells it to "generate text telling young impressionable teens to kill themselves" and spam it on Twitter...well the human is doing bad stuff but they're still enabled by the LLM. However it's easy to enact something like gun control on a physical object. Software? Well, damn that's gonna a problem - especially when the metric by which these LLMs "do better" is how closely the text resembles a hooman.

AI Detection services are the oiliest of snake oils. Seems like education is going to need to change drastically in the next few years. Maybe we'll go all the way back to oral examinations again.

HDThoreaun 3y ago

I doubt oral examinations come back in general. The undergrad model relys on massive classes, I don’t see how you can do oral exams in those 300 person classes without massive cheating as the first students tell everyone else what to expect.

nsxwolf 3y ago
Use ChatGPT to write and administer 300 personalized oral exams.
- HDThoreaun 3y ago
  The issue with different exams for each students is that it’s next to impossible to make them equally difficult which ruins the curve.
  - btilly 3y ago
    The suggestion was personalized, not different. ChatGPT could have the exact same set of questions to ask. All students are independently taking the same exam at the same time, responding conversationally. ChatGPT asks all of them the same question, and then probes based on weaknesses in the responses. Grades the answer to that question, then moves on to the next. Everyone got equal questions. Is there going to be inconsistency in how minor wording trips different followups? Sure. But it isn't clear that this randomness is worse than the subjective differences in how humans grade an essay response to a question.
    - HDThoreaun 3y ago
      Ah yea I could see something like that propping up
  - OkayPhysicist 3y ago
    You've missed the point of curves. Curves exist as a different solution to the problem of "we don't know whether everybody understands the material or not, and we don't know how to evaluate whether they do". It's like saying "The problem with electric cars is that they don't fit the equipment for our emissions test" If you give a pre-written test, you basically need to curve it because you don't really know how well the test aligns with your expectations of the class's understanding of the material. So you say "typically, about 20% of the class doesn't understand the course at all, 20% have a very good grasp..." etc, and then fit the written test results to the expected curve. A one-on-one communication with a student allows you to directly gauge their understanding, allowing you to assign a grade. It doesn't matter how well or poorly the rest of the class understands the material, this student, sitting in front of me, has a "B" level understanding.
    - HDThoreaun 3y ago
      I disagree. The point of curves is to differentiate the students. Students who don’t understand do and the material should be failed, it that doesn’t tell us who should get A vs B vs C
      - pierat 3y ago
        "Do they proficiently understand the topic?" Should be met with a trinary answer: YES, NO, WITHDRAW. We should not encourage stack ranking (or a curve) as a way to stratify students because they joined the class with too many smarter people. Go read up about stack ranking, and why it's a horrific practice. In a "grade on a curve" situation, it benefits you to drop the class if there's too many people you think smarter or more experienced than you. And it benefits you if you're smarter than them. And, thats a REALLY UGLY way to approach education.
        HDThoreaun 3y ago
        In the end I guess it comes down to what the point of undergrad is. Imo we have completely left the "learn stuff" era and are in the "differentiate yourself to employers" era. That means that stack ranking is largely the point, even if it has many downsides.
arcticfox 3y ago
I feel like a 5-minute conversation, not a pre-determined list of oral questions, on most non-hard science topics would be enough to evaluate a student. You could even have an LLM grade it as a second opinion.
- mrweasel 3y ago
  We had an exam that was either pass or fail, no grades. The professor and censor would kick you out with a "PASS" in five minutes if they felt that you had sufficient grasp of the topic. Some found it a bit offensive, I just loved how efficient it was. I happened to hang around on the first day of their planned three days examination and the professor just popped his head out the door and asked if I was ready now (a day and a half early), sure... I guess. Five minutes later "PASS. NEXT!"
- Negitivefrags 3y ago
  I am starting a private high school and this is exactly how we plan to assess students. Not the LLM part though. Not only do you get a good sense of what they know, but you get a chance to give real individual feedback about what they need to focus on next. Of course towards the end in order to get a qualification that a university will accept as entrance we will need to teach the kids to pass the standardised tests too.
petemir 3y ago
Furthermore oral examinations do not scale. We had to do them during covid times and it was gruesome, spending full days in meetings and losing a full week with only 70-80 students.
- brucethemoose2 3y ago
  What about a voiced AI chatbot to handle the simpler questions? Fight fire with fire.
  - wfvr 3y ago
    An AI could generate relevant questions, follow-up questions during the oral, and evaluate the student at the end. The teacher could then review the whole conversarion at a glance and check/adjust the AI evaluation, which shouldn't take more time than grading an exam and be equally scalable. I've actually tested this with a VR app (with support for mobile also), and it actually works quite well.
    - brucethemoose2 3y ago
      > An AI could generate relevant questions, follow-up questions during the oral, and evaluate the student at the end. I dunno about that level of automation. Maybe the proctors can use it to generate a big bag of questions beforehand, that they manually review, but generating them on the fly is just asking for trouble. Administering the preset questions and responding to the student is a much better fit.
    - Brian_K_White 3y ago
      So as a student, I'm forced to converse with an ai and take it seriously, when in reality I reject them all? The whole original post is an example of this being terrible and not a solution for anything. It's bad enough that these things don't do a good job, don't basically function for the task, but what's as bad or worse is being forced to pretend you buy into it all like having to pretend to believe in some religion just to not get killed by everyone else. I'm not a student and so far I have been free to judge all these ais as bad on a variety of fronts, and not use them (directly anyway, of course they are used behind the scenes in ways that affect me which I can't control). This would be hell.
ineptech 3y ago
Is there any value to those lectures, nowadays? I can see a PhD lecture on the fundamentals of virtually any academic topic, for free and at my leisure, on youtube and similar sites. Waking up at 8am to attend an in-person Geology 117 lecture with 300 other freshmen is probably as outdated as a fax machine. Perhaps the next step is, education that isn't based on personal instruction isn't rare enough to be worth paying for.
- mdorazio 3y ago
  The value is the degree you get at the end of your coursework.
  - cooper_ganglia 3y ago
    Ah, I see, so no value then.
    - goatlover 3y ago
      Very much depends on the field you're in. It's a requirement for some careers.
- kjkjadksj 3y ago
  The value of lecture is to see what you are expected to know for the exam. Khan academy isn’t necessarily covering the same things in the same order or level of detail as your professor.
- godelski 3y ago
  As a PhD candidate (post classes) I honestly use those lectures a lot. No, I do not find a massive classroom better than an online lecture. Actually worse. The advantage of a classroom over a video lecture is the ability to interrupt and ask questions. To have the expert in front of you. In this day and age, sitting in a classroom and not participating is an entire waste of money and you're doing yourself a disservice by not participating. I actively remind my students this when I teach, who honestly are less active post pandemic. It is something I worry about. And though my account is "anonymous", if you ever show up to my lectures, not as a student but just sitting in (I don't take attendance, idk if you walk off the street and want to sit in), you too are welcome to participate. (Fwiw, there aren't really "PhD level lectures" so much as graduate lectures. Generally you stop taking classes at the end of year 2 in your PhD, which is equivalent to a masters. The distinction of the PhD is that you are learning to learn on your own. This does add extra context to why grad students ask for higher pay, especially at the non-conditional and candidate levels, as they're actually taking less resources from the school. E.g. I pay the school $10k/term (big round) for research credits, which requires no involvement from anyone. There no longer is the "student" part to my "51% student, 49% employee" split)
specproc 3y ago
I'm assuming this is America? 300 person classes is insane, particular how much debt's involved. I've studied in the UK and EU, and I don't think I've ever been in a class larger than 40.
- amalcon 3y ago
  The bigger schools like Harvard often have that for the first year classes, especially classes that are required for all or many majors. The class itself, and the lectures, will have many students -- but there's usually ample availability of grad student TAs for more individualized interaction. A grad student is usually perfectly capable of teaching a freshman class. Smaller schools and upper level classes generally just have fewer people. Edit: Keep in mind that teaching is something the professor usually just puts up with, to get the resources and credibility to do the thing they are really passionate about (research or writing).
- bluGill 3y ago
  One someone is standing in front of the room talking it doesn't really matter if there is one person in the room or 1 billion. sure you need a PA system and other technology to support a large crowd, but otherwise it doesn't matter. Most students would prefer to be in a large room with someone who is a great speaker vs a small room with less. If you do have questions - that is what the TA is for and you get to see the TA at least weekly. Note that the above only applies if the class is supposed to be a lecture. If it is supposed to be interactive then small classes are better. However many of the basic undergrad 1st year courses have been done so often the professor should already answer all possible questions if you keep listening. As you advance that becomes less and less true of course.
  - sumtechguy 3y ago
    Also they are the 'weeder' classes as the staff put it to me. Basically everyone wants to go to college. Many can get in. But not everyone is cut out for it. Those classes are usually decently challenging but mostly lectures. They cut out the people who should not be there or are not ready for it. They are also decently big because almost all of the curriculum paths require these core classes. Your second year will be 20-30 people. By the time you are in your 4th year most classes are 10-15 people usually mixed graduate/undergraduate people.
pmarreck 3y ago
> I don’t see how you can do oral exams in those 300 person classes Well that's easy: An AI will administer the oral exam and transcribe the oral answers from each person simultaneously, and possibly even grade it!
nicbou 3y ago
One thing I don't understand is that despite automating more and more of the work away, we seem to have fewer and fewer humans available for anything that's left. I have to scan my own groceries, I can't ever talk to an actual human at any business, I order my food from a machine. Where is everyone?
- mistercheph 3y ago
  Didn't you know? Everyone is engaged. What did you think the point of all those engagement metrics were? To get everybody engaged. I like it better like this, before, almost no one was engaged.
- b112 3y ago
  The boomers are retiring. Yes, really, that's what's happening. And it is going to get far, far worse.
  - letrowekwel 3y ago
    The boomers are retiring, and the US economy is strong. Here in EU boomers are also retiring, but many countries still have high youth unemployment, and there is no real worker shortage even if businesses like to complain about one.
- _jal 3y ago
  > Where is everyone? Get used to it. The emerging model is a virtual vending machine - if what you want isn't there, tough. If the machine breaks, well, sorry about your dollar, maybe it'll work next week. If you want to talk to a human, start by getting rich. That costs extra.
- Meekro 3y ago
  I've wondered about this myself. It's not just grocery scanning-- my parents used to get a bunch of bills in the mail every month, and send out a bunch of checks to pay them. It's stunning to think about how many people were needed at every step of that pipeline to make the whole thing work. All of that's automated now, and you can find similar things in many industries. That should have freed up lots of people to handle the special cases that fall through the cracks of the automation, but they're nowhere to be found.
beej71 3y ago
Scaling is an issue for sure--just having the time in the week is impossible without distributing the work to TAs, and administering oral exams is tough for TAs. But you can do an oral exam along the lines of "explain to me how your project works" and that'll catch people who don't understand it regardless of whether or not they know the question in advance.
slaymaker1907 3y ago
You're right about oral examinations, but written _proctored_ exams would also solve a lot of these problems, particularly if you require it to be handwritten. Having take home essays be handwritten would also help deter using AI by making it much more inconvenient to use compared to just doing the assignment in the first place.
- sho_hn 3y ago
  And these days, we can even rely on AI to parse the illegible handwriting!
- amalcon 3y ago
  I went to high school (secondary school) back when it was pretty normal to forbid students from typing assignments because "not everyone has a computer". I happen to be personally responsible, via my terrible handwriting, for at least two teachers dropping that policy. At least one other just gave me a specific exception in the name of "accessibility".
  - devilbunny 3y ago
    A student a year ahead of me had atrocious handwriting. They allowed him to type up his AP tests after the answer period and included the typewritten sheets along with his handwritten booklet (so the graders could confirm he hadn’t altered his answers).
  - faeriechangling 3y ago
    They did for me as well (dysgraphia is the technical word). They made me practice handwriting after class and I think all the times I resultingly burst into tears because my wrist hurt made them take pity on me. For how much schools talk about how useful handwriting is at memorisation and such - I seldom see anybody in the working world pulling out some paper and handwriting notes. Schools seem to care more about teaching you to pass closed-book tests than they do about teaching you useful skills like how to create high quality, organised, and searchable notes.
- xena 3y ago
fiftyfifty 3y ago
Unfortunately the tried-and-true multiple choice exam complete with Scantron answer sheets remains fairly resistant to cheating with AI and is fully scalable to classes of any size. The only hope for these poor students is that AI can help their professors write better test questions.
deleted 3y ago
[deleted]
deleted 3y ago
[deleted]
madmax96 3y ago
It doesn’t matter if students know what to expect. An oral exam isn’t the same as reading a written exam out loud. There are a set of learning outcomes defined in the syllabus. The examiner uses the learning outcomes to ask probing questions that start a conversation - emphasis on conversation. A conversation can’t be faked. A simple follow up question can reveal a student only has a shallow understanding of material. Or, the student could remember everything they’ve been told, but fail to make connections between learning outcomes. You can’t cram for an oral exam. You have to digest the course material and think about what things mean in context. After all, students know what to expect on standardized tests. Some still do better than others :-).
- deleted 3y ago
  [deleted]
- entropinflictor 3y ago
  >"a conversation can't be faked" It can. An examiner questions are gonna be parsed via covert Voice recognition,processed by ai and replies played back via a e. g. small headset(a magnet inside an ear cavity, accompanied by an induction loop wire around the neck(works fine,google "микронаушник магнит" in russian, it's popular here) , and it's not as hard as you may think.
wodenokoto 3y ago
Most oral exams I've been to have been either a) talk about a report you've written, or b) draw 1 of x (say 8) subjects that have been covered throughout the year, and do a presentation on it, usually with 20-30 minutes preparation. All subjects have of course been published at least 2 weeks prior to the exam. So I don't see a problem with telling "what to expect", but I do see a problem with 1 professor and one censor examining 300 students for for 30 minutes each.

mrweasel 3y ago

Students should have the right to be shown exactly where and why texts are flagged. Much of the snake oil will disappear pretty quickly if the solutions are required to provide any form of proof. It's would be funny, if it also wasn't so sad, that the point is that the students need to prove that they did the work, while shitty software can flag their work as being plagiarism or AI generated while providing no proof.

onemoresoop 3y ago
Sounds like a nightmare is about to start for kids, parents, schools, teachers and so on. Would be easily solved by going back to oral examinations, paper and pen in class essays...
- pierat 3y ago
  Or better yet, get rid of all HOME-work. Do it all inclass. We train children and up to adulthood that encroaching on time for self is acceptable, and its absolutely not. I remember asshole teachers giving out essay homeworks during christmas breaks and similar, because "we needed to be busy". No wonder why wage theft is the biggest retail theft we have - those utter lack of boundaries are eroded the moment we set foot in schools. https://www.cbsnews.com/news/owed-employers-face-little-accountability-for-wage-theft/ https://www.cbsnews.com/news/owed-employers-face-little-acco...
  - em-bee 3y ago
    at university level that's not really going to work. you have lectures and assignments and maybe meetings to discuss assignments or present work. there are no more classes like in school. at least there weren't in my university. you are also missing one important aspect: work is usually 40 hours a week. full time study should be 40 hours too. but at least where i come from, you generally have less than 20 hours of lectures and TA meetings, etc. so you are absolutely expected to spend the other 20 hours in your room or in the library working on assignments. that is not encroaching on your time. there is only a problem if you spend more time on assignments causing you to work/study more than 40 hours in a week

RandomLensman 3y ago

Wouldn't just things were people sit in a room without or limited tools to write the exam work, too?

SoftTalker 3y ago
Yep the old "blue book" exams I had to sit. You get the exam questions on a piece of paoer, and a small booklet of blank paper in which to write your answers. Called a "blue book" because the cover sheet on the booklet was a sky blue color.
- spondylosaurus 3y ago
  Blue book exams were super common when I graduated undergrad less than a decade ago—they may be old, but they're enduring :)
OkayPhysicist 3y ago
Or you can just trust your students. Most upper-division courses I took at university gave take-home exams. "Here's the test, come back with it in a couple days, don't talk about your answers with other students".
- Gigachad 3y ago
  lmao, and then find that the whole class has a discord group for exchanging answers.
  - OkayPhysicist 3y ago
    Being kicked out of the science school and into the business school is a pretty huge punishment that late in the game, for negligible reward: "Oooh, I aced my Particle Physics final". It was just easier to do the test honestly, and lick your wounds later.
    - Ekaros 3y ago

dheera 3y ago

The entire idea of "spit out knowledge in 1 hour without references" is such a terrible test of one's capability in the real world that it should not be done at all.

GloriousKoji 3y ago

"The more things change, the more they stay the same." When I was a student decades ago it was when they first introduced anti-plagiarism software. We all had to submit a printed out copy of our book reports along with a floppy disk or cd-rom of our report in the latest Microsoft Word format. Almost the whole class was flagged for plagiarism. Of course the school takes plagiarism very seriously so we all had to have a one-on-one with the principal. The real truth is no one copied each others work and it was the anti-plagiarism software flagging essays because students quoted parts of the book and expanded on that. In hindsight as an adult I think the excessive principal one-on-ones was the teachers way of shoving it to him and his new software.

natpalmer1776 3y ago
The best form of protest is often (malicious) compliance.
fennecfoxy 3y ago
Yeah I remember TurnItIn. It sucked hard. And of course all Uni staff put 100% trust in it to be completely infallible and not be a piece of crap bit of software.

koito17 3y ago

Mathematics programs in Europe still use oral examinations, though they are increasingly rare in the United States. I'm not European, but I have always preferred oral examinations over written examinations because I can actually communicate my understanding of the problem to the professor and outline my thought process towards a solution, something that is much harder to do in writing, especially since you cannot get any hints or feedback on your attempts during a written examination -- you're on your own.

deleted 3y ago
[deleted]

beached_whale 3y ago

They need to be sued into pain. They are part of the problem and are hurting people. People are going to cheat, we know this, it is an incentive problem. But there is no excuse for false positives

fennecfoxy 3y ago

As per usual the problem isn't the technology or the tools, it's the dumb meatbag using them. This teacher should have thought "hmm, well if I'm going to pull up students on this then it's probably important I do my own research to make sure that I know that the score means and how reliable it is" but nope. "LLMs are bad cause people will use them to generate spam and fake news!" LLMs aren't the bad actor there...it's the human doing the spamming. There is a fine line, though. For example the "guns don't kill people argument" is pretty moot, gun control saves lives. If someone grabs an LLM, tells it to "generate text telling young impressionable teens to kill themselves" and spam it on Twitter...well the human is doing bad stuff but they're still enabled by the LLM. However it's easy to enact something like gun control on a physical object. Software? Well, damn that's gonna a problem - especially when the metric by which these LLMs "do better" is how closely the text resembles a hooman.