5 ms·

I explicitly allow plagiarism and LLMs, but even give them reference solutions if they get stuck. Everything else seems either too much work or ineffective to f

by riedel 1y ago

I explicitly allow plagiarism and LLMs, but even give them reference solutions if they get stuck. Everything else seems either too much work or ineffective to find plagiarism. However, I do a short oral exam where I 'audit' the solution by picking out single lines and asking them to explain it in detail.

threeducks 1y ago
> I do a short oral exam I tried that, but some students who had cheated became really aggressive when it turned out they did not have a single clue how their AI-generated solutions worked and still demanded a pass. We now do more written exams where we have some distance.
- simianwords 1y ago
  Perhaps oral exams leave too much room for ambiguity
  - threeducks 1y ago
    I can understand where you are coming from, but in the cases we had to deal with, there was absolutely no room for ambiguity. To put things into perspective, those were students who did not know what a HashMap is or could not derive the algorithmic complexity of two nested for-loops, yet somehow cheated their way into a master's level course. With the current level of oversight, degrees obtained after 2022 are basically worthless.
    - simianwords 1y ago
      Wow. Why not have written exams? But to be fair, there is no ambiguity in this extreme example but ambiguity can be misused by either party in other cases where it is not so cut and clear.
      - threeducks 1y ago
        > Why not have written exams? That is what we do now. Some reasons why we preferred oral exams before: 1. In order to write a written exam, you have to create a written exam. 2. Scheduling is more difficult. You have to reserve a room for the exam, ensure that the date does not collide with other exams, and students have to plan their vacations around it. For oral exams, you only have to schedule two people, which allows for much greater flexibility. 3. During oral exams, you can probe a bit deeper if you notice that a student did not understand the question correctly, but still knows the solution. In contrast, in a written exam, there is no way for the student to recover from an initially incorrect understanding. 4. It is much easier to cheat on a written exam, especially since there have been all kinds of technological improvements, like ear canal earphones, tiny cameras, glasses with integrated AI assistants, polarized smart watches and who knows what else.
  - izacus 1y ago
    Oral exams are the best way to exam because they allow for ambiguity and are very very hard to fake.
    - criddell 1y ago
      At universities with a large international student population, oral exams can be difficult for those students who don’t speak English well or speak with a very heavy accent. If it’s a more theoretical course, you can probably give a whiteboard exam where they can work out a proof in front of you. Maybe language translation is a case where an AI should be allowed during exams.
- bluenose69 1y ago
  I think written examinations are the only solution, and "distance" is the right word. Also, oral examinations are costly to administer. This work needs to be done with a panel, to avoid complaints. And the panel members cannot include TAs, who are after all just students themselves. A 10-minute examination can have start-up problems, and doesn't allow for in-depth followup questions. With a panel of say 3 professors replacing 1 professor, and interviews of say 1 hour replacing what might be 20 minutes of grading, we are talking about quite a lot of increased workload.
- dgellow 1y ago
  I guess I’m missing something. Isn’t it a direct “failed” if they become aggressive towards the professors?
  - threeducks 1y ago
    Those students had already failed before that, but you still have to get them out of your office somehow.
miki123211 1y ago
I'm sure there were (pre-LLM, non-plagiarized) assignments I turned in at college where I would fail this. "Umm... this is how I've always done it" / "I copied it from my solution for assignment 1 and I no longer remember why I did it that way" / "there was an error and somebody suggested this as a workaround in the Github issue" are all perfectly valid answers in my opinion.

by riedel 1y ago

threeducks 1y ago

> I do a short oral exam I tried that, but some students who had cheated became really aggressive when it turned out they did not have a single clue how their AI-generated solutions worked and still demanded a pass. We now do more written exams where we have some distance.

simianwords 1y ago
Perhaps oral exams leave too much room for ambiguity
- threeducks 1y ago
  I can understand where you are coming from, but in the cases we had to deal with, there was absolutely no room for ambiguity. To put things into perspective, those were students who did not know what a HashMap is or could not derive the algorithmic complexity of two nested for-loops, yet somehow cheated their way into a master's level course. With the current level of oversight, degrees obtained after 2022 are basically worthless.
  - simianwords 1y ago
    Wow. Why not have written exams? But to be fair, there is no ambiguity in this extreme example but ambiguity can be misused by either party in other cases where it is not so cut and clear.
    - threeducks 1y ago
      > Why not have written exams? That is what we do now. Some reasons why we preferred oral exams before: 1. In order to write a written exam, you have to create a written exam. 2. Scheduling is more difficult. You have to reserve a room for the exam, ensure that the date does not collide with other exams, and students have to plan their vacations around it. For oral exams, you only have to schedule two people, which allows for much greater flexibility. 3. During oral exams, you can probe a bit deeper if you notice that a student did not understand the question correctly, but still knows the solution. In contrast, in a written exam, there is no way for the student to recover from an initially incorrect understanding. 4. It is much easier to cheat on a written exam, especially since there have been all kinds of technological improvements, like ear canal earphones, tiny cameras, glasses with integrated AI assistants, polarized smart watches and who knows what else.
- izacus 1y ago
  Oral exams are the best way to exam because they allow for ambiguity and are very very hard to fake.
  - criddell 1y ago
    At universities with a large international student population, oral exams can be difficult for those students who don’t speak English well or speak with a very heavy accent. If it’s a more theoretical course, you can probably give a whiteboard exam where they can work out a proof in front of you. Maybe language translation is a case where an AI should be allowed during exams.
bluenose69 1y ago
I think written examinations are the only solution, and "distance" is the right word. Also, oral examinations are costly to administer. This work needs to be done with a panel, to avoid complaints. And the panel members cannot include TAs, who are after all just students themselves. A 10-minute examination can have start-up problems, and doesn't allow for in-depth followup questions. With a panel of say 3 professors replacing 1 professor, and interviews of say 1 hour replacing what might be 20 minutes of grading, we are talking about quite a lot of increased workload.
dgellow 1y ago
I guess I’m missing something. Isn’t it a direct “failed” if they become aggressive towards the professors?
- threeducks 1y ago
  Those students had already failed before that, but you still have to get them out of your office somehow.

miki123211 1y ago

I'm sure there were (pre-LLM, non-plagiarized) assignments I turned in at college where I would fail this. "Umm... this is how I've always done it" / "I copied it from my solution for assignment 1 and I no longer remember why I did it that way" / "there was an error and somebody suggested this as a workaround in the Github issue" are all perfectly valid answers in my opinion.