8 ms·

As a CS professor myself, I don't think ChatGPT really creates entirely new problems for assessment, but rather makes problems that we had been sweeping under t

As a CS professor myself, I don't think ChatGPT really creates entirely new problems for assessment, but rather makes problems that we had been sweeping under the carpet more obvious and harder to ignore. Any assignment that can be cheated by using ChatGPT could already be cheated before by asking a friend, an expert or paying someone else to do it. But most teachers assumed this doesn't happen often, and thus acted as if it wasn't a thing. Making life much simpler and easier. Facing ChatGPT will probably help us make assessment fairer in the long run (and the path is probably going to be a lot of traditional on-site assessment, abandoning the continuous assessment fad that has always been rather disastrous for equity, even if again, life is easier ignoring this).

jraph 4y ago
> the continuous assessment fad that has always been rather disastrous for equity Can you expand on this? I lived the two models as a student and still can't make up my mind on which is best. I feel like it just takes catching some bad cold the week of the exams or a bad exam subject / topic to fail them which seems pretty unfair, but I also know my shit better and have more perspective the week of the exam and prefer not having to stress every week because of the continuous assessment.
- arvidkahl 4y ago
  There is also an option outside "one exam" and "many exams" — and that's "no exams." Obviously, this is spectacularly hard to fit into the current system, but you see this a lot in vocational schools where project-based work makes up the majority of the grade and exams are held to comply with regulatory requirements. ChatGPT might just fit into an educational world that values outcomes over fact retention. One may dream.
  - MengerSponge 4y ago
    Well-structured exams are the single most effective teaching tool available. Assessment is a fringe benefit. I'm a strong advocate for "even more exams", where you take the highest score on particular topics. There are variations, but if you're curious check out "Standards-based grading" or Mastery-based grading. It's in a weird and shitty limbo where a lot of primary and secondary schools have adopted it partially with inadequate training/support, and it's also getting fucked by our shitty publishing industry, same as the botched common-core rollout.
- nemo44x 4y ago
  They’re saying that people that were allowed into their programs/universities because of quotas and not merit struggle to prove competency through assessment.
  - fidgewidge 4y ago
    Wouldn't that be an issue with point in time assessment too?
    - nemo44x 4y ago
      How so? When you’re admitting under powered and under prepared people into a program then they will fail. Unless you dumb down the assessment and curriculum. Which universities have done to make their numbers achieve equity.
      - fidgewidge 4y ago
        Yes but I don't quite get the difference between exams and coursework here.
- Al-Khwarizmi 4y ago
  This is probably an unpopular opinion, but in my view, traditional final exams tend to be a great equalizer. Continuous assessment is affected much more by personal circumstances. For example, continuous assessment tends to force students to devote a constant amount of time throughout the course, which is fine for those who can afford to be full-time students, but disastrous for students who need to have a job to pay the bills. I have seen plenty of students like that getting frustrated because they lose points to this. On the other hand, continuous assessment typically relies on assignments at home and these are deeply unequal and classist as well. The difficulty of these assignments depends on whether the student has someone more experienced to ask, whether they are paying for a private tutor, or whether they can plainly buy the assignment. I have known cases of students that basically bought every project and assignment that could be done at home, you know what made them sweat because it couldn't be bought? The final exam. But almost no one cares about these things, continuous assessment is seen as the "progressive" thing to do because it's more modern and different from tradition, even if it's extremely classist and disproportionately harms working-class students. On the other hand, it is true, as you point out, that the final exam model disadvantages students who have the bad luck of getting sick in that week. It's a real drawback, sadly. I still think it's better all things considered, for the reasons above.
  - jraph 4y ago
    Okay, totally makes sense. Thanks! I wish people didn't have to work during studies. That's one of the biggest reasons of failing.
makstaks 4y ago
As a parent, I'm hoping this will reduce the at-home workloads and shift more work towards school. What's the real life value of having students go to school and then spend all-nighters trying to do homework and all the expected extra-curricular?
- KSteffensen 4y ago
  As I remember my education time from high-school onwards, all-nighters were usually a consequence of slacking off during school time. In college/university a fairly big part of the education was learning what was compulsory reading and what was optional. There was always more reading than hours in the day.
kurthr 4y ago
As a first year grad student, I had a professor in material science who gave the same assignments and mostly same exams every year. Of course a) I didn't know this b) using previous works was prohibited c) cheating was rampant. I only found this out after getting heavily marked down on the first HW and joining a study group which was moderately chaste (eg only used the answers to check our work before turning it in). Missing a single question on HW or the exam would drop your grade a letter, and three or more was likely D/F range. When I brought this up with the professor before the final he at least acted completely shocked as I asked how likely he thought it was that half the class never missed a question (yes half the class got As). Really weird conversation, but most of the class was undergrads, and I think they liked the easy grade with minimal work and so he got 'good ratings'. Undergrad college had 'infinite time' and 'open everything but people' exams, but graded on a curve so that sometimes getting half the questions right was an A. Much more stressful, learned more, and basically no cheating. Certainly, I had one CS class with a newly invented language (compiled). In the days of Fortran, Pascal, and K&R that was kinda par for the course. Not much cheating on that either.
- Aperocky 4y ago
  Learning over the centuries has always taken a character of reciting information, since the ability to do so was valued at the time. With the technologies of today, the ability to recite knowledge (versus using said knowledge when provided, and the ability to look for said knowledge) is rapidly losing value. But our education system has not moved on and are mostly stuck at using recital to judge student capability. A complete rethinking of how to assess student performance is long overdue. For subjects that cannot be assessed without recital, maybe rethink the necessity of existence of those subject for teaching.
  - andsoitis 4y ago
    > With the technologies of today, the ability to recite knowledge (versus using said knowledge when provided, and the ability to look for said knowledge) is rapidly losing value. I didn’t think you are implying that having knowledge (ready to go) is of no value. Having more knowledge gives a huge advantage in time and space. Not only can you solve problems much faster (you don’t need to find, read, understand, and internalize the “knowledge”) you can also make connections that others would miss because to gain knowledge is deeper than mere recall.
deleted 4y ago
[deleted]
yodsanklai 4y ago
> going to be a lot of traditional on-site assessment, abandoning the continuous assessment fad Maybe because I'm from a latin country, but I was always under the impression that only on-site assessment mattered, as continuous assessment provides no signal considering many students cheat. Even what we call "continuous assessment" is done on site.
whazor 4y ago
For my university, not that long ago, it was possible to finish computer science, without touching a computer. Still in my time most exams were done on paper. Think of algorithms in pseudocode. But also a lot of logic like reasoning, with process automata, or relational algebra (databases).
- belter 4y ago
  The graduates from your University must kill it on whiteboard interviews :-)
- deleted 4y ago
  [deleted]
raydev 4y ago
> asking a friend, an expert or paying someone else to do it Surely you understand ChatGPT is infinitely more scalable. How many people can just "ask a friend" or have a spare money to pay someone else multiple times?
- js8 4y ago
  There is also a risk associated with asking another person to be an acomplice. Unlike ChatGPT, they might turn on you.
- Al-Khwarizmi 4y ago
  That's exactly what I'm saying, right? The problem has always been there but it was largely ignored because it wasn't that frequent (although I think it's definitely more frequent than most people make themselves believe). Now that everyone has a "friend" to ask thanks to ChatGPT, people begin to care. How dare the plebs have the same options to cheat as privileged students!

jraph 4y ago

> the continuous assessment fad that has always been rather disastrous for equity Can you expand on this? I lived the two models as a student and still can't make up my mind on which is best. I feel like it just takes catching some bad cold the week of the exams or a bad exam subject / topic to fail them which seems pretty unfair, but I also know my shit better and have more perspective the week of the exam and prefer not having to stress every week because of the continuous assessment.

arvidkahl 4y ago
There is also an option outside "one exam" and "many exams" — and that's "no exams." Obviously, this is spectacularly hard to fit into the current system, but you see this a lot in vocational schools where project-based work makes up the majority of the grade and exams are held to comply with regulatory requirements. ChatGPT might just fit into an educational world that values outcomes over fact retention. One may dream.
- MengerSponge 4y ago
  Well-structured exams are the single most effective teaching tool available. Assessment is a fringe benefit. I'm a strong advocate for "even more exams", where you take the highest score on particular topics. There are variations, but if you're curious check out "Standards-based grading" or Mastery-based grading. It's in a weird and shitty limbo where a lot of primary and secondary schools have adopted it partially with inadequate training/support, and it's also getting fucked by our shitty publishing industry, same as the botched common-core rollout.
nemo44x 4y ago
They’re saying that people that were allowed into their programs/universities because of quotas and not merit struggle to prove competency through assessment.
- fidgewidge 4y ago
  Wouldn't that be an issue with point in time assessment too?
  - nemo44x 4y ago
    How so? When you’re admitting under powered and under prepared people into a program then they will fail. Unless you dumb down the assessment and curriculum. Which universities have done to make their numbers achieve equity.
    - fidgewidge 4y ago
      Yes but I don't quite get the difference between exams and coursework here.
Al-Khwarizmi 4y ago
This is probably an unpopular opinion, but in my view, traditional final exams tend to be a great equalizer. Continuous assessment is affected much more by personal circumstances. For example, continuous assessment tends to force students to devote a constant amount of time throughout the course, which is fine for those who can afford to be full-time students, but disastrous for students who need to have a job to pay the bills. I have seen plenty of students like that getting frustrated because they lose points to this. On the other hand, continuous assessment typically relies on assignments at home and these are deeply unequal and classist as well. The difficulty of these assignments depends on whether the student has someone more experienced to ask, whether they are paying for a private tutor, or whether they can plainly buy the assignment. I have known cases of students that basically bought every project and assignment that could be done at home, you know what made them sweat because it couldn't be bought? The final exam. But almost no one cares about these things, continuous assessment is seen as the "progressive" thing to do because it's more modern and different from tradition, even if it's extremely classist and disproportionately harms working-class students. On the other hand, it is true, as you point out, that the final exam model disadvantages students who have the bad luck of getting sick in that week. It's a real drawback, sadly. I still think it's better all things considered, for the reasons above.
- jraph 4y ago
  Okay, totally makes sense. Thanks! I wish people didn't have to work during studies. That's one of the biggest reasons of failing.

makstaks 4y ago

As a parent, I'm hoping this will reduce the at-home workloads and shift more work towards school. What's the real life value of having students go to school and then spend all-nighters trying to do homework and all the expected extra-curricular?

KSteffensen 4y ago
As I remember my education time from high-school onwards, all-nighters were usually a consequence of slacking off during school time. In college/university a fairly big part of the education was learning what was compulsory reading and what was optional. There was always more reading than hours in the day.

kurthr 4y ago

As a first year grad student, I had a professor in material science who gave the same assignments and mostly same exams every year. Of course a) I didn't know this b) using previous works was prohibited c) cheating was rampant. I only found this out after getting heavily marked down on the first HW and joining a study group which was moderately chaste (eg only used the answers to check our work before turning it in). Missing a single question on HW or the exam would drop your grade a letter, and three or more was likely D/F range. When I brought this up with the professor before the final he at least acted completely shocked as I asked how likely he thought it was that half the class never missed a question (yes half the class got As). Really weird conversation, but most of the class was undergrads, and I think they liked the easy grade with minimal work and so he got 'good ratings'. Undergrad college had 'infinite time' and 'open everything but people' exams, but graded on a curve so that sometimes getting half the questions right was an A. Much more stressful, learned more, and basically no cheating. Certainly, I had one CS class with a newly invented language (compiled). In the days of Fortran, Pascal, and K&R that was kinda par for the course. Not much cheating on that either.

Aperocky 4y ago
Learning over the centuries has always taken a character of reciting information, since the ability to do so was valued at the time. With the technologies of today, the ability to recite knowledge (versus using said knowledge when provided, and the ability to look for said knowledge) is rapidly losing value. But our education system has not moved on and are mostly stuck at using recital to judge student capability. A complete rethinking of how to assess student performance is long overdue. For subjects that cannot be assessed without recital, maybe rethink the necessity of existence of those subject for teaching.
- andsoitis 4y ago
  > With the technologies of today, the ability to recite knowledge (versus using said knowledge when provided, and the ability to look for said knowledge) is rapidly losing value. I didn’t think you are implying that having knowledge (ready to go) is of no value. Having more knowledge gives a huge advantage in time and space. Not only can you solve problems much faster (you don’t need to find, read, understand, and internalize the “knowledge”) you can also make connections that others would miss because to gain knowledge is deeper than mere recall.

deleted 4y ago

[deleted]

yodsanklai 4y ago

> going to be a lot of traditional on-site assessment, abandoning the continuous assessment fad Maybe because I'm from a latin country, but I was always under the impression that only on-site assessment mattered, as continuous assessment provides no signal considering many students cheat. Even what we call "continuous assessment" is done on site.

whazor 4y ago

For my university, not that long ago, it was possible to finish computer science, without touching a computer. Still in my time most exams were done on paper. Think of algorithms in pseudocode. But also a lot of logic like reasoning, with process automata, or relational algebra (databases).

belter 4y ago
The graduates from your University must kill it on whiteboard interviews :-)
deleted 4y ago
[deleted]

raydev 4y ago

> asking a friend, an expert or paying someone else to do it Surely you understand ChatGPT is infinitely more scalable. How many people can just "ask a friend" or have a spare money to pay someone else multiple times?

js8 4y ago
There is also a risk associated with asking another person to be an acomplice. Unlike ChatGPT, they might turn on you.
Al-Khwarizmi 4y ago
That's exactly what I'm saying, right? The problem has always been there but it was largely ignored because it wasn't that frequent (although I think it's definitely more frequent than most people make themselves believe). Now that everyone has a "friend" to ask thanks to ChatGPT, people begin to care. How dare the plebs have the same options to cheat as privileged students!