4 ms·

I've maintained for a long time that if your exam can be invalidated by the existence of a web browser, its a worthless test that prioritizes arbitrary memoriza

I've maintained for a long time that if your exam can be invalidated by the existence of a web browser, its a worthless test that prioritizes arbitrary memorization over a deeper understanding of the material. Software like Proctorio's proliferation just reinforces the blatant farce and I'm so glad to be out of it.

andrepd 6y ago
It's a fine balance. One one hand you want to let the students google material, consult books, use a calculator... on the other hand given free internet access what stops them from just asking (or paying) someone to solve the exam for them, or googling long enough until they found a solution someone posted online and just copy it, or cribbing from their mates? A good balance, I found (pre covid), was having the students sit a test with textbooks of their choosing and/or their own cheatsheets and lecture notes. No bullshit cramming and memorisation (you can consult literature as you would elsewhere), but also no cheating.
- 0_gravitas 6y ago
  > One one hand you want to let the students google material, consult books, use a calculator... on the other hand given free internet access what stops them from just asking (or paying) someone to solve the exam for them, or googling long enough until they found a solution someone posted online and just copy it, or cribbing from their mates? DING DING DING QUESTION OF THE CENTURY! I don't mean that sarcastically, my point is this: There is a clear, vulgar, mismatch of values here (and there sort of always has been). Shouldn't the purpose of schooling (especially higher education) be learning things, and not maximizing your score on arbitrary measurements of "performance"? I think it obviously should be, but that's not the game, because its non-trivial to measure and counter to the raison detre of the system at large. The very phrase "Competitive School" is a perversion.
  - irjustin 6y ago
    That only makes sense if the goal of college and education is pure learning but we all know it's not. Learning is part of it for sure but Companies use college as a filter is another. Until jobs are unnecessary there's going to be this tug between learning and performing that's never going to good for either. All I'm saying is the answer is never so simple.
    - thwarted 6y ago
      Companies use college as a filter because it's difficult to assess proficiency during an interview and they'd like some way to cull the wheat from the chaff, and avoid wasting time talking to and hiring charlatans and posers. Meanwhile, if you ask google-style brain-teasers during interviews (which can have a legit purpose, but it's become a cliché), people complain about that. On the third hand, if you ask candidates do actual assess-able work during an interview, the extremes are "white board coding" and "take home programming problems", people balk at whatever you do for that too. None of these are necessarily good, nor necessarily better than the others. HN is lousy with complaints about all the different ways to filter and assess candidates.
      - peterwoerner 6y ago
        Sure all the systems suck currently (except hire by internal reference, that one works great). Just because all the systems suck doesn't mean that something better might be possible. And part of finding the something better is understanding what doesn't work about the current systems.
      - eru 6y ago
        Companies use university as a filter for long-term conformity. That's much more pronounced in non-programming jobs, of course. For many white collar jobs you just need any kind of degree, doesn't matter too much in what.
  - mannykannot 6y ago
    Learning how to look up someone else's solutions to problems, or what someone else thinks about something, isn't much of an education. It might well be something to add to the traditional 3Rs, but it would be what you call a mismatch of values to expect a degree for it. This is not intended as an endorsement of Proctorio, and its use of the DMCA in this way would be ridiculous even if the criticisms were without merit.
  - cortesoft 6y ago
    So a formal education is supposed to meet two needs: one, to educate the student. For this goal, we don't need to worry about cheating at all. The second purpose, however, is as a way to signal other people that the student has been educated. This is what a college degree is: a certification that the person has been educated to a particular standard. For that, we need some way to verify that the person has learned what they were supposed to learn. Tests are a way of doing that, and cheating prevents the ability to measure the success of the education.
    - semi-extrinsic 6y ago
      It goes even further than your second point, for many professions: We don't just want to signal other people that doctors, nurses, engineers etc. have been educated. We want to validate that they've actually learned what they need to know in order to not cause uneccessary injuries and death to other people.
  - behringer 6y ago
    Some skills truly require memorization. For example, being a good doctor requires incredible amounts of memorization. A traditional test is a good way to test memorization. I would agree that not all skills require memorization.
  - mlyle 6y ago
    Assessment is a usually necessary part of education. You need to figure out what your class knows. And a fair proportion of your students won't bother to really know stuff unless it'll be formally assessed. And, yes, also there's the issue with granting credentials: you don't want to give them to people who haven't reached a minimum standard. I'm fortunate in that the classes I teach are centered project-based learning and ... not too easy to crib on. But if I were teaching core academic sequences, it's a ridiculous amount of work to make problem sets and exercises of appropriate difficulty and develop rubric for them. You want to be able to leverage other peoples' work, and your past work, in building assessments. And you want to have a time advantage in grading: a 30 minute assessment that you give to 100 students that in turn takes you 30 minutes to grade each is a recipe for no fun. But all of these things-- these economies of scale in instruction-- open the door to cheating. A well-proctored, well-designed exam can mostly patch over the vulnerability.
    - thaumasiotes 6y ago
      > Assessment is a usually necessary part of education. You need to figure out what your class knows. And a fair proportion of your students won't bother to really know stuff unless it'll be formally assessed. I don't think this approach is right. There is an obvious conflict of interest when the same party provides (1) education) and (2) assessment. The better approach is actually that taken by the College Board -- they do assessment, and only assessment. If you'd like to do well on their test, you're free to get whatever training you see fit. And that training is, theoretically[1], evaluated by how you eventually do on the SAT, not by how well you do on practice exams provided by the educator. This should be the practice everywhere else: credentials are given by assessment, and the assessor doesn't do anything other than assessment. If the assessor sells training, that corrupts their assessment. [1] In fact, the system seems to have broken down; SAT training is widely accepted to have large effects, while actual measurements of those effects have difficulty distinguishing them from zero.
      - mlyle 6y ago
        Hey, I love third party and objective measurements. But we can hardly give the SAT every week. Educators need quizzes and other assessment measures. That way, I can know how my class is doing. My students can understand where their performance falls short. And their parents want to know how the class is going along the way. These let all the involved parties adjust for better performance. And these interim assessment steps need to count for at least something to get a reasonable level of effort from students. (My classes are "fun", but that only goes so far: there's the fraction of the material that's intrinsically less fun, and there's the fraction of students that will not do anything without a grade incentive). In turn, these quizzes and other assessment measures that I use, let me eventually score well on the metrics of engagement and scholastic progress that are used assessing me. ;)
  - andrepd 6y ago
    The purpose of school is both learning things, and providing proof that you learned those things. In other words, education and accreditation. Therefore, you need some way to assess that the students did indeed learn the material so you can issue the corresponding certification.
- mbo 6y ago
  The University of New South Wales (UNSW) wrote some case studies on online assessment during the initial Australian lockdown back in March. https://teaching.unsw.edu.au/academic-integrity/case-studies https://teaching.unsw.edu.au/academic-integrity/case-studies Some really interesting results: > In my course’s final online exam, students have access to a Microsoft Teams channel as they complete their exam paper. Students are allowed to ask for help from each other, post specific questions regarding how to solve the problems, post photographs of their working out and indeed their answers. > A near perfect asymmetric Gaussian (bell-curve) distribution was achieved in the final exam with an average mark, pass and HD all being typical for the course were it to be run on face-to-face on campus. Additionally, my course achieved 100.0% satisfaction in the myExperience student survey. My students commented that this assessment approach was a very forward-thinking approach to online examination.
  - ejohnson9912 6y ago
    Thanks for this, hadn't run into this source before. Passing along to some professors!
  - Leherenn 6y ago
    Happened to me a while back, but with an in-person test. They had a French teacher invigilating a biology test, and he didn't care at all. In the end, "every one" was working together, the solutions were written on the blackboard, and every one passed with flying colour, most of them with the same copy since they basically straight copied the blackboard. Of course, the test was redone (with a different invigilator) since the cheating was so obvious. I am not sure how the Australian professor managed to get students to fail in these conditions. Either most of the students were inept and couldn't even cheat properly or the good students didn't want to share their answers. After reading the linked list, that seems to be the case, the chat was monitored and the best students didn't want to share their answers. I think it works fine until someone cross the invisible line, and then widespread cheating ensues. In my case it was also high school vs university.
- raxxorrax 6y ago
  By employing such systems you are teaching not only the content of the class at hand, you also teach that cheating is worthwhile as long as you can trick surveillance and don't get caught. There is also some balance here.
base3 6y ago
This is the difference between education and training. Use of Proctorio is prima facie evidence you're wasting your tuition. (100% original content, hereby placed in the public domain.)
- sildur 6y ago
  I said it before, any education before college is training. Seems like I hit a nerve, as I was downnvoted without explanation. We all went through school and high school, we all know they only want you to memorize the answers, no critical thinking allowed The teacher almost always acts like he is Moses coming down from Mount Sinai. That was my personal experience and everyone's I know.
  - Talanes 6y ago
    I don't think I'm part of "we all," because I only started attending a proper school in high school, but that wasn't my experience at all. There were a lot of teachers who behaved differently and adhered to different teaching philosophies. I find it hard to believe you attended more schools and were exposed to less variety.
    - sildur 6y ago
      Well, I'm honesty happy that my experience and the experience of the people around me is not universal. Maybe our education system was crappy when I was a kid. You gave me hope.
lopmotr 6y ago
If you've been saying the same thing for a long time, you probably don't understand the situation or aren't listening to people's responses and adapting what you say next time. I can see this below where you're rejecting people's valid criticisms in favor of a vague ideology but without having found proven practical solutions to the practical problems. Other people aren't all idiots all the time.
- 0_gravitas 6y ago
  > or aren't listening to people's responses Perhaps I've heard them and remain unconvinced. > a vague ideology but without having found proven practical solutions to the practical problems "You can't point out a flawed system if you don't have a complete and total airtight solution to it yourself, that can be implemented with zero cost and effort by all of society, and of which none would ever dare disagree with as it's so ingenious." > Other people aren't all idiots all the time. When did I say that??

andrepd 6y ago

It's a fine balance. One one hand you want to let the students google material, consult books, use a calculator... on the other hand given free internet access what stops them from just asking (or paying) someone to solve the exam for them, or googling long enough until they found a solution someone posted online and just copy it, or cribbing from their mates? A good balance, I found (pre covid), was having the students sit a test with textbooks of their choosing and/or their own cheatsheets and lecture notes. No bullshit cramming and memorisation (you can consult literature as you would elsewhere), but also no cheating.

0_gravitas 6y ago
> One one hand you want to let the students google material, consult books, use a calculator... on the other hand given free internet access what stops them from just asking (or paying) someone to solve the exam for them, or googling long enough until they found a solution someone posted online and just copy it, or cribbing from their mates? DING DING DING QUESTION OF THE CENTURY! I don't mean that sarcastically, my point is this: There is a clear, vulgar, mismatch of values here (and there sort of always has been). Shouldn't the purpose of schooling (especially higher education) be learning things, and not maximizing your score on arbitrary measurements of "performance"? I think it obviously should be, but that's not the game, because its non-trivial to measure and counter to the raison detre of the system at large. The very phrase "Competitive School" is a perversion.
- irjustin 6y ago
  That only makes sense if the goal of college and education is pure learning but we all know it's not. Learning is part of it for sure but Companies use college as a filter is another. Until jobs are unnecessary there's going to be this tug between learning and performing that's never going to good for either. All I'm saying is the answer is never so simple.
  - thwarted 6y ago
    Companies use college as a filter because it's difficult to assess proficiency during an interview and they'd like some way to cull the wheat from the chaff, and avoid wasting time talking to and hiring charlatans and posers. Meanwhile, if you ask google-style brain-teasers during interviews (which can have a legit purpose, but it's become a cliché), people complain about that. On the third hand, if you ask candidates do actual assess-able work during an interview, the extremes are "white board coding" and "take home programming problems", people balk at whatever you do for that too. None of these are necessarily good, nor necessarily better than the others. HN is lousy with complaints about all the different ways to filter and assess candidates.
    - peterwoerner 6y ago
      Sure all the systems suck currently (except hire by internal reference, that one works great). Just because all the systems suck doesn't mean that something better might be possible. And part of finding the something better is understanding what doesn't work about the current systems.
    - eru 6y ago
      Companies use university as a filter for long-term conformity. That's much more pronounced in non-programming jobs, of course. For many white collar jobs you just need any kind of degree, doesn't matter too much in what.
- mannykannot 6y ago
  Learning how to look up someone else's solutions to problems, or what someone else thinks about something, isn't much of an education. It might well be something to add to the traditional 3Rs, but it would be what you call a mismatch of values to expect a degree for it. This is not intended as an endorsement of Proctorio, and its use of the DMCA in this way would be ridiculous even if the criticisms were without merit.
- cortesoft 6y ago
  So a formal education is supposed to meet two needs: one, to educate the student. For this goal, we don't need to worry about cheating at all. The second purpose, however, is as a way to signal other people that the student has been educated. This is what a college degree is: a certification that the person has been educated to a particular standard. For that, we need some way to verify that the person has learned what they were supposed to learn. Tests are a way of doing that, and cheating prevents the ability to measure the success of the education.
  - semi-extrinsic 6y ago
    It goes even further than your second point, for many professions: We don't just want to signal other people that doctors, nurses, engineers etc. have been educated. We want to validate that they've actually learned what they need to know in order to not cause uneccessary injuries and death to other people.
- behringer 6y ago
  Some skills truly require memorization. For example, being a good doctor requires incredible amounts of memorization. A traditional test is a good way to test memorization. I would agree that not all skills require memorization.
- mlyle 6y ago
  Assessment is a usually necessary part of education. You need to figure out what your class knows. And a fair proportion of your students won't bother to really know stuff unless it'll be formally assessed. And, yes, also there's the issue with granting credentials: you don't want to give them to people who haven't reached a minimum standard. I'm fortunate in that the classes I teach are centered project-based learning and ... not too easy to crib on. But if I were teaching core academic sequences, it's a ridiculous amount of work to make problem sets and exercises of appropriate difficulty and develop rubric for them. You want to be able to leverage other peoples' work, and your past work, in building assessments. And you want to have a time advantage in grading: a 30 minute assessment that you give to 100 students that in turn takes you 30 minutes to grade each is a recipe for no fun. But all of these things-- these economies of scale in instruction-- open the door to cheating. A well-proctored, well-designed exam can mostly patch over the vulnerability.
  - thaumasiotes 6y ago
    > Assessment is a usually necessary part of education. You need to figure out what your class knows. And a fair proportion of your students won't bother to really know stuff unless it'll be formally assessed. I don't think this approach is right. There is an obvious conflict of interest when the same party provides (1) education) and (2) assessment. The better approach is actually that taken by the College Board -- they do assessment, and only assessment. If you'd like to do well on their test, you're free to get whatever training you see fit. And that training is, theoretically[1], evaluated by how you eventually do on the SAT, not by how well you do on practice exams provided by the educator. This should be the practice everywhere else: credentials are given by assessment, and the assessor doesn't do anything other than assessment. If the assessor sells training, that corrupts their assessment. [1] In fact, the system seems to have broken down; SAT training is widely accepted to have large effects, while actual measurements of those effects have difficulty distinguishing them from zero.
    - mlyle 6y ago
      Hey, I love third party and objective measurements. But we can hardly give the SAT every week. Educators need quizzes and other assessment measures. That way, I can know how my class is doing. My students can understand where their performance falls short. And their parents want to know how the class is going along the way. These let all the involved parties adjust for better performance. And these interim assessment steps need to count for at least something to get a reasonable level of effort from students. (My classes are "fun", but that only goes so far: there's the fraction of the material that's intrinsically less fun, and there's the fraction of students that will not do anything without a grade incentive). In turn, these quizzes and other assessment measures that I use, let me eventually score well on the metrics of engagement and scholastic progress that are used assessing me. ;)
- andrepd 6y ago
  The purpose of school is both learning things, and providing proof that you learned those things. In other words, education and accreditation. Therefore, you need some way to assess that the students did indeed learn the material so you can issue the corresponding certification.
mbo 6y ago
The University of New South Wales (UNSW) wrote some case studies on online assessment during the initial Australian lockdown back in March. https://teaching.unsw.edu.au/academic-integrity/case-studies https://teaching.unsw.edu.au/academic-integrity/case-studies Some really interesting results: > In my course’s final online exam, students have access to a Microsoft Teams channel as they complete their exam paper. Students are allowed to ask for help from each other, post specific questions regarding how to solve the problems, post photographs of their working out and indeed their answers. > A near perfect asymmetric Gaussian (bell-curve) distribution was achieved in the final exam with an average mark, pass and HD all being typical for the course were it to be run on face-to-face on campus. Additionally, my course achieved 100.0% satisfaction in the myExperience student survey. My students commented that this assessment approach was a very forward-thinking approach to online examination.
- ejohnson9912 6y ago
  Thanks for this, hadn't run into this source before. Passing along to some professors!
- Leherenn 6y ago
  Happened to me a while back, but with an in-person test. They had a French teacher invigilating a biology test, and he didn't care at all. In the end, "every one" was working together, the solutions were written on the blackboard, and every one passed with flying colour, most of them with the same copy since they basically straight copied the blackboard. Of course, the test was redone (with a different invigilator) since the cheating was so obvious. I am not sure how the Australian professor managed to get students to fail in these conditions. Either most of the students were inept and couldn't even cheat properly or the good students didn't want to share their answers. After reading the linked list, that seems to be the case, the chat was monitored and the best students didn't want to share their answers. I think it works fine until someone cross the invisible line, and then widespread cheating ensues. In my case it was also high school vs university.
raxxorrax 6y ago
By employing such systems you are teaching not only the content of the class at hand, you also teach that cheating is worthwhile as long as you can trick surveillance and don't get caught. There is also some balance here.

base3 6y ago

This is the difference between education and training. Use of Proctorio is prima facie evidence you're wasting your tuition. (100% original content, hereby placed in the public domain.)

sildur 6y ago
I said it before, any education before college is training. Seems like I hit a nerve, as I was downnvoted without explanation. We all went through school and high school, we all know they only want you to memorize the answers, no critical thinking allowed The teacher almost always acts like he is Moses coming down from Mount Sinai. That was my personal experience and everyone's I know.
- Talanes 6y ago
  I don't think I'm part of "we all," because I only started attending a proper school in high school, but that wasn't my experience at all. There were a lot of teachers who behaved differently and adhered to different teaching philosophies. I find it hard to believe you attended more schools and were exposed to less variety.
  - sildur 6y ago
    Well, I'm honesty happy that my experience and the experience of the people around me is not universal. Maybe our education system was crappy when I was a kid. You gave me hope.

lopmotr 6y ago

If you've been saying the same thing for a long time, you probably don't understand the situation or aren't listening to people's responses and adapting what you say next time. I can see this below where you're rejecting people's valid criticisms in favor of a vague ideology but without having found proven practical solutions to the practical problems. Other people aren't all idiots all the time.

0_gravitas 6y ago
> or aren't listening to people's responses Perhaps I've heard them and remain unconvinced. > a vague ideology but without having found proven practical solutions to the practical problems "You can't point out a flawed system if you don't have a complete and total airtight solution to it yourself, that can be implemented with zero cost and effort by all of society, and of which none would ever dare disagree with as it's so ingenious." > Other people aren't all idiots all the time. When did I say that??