7 ms·

Recent gatech OMSCS graduate here. I fail to recognize any real issue with proctoring. I have used proctortrack and honorlock. * proctoring software can be ins

Recent gatech OMSCS graduate here. I fail to recognize any real issue with proctoring. I have used proctortrack and honorlock. * proctoring software can be installed before an exam and uninstalled afterwards. It doesn’t need to stay on your pc/browser, and its activation is very obvious. I have no info about permanent security issues caused by such software. * what proctoring does is mostly recording video and audio (possibly with room scans at the beginning ) and uploading it to a remote server for later verification. Then, such footage is automatically scanned for anomalies. The teacher can then check what happened in flagged content parts and choose what to do. The system does not impose a “cheater” label by itself. * proctoring systems don’t force a closed book approach. I took open book exams where the only enforcement was “be alone and no collaboration with other people during the exam”. It seems that some people are concerned with some very specific details of some implementations, or with shitty teachers that say some people cheated just because they were flagged for whatever reason. Some things that happened to me and I still passed the exam: * a coworker accidentally entered the room where I was taking the exam and I had to talk with him to send him away. * I had to change my position since I had setup my laptop in a way that was unbearable for a many-hours exam, and I briefly exited the webcam view. I contacted the teachers when this happened and I got no issue at all. Of course some teachers happened to be silly, at the beginning of the pandemics: I heard things like “no drinking, no eating, no restroom” for 4 hours exams. Blame the people, not the software.

reayn 5y ago
1. The software should’ve never existed, it’s a testimony to the failure of our education system. 2. You are far luckier than most when it comes to your circumstances, i have seen myself people failing exams for similar mishaps to yours.
- alanfranz 5y ago
  1. Explain why. 2. Shitty instructors are not the software’s fault.
  - reayn 5y ago
    1. Re-stating what half the comments on this thread are already saying would be a waste of my time. 2. This has nothing to do with instructors (in my experience instructors have little to no say or participation in the usage of these systems), badly written algorithms will frequently kick students out of tests for non-adverse behavior or failure in facial recognition (once again oft-mentioned in this thread).
    - alanfranz 5y ago
      1. I can’t read your mind. I assume you’re unwilling or unable to discuss the issue further. 2. Some issues, as I said, are specific to certain implementations and don’t invalidate eproctoring as a whole.
kevincox 5y ago
I think the main problem is that these are all malware by definition. They are grabbing all of the data they can get and the users are not the ones who what that. I could install this on a separate user account which should restrict its access to my stuff but it is still a lot to ask. And a lot of these software requires admin access, which is absolutely not ok even for a second on my personal PC. If the school wants to ship me their PC with it installed I have very little complaints other than the biases in the video analysis. There are other concerns, such as they often only support a few proprietary operating systems, however many programs require these anyways, so that isn't a major step back in most cases.
nonameiguess 5y ago
I was a TA in the first OMSCS cohort. I think the biggest difference between the OMSCS experience and what other people are giving here as anecdotes is Georgia Tech was prepared for this. They created an online program on purpose that was well thought-out and planned in advance. Many universities that did this in the past year, in contrast, were forced into it by the pandemic and scrambling to do whatever they could with no real plan, bad or no policies in place, professors and TAs that were not properly prepared. Though I actually agree here with the people saying this type of software is usually a rootkit by design and unacceptable for that reason alone. I understand why the school did it, but in my experience, we never caught a person cheating on an exam. Where we caught cheating was in project submissions, and that was often quite blatant with people turning in identical code or code identical to code on the Internet, sometimes even identical comments. No need for malware on student's personal home networks to catch that. There is somewhat of a fundamental conflict in the structure of the OMSCS courses, though. Personally, I don't think there is much value in even having the exams at all on top of the usually extremely comprehensive practical project assignments, for the terminal masters students. But that's everyone in OMSCS. The exams tend to test your knowledge of prior research, which is mostly valuable for people going on to a PhD afterward. Of course, some students do that, even if you need to separately apply after completing the terminal masters and can't just move straight into it like the on-campus grad students. So I don't really know the answer, but the structure right now is imposing a heavily research-focused curriculum on students who are overwhelmingly not going into research. And the OMSCS program largely exists because of sponsorship from industry that wanted stronger industry-focused CS education for their own workers from a school more reputable than a plain MOOC with no university behind it. They do an amazingly good job of best of both worlds right now, but it is a conflict as I think many of the exams are unnecessarily difficult for people who are never going to become researchers. I guess it's the classic Google approach to hiring, though. You end up filtering a bunch of students who may have been perfectly good as programmers, but you're probably never going to let anyone through who wasn't, and that is ultimately more important for maintaining the school's strong reputation in industry.
poplarstand 5y ago
Current (almost finished) OMSA student here. I agree that the tools themselves aren't terrible, but I do believe they're an example of "doing the wrong thing faster". My claim would be that all time-limited exams are bad news. 1. The strictness and quality of exams varies significantly by instructor (as you mentioned in your post) 2. Exams aren't representative of any real-world analogue (how often does your boss lock you in a room and tell you to solve the Lagrangian by hand, alone, under a time limit) 3. They don't actually stop determined cheaters (anyone with the foresight to put up notesheets in their bathroom) In short, proctored exams do not effectively assess student proficiency in a subject, and do not effectively prevent cheating. Making proctoring more scalable means propogating a bad practice at ever-greater rates. What I would suggest instead, and what many classes in OMSA have already done, is to leverage project-based assessments. * Demonstrates learning by actually using the material, instead of hollow repeat-backs * Encourages further learning since practice entails dealing with real-world complications * Helps students develop a portfolio so they have more to show at graduation than transcripts There are many other options though if that particular solution doesn't tickle your fancy.
mannerheim 5y ago
> * proctoring software can be installed before an exam and uninstalled afterwards. It doesn’t need to stay on your pc/browser, and its activation is very obvious. I have no info about permanent security issues caused by such software. I wouldn't trust a computer that had spyware installed on it without a full system wipe.
- alanfranz 5y ago
  Are there any actual accounts of security incidents related to such software?
  - mannerheim 5y ago
    Spyware created by a corporation to spy on customers? That was what the Sony rootkit scandal was about: https://en.wikipedia.org/wiki/Sony_BMG_copy_protection_rootkit_scandal https://en.wikipedia.org/wiki/Sony_BMG_copy_protection_rootk... > Soon after Russinovich's first post, there were several trojans and worms exploiting XCP's security holes.
    - alanfranz 5y ago
      No. Security incidents related to eproctoring software that was willingly installed.
mindslight 5y ago
> proctoring software can be installed before an exam and uninstalled afterwards. It doesn’t need to stay on your pc/browser, and its activation is very obvious. I have no info about permanent security issues caused by such software. The permanent security issue is that violating a security assertion (eg running an untrusted program) on your machine means you can never trust that machine until it gets completely wiped. I'm well past the age where I have to worry about bullshit like eproctoring and my computing environment has become much more deliberate, but even in college I was running Linux as my primary. Reading how it's being normalized that in order to pass a class you're forced to install random spyware on your own machine gives me the willies. This is not what we should be teaching kids for information security practice!
- cesarb 5y ago
  > The permanent security issue is that violating a security assertion (eg running an untrusted program) on your machine means you can never trust that machine until it gets completely wiped. That hasn't been the case ever since firmware became updatable. Nowadays, even wiping the machine completely is not enough, because the firmware might have been maliciously modified by whatever violated the security boundary. And it might not even be the firmware you'd expect (the SPI chip containing the UEFI); see the recent post here (https://news.ycombinator.com/item?id=27740295 https://news.ycombinator.com/item?id=27740295) about someone who modified his laptop's EC firmware and his laptop's wireless card firmware, making the first intercept all key presses (the EC is also the keyboard controller) and send them to the second to be broadcast over the radio, completely invisible to both the operating system and the UEFI firmware.
  - mindslight 5y ago
    I hadn't wanted to go into that level of detail, but you're right to. My own definition of wipe includes reflashing libreboot.
    - cesarb 5y ago
      My point was, even reflashing libreboot wouldn't be enough, you'd have to also reflash all the small controllers here and there (hoping you hadn't missed some), and somehow do it in a way each controller cannot lie to you and just pretend it has been reflashed.
908B64B197 5y ago
> proctoring software can be installed before an exam and uninstalled afterwards. It doesn’t need to stay on your pc/browser, and its activation is very obvious. I have no info about permanent security issues caused by such software. If it installs drivers, I have news for you...
- alanfranz 5y ago
  It doesn’t. At least proctortrack and honorlock Don’t. Btw: implementation details. It doesn’t invalidate all eproctoring solutions.

reayn 5y ago

1. The software should’ve never existed, it’s a testimony to the failure of our education system. 2. You are far luckier than most when it comes to your circumstances, i have seen myself people failing exams for similar mishaps to yours.

alanfranz 5y ago
1. Explain why. 2. Shitty instructors are not the software’s fault.
- reayn 5y ago
  1. Re-stating what half the comments on this thread are already saying would be a waste of my time. 2. This has nothing to do with instructors (in my experience instructors have little to no say or participation in the usage of these systems), badly written algorithms will frequently kick students out of tests for non-adverse behavior or failure in facial recognition (once again oft-mentioned in this thread).
  - alanfranz 5y ago
    1. I can’t read your mind. I assume you’re unwilling or unable to discuss the issue further. 2. Some issues, as I said, are specific to certain implementations and don’t invalidate eproctoring as a whole.

kevincox 5y ago

I think the main problem is that these are all malware by definition. They are grabbing all of the data they can get and the users are not the ones who what that. I could install this on a separate user account which should restrict its access to my stuff but it is still a lot to ask. And a lot of these software requires admin access, which is absolutely not ok even for a second on my personal PC. If the school wants to ship me their PC with it installed I have very little complaints other than the biases in the video analysis. There are other concerns, such as they often only support a few proprietary operating systems, however many programs require these anyways, so that isn't a major step back in most cases.

nonameiguess 5y ago

I was a TA in the first OMSCS cohort. I think the biggest difference between the OMSCS experience and what other people are giving here as anecdotes is Georgia Tech was prepared for this. They created an online program on purpose that was well thought-out and planned in advance. Many universities that did this in the past year, in contrast, were forced into it by the pandemic and scrambling to do whatever they could with no real plan, bad or no policies in place, professors and TAs that were not properly prepared. Though I actually agree here with the people saying this type of software is usually a rootkit by design and unacceptable for that reason alone. I understand why the school did it, but in my experience, we never caught a person cheating on an exam. Where we caught cheating was in project submissions, and that was often quite blatant with people turning in identical code or code identical to code on the Internet, sometimes even identical comments. No need for malware on student's personal home networks to catch that. There is somewhat of a fundamental conflict in the structure of the OMSCS courses, though. Personally, I don't think there is much value in even having the exams at all on top of the usually extremely comprehensive practical project assignments, for the terminal masters students. But that's everyone in OMSCS. The exams tend to test your knowledge of prior research, which is mostly valuable for people going on to a PhD afterward. Of course, some students do that, even if you need to separately apply after completing the terminal masters and can't just move straight into it like the on-campus grad students. So I don't really know the answer, but the structure right now is imposing a heavily research-focused curriculum on students who are overwhelmingly not going into research. And the OMSCS program largely exists because of sponsorship from industry that wanted stronger industry-focused CS education for their own workers from a school more reputable than a plain MOOC with no university behind it. They do an amazingly good job of best of both worlds right now, but it is a conflict as I think many of the exams are unnecessarily difficult for people who are never going to become researchers. I guess it's the classic Google approach to hiring, though. You end up filtering a bunch of students who may have been perfectly good as programmers, but you're probably never going to let anyone through who wasn't, and that is ultimately more important for maintaining the school's strong reputation in industry.

poplarstand 5y ago

Current (almost finished) OMSA student here. I agree that the tools themselves aren't terrible, but I do believe they're an example of "doing the wrong thing faster". My claim would be that all time-limited exams are bad news. 1. The strictness and quality of exams varies significantly by instructor (as you mentioned in your post) 2. Exams aren't representative of any real-world analogue (how often does your boss lock you in a room and tell you to solve the Lagrangian by hand, alone, under a time limit) 3. They don't actually stop determined cheaters (anyone with the foresight to put up notesheets in their bathroom) In short, proctored exams do not effectively assess student proficiency in a subject, and do not effectively prevent cheating. Making proctoring more scalable means propogating a bad practice at ever-greater rates. What I would suggest instead, and what many classes in OMSA have already done, is to leverage project-based assessments. * Demonstrates learning by actually using the material, instead of hollow repeat-backs * Encourages further learning since practice entails dealing with real-world complications * Helps students develop a portfolio so they have more to show at graduation than transcripts There are many other options though if that particular solution doesn't tickle your fancy.

mannerheim 5y ago

> * proctoring software can be installed before an exam and uninstalled afterwards. It doesn’t need to stay on your pc/browser, and its activation is very obvious. I have no info about permanent security issues caused by such software. I wouldn't trust a computer that had spyware installed on it without a full system wipe.

alanfranz 5y ago
Are there any actual accounts of security incidents related to such software?
- mannerheim 5y ago
  Spyware created by a corporation to spy on customers? That was what the Sony rootkit scandal was about: https://en.wikipedia.org/wiki/Sony_BMG_copy_protection_rootkit_scandal https://en.wikipedia.org/wiki/Sony_BMG_copy_protection_rootk... > Soon after Russinovich's first post, there were several trojans and worms exploiting XCP's security holes.
  - alanfranz 5y ago
    No. Security incidents related to eproctoring software that was willingly installed.

mindslight 5y ago

> proctoring software can be installed before an exam and uninstalled afterwards. It doesn’t need to stay on your pc/browser, and its activation is very obvious. I have no info about permanent security issues caused by such software. The permanent security issue is that violating a security assertion (eg running an untrusted program) on your machine means you can never trust that machine until it gets completely wiped. I'm well past the age where I have to worry about bullshit like eproctoring and my computing environment has become much more deliberate, but even in college I was running Linux as my primary. Reading how it's being normalized that in order to pass a class you're forced to install random spyware on your own machine gives me the willies. This is not what we should be teaching kids for information security practice!

cesarb 5y ago
> The permanent security issue is that violating a security assertion (eg running an untrusted program) on your machine means you can never trust that machine until it gets completely wiped. That hasn't been the case ever since firmware became updatable. Nowadays, even wiping the machine completely is not enough, because the firmware might have been maliciously modified by whatever violated the security boundary. And it might not even be the firmware you'd expect (the SPI chip containing the UEFI); see the recent post here (https://news.ycombinator.com/item?id=27740295 https://news.ycombinator.com/item?id=27740295) about someone who modified his laptop's EC firmware and his laptop's wireless card firmware, making the first intercept all key presses (the EC is also the keyboard controller) and send them to the second to be broadcast over the radio, completely invisible to both the operating system and the UEFI firmware.
- mindslight 5y ago
  I hadn't wanted to go into that level of detail, but you're right to. My own definition of wipe includes reflashing libreboot.
  - cesarb 5y ago
    My point was, even reflashing libreboot wouldn't be enough, you'd have to also reflash all the small controllers here and there (hoping you hadn't missed some), and somehow do it in a way each controller cannot lie to you and just pretend it has been reflashed.

908B64B197 5y ago

alanfranz 5y ago
It doesn’t. At least proctortrack and honorlock Don’t. Btw: implementation details. It doesn’t invalidate all eproctoring solutions.