5 ms·

The state of remote proctoring software is really, really bad. Students are forced to install software that records video+audio of them, performs facial recogn

by ve55 6y ago

The state of remote proctoring software is really, really bad. Students are forced to install software that records video+audio of them, performs facial recognition, records their every keystroke for analysis, records their browsing history and open applications, aks them to show their entire room to the camera, keeps this data indefinitely with no privacy policy, and sometimes even gets compromised. Of course, sometimes it also doesn't even work, which makes it even worse! There is generally no recourse for students who do not want to do this. Linux students are told to use Windows or fail the class. VM users are told they may be failed if they try to run the software in a VM (which is not allowed by the software and must be REed to attempt to begin with), and students that value their privacy or security (as this software is malware) have no recourse whatsoever often. Regardless, droves of schools continue to spend absurd amounts of their endowment on expensive contracts with these remote proctoring malware companies, desperately attempting to hold up the facade that their degrees have a value that is somewhere near how much they charge in tuition. I feel like I only escaped school recently myself, but it looks like it would be an even more miserable experience if I had to start the process again in 2020.

manquer 6y ago
Disclaimer : one of my products is used for remote proctoring. It depends on the vendor of the exam too, our proctoring sdk works on chrome and Firefox which means all major OSes too, we try hard not go outside browser context, we support android and iOS natively. However many exam apps are very poorly written and have limited support for different OS, and usually never for mobile. I have seen full flash apps used today with 3months to deprecation. we do face detection but not recognition and recommend using human live proctors for high stakes requirements. The OS vendors are doing their part as well, we partner with both Apple and MS and give constant feedback, Apple announced AAC mode this WWDC and MS had number of APIs Linux support is a factor in the education assements space due to chromebooks, most of my devs and half of my ops use linux , so I can say atleast one vendor cares. Today there is profussion of vendors and overnight shops attempting to build in this space post covid-19, sales here is like any other enterprise product , tech merit is only one component, and nuances are lost on the customer in the hurry post covid 19, exam/certificate bodies have little scrutiny and rarely choose the better product for the user. However in few years things will improve, buyers will understand the tech better, vendors will consolidate.
- chaps 6y ago
  Hearing that one of the "vendors that cares" still does facial analysis gives me bad vibes. Can you explain a bit about how the face detection works and what sort of detection data is sent outside the local client? If so, do you store or send it anywhere?
  - manquer 6y ago
    Your feed usually is captured , it is no different than a hangout or zoom recording. Many providers actually use zoom . Face detection is detection of a human face in a feed, not a specific face recognition. Usually such detection is used in negative sense, i.e. lack of face in the feed would prompt for human review. Local context is not feasible, you need to store the feed remotely for compliance in case of dispute etc, it same as video call or a support call that is recorded.
- jimz 6y ago
  I'm actually quite curious as to where you guys end up taking the product, because at least pertaining to the legal profession, as of a few years ago, the entire system from your first proctored exam as a 1L to the bar exam didn't actually require the browser context at all. The UI we used had a heavy Windows 98 aesthetic going on, and mostly just consisted of native text-boxes with pretty minimal formatting tools, timer, and not much of anything else. But at the same time it's also understood on some level everybody knows that the rigamarole is done for its own sake entirely. It's always understood on some level that it's not the substantive law that these exams test for but instead how ably is one able to jump through arbitrary hoops in a prescribed time and format, which is one way of looking at the legal profession as a whole. It's entirely possible that as the vendor you may end up caring more about the exam than the actual test-takers themselves.
ponker 6y ago
what's the alternative to do the best they can to thwart cheating?
- colejohnson66 6y ago
  In person proctoring at a library?
  - manquer 6y ago
    There is an pandemic going on , that would not be feasible today ?
    - deleted 6y ago
      [deleted]
- 10000truths 6y ago
  The solution is to not rely on remote proctoring at all. You ultimately cannot fully monitor an environment you don’t physically control, and in trying to do so, you’ll only end up pissing a lot of people off without doing anything to solve the problem. If potential cheating is such a critically important issue that proctors are ABSOLUTELY necessary, then don’t administer the tests online in the first place. Just augment the testing facility with COVID-19 related precautions, like required masks, hand sanitizer and spaced out desks.
  - manquer 6y ago
    Remote proctoring does not have to be automated. Plenty of tools do it with proctors monitoring the candidate. You don't need it to be at location , you need human proctors . Human proctors are expensive the bar exams are to blame here, if their question papers banks also too narrow so they have to do it the same time. Good exams will give you slots you can choose from , spreads the load. No provider can come up with 30,000 proctors for just a day , even if bar exam boards were ready to pay, which they don't seem to be

by ve55 6y ago

manquer 6y ago

Disclaimer : one of my products is used for remote proctoring. It depends on the vendor of the exam too, our proctoring sdk works on chrome and Firefox which means all major OSes too, we try hard not go outside browser context, we support android and iOS natively. However many exam apps are very poorly written and have limited support for different OS, and usually never for mobile. I have seen full flash apps used today with 3months to deprecation. we do face detection but not recognition and recommend using human live proctors for high stakes requirements. The OS vendors are doing their part as well, we partner with both Apple and MS and give constant feedback, Apple announced AAC mode this WWDC and MS had number of APIs Linux support is a factor in the education assements space due to chromebooks, most of my devs and half of my ops use linux , so I can say atleast one vendor cares. Today there is profussion of vendors and overnight shops attempting to build in this space post covid-19, sales here is like any other enterprise product , tech merit is only one component, and nuances are lost on the customer in the hurry post covid 19, exam/certificate bodies have little scrutiny and rarely choose the better product for the user. However in few years things will improve, buyers will understand the tech better, vendors will consolidate.

chaps 6y ago
Hearing that one of the "vendors that cares" still does facial analysis gives me bad vibes. Can you explain a bit about how the face detection works and what sort of detection data is sent outside the local client? If so, do you store or send it anywhere?
- manquer 6y ago
  Your feed usually is captured , it is no different than a hangout or zoom recording. Many providers actually use zoom . Face detection is detection of a human face in a feed, not a specific face recognition. Usually such detection is used in negative sense, i.e. lack of face in the feed would prompt for human review. Local context is not feasible, you need to store the feed remotely for compliance in case of dispute etc, it same as video call or a support call that is recorded.
jimz 6y ago
I'm actually quite curious as to where you guys end up taking the product, because at least pertaining to the legal profession, as of a few years ago, the entire system from your first proctored exam as a 1L to the bar exam didn't actually require the browser context at all. The UI we used had a heavy Windows 98 aesthetic going on, and mostly just consisted of native text-boxes with pretty minimal formatting tools, timer, and not much of anything else. But at the same time it's also understood on some level everybody knows that the rigamarole is done for its own sake entirely. It's always understood on some level that it's not the substantive law that these exams test for but instead how ably is one able to jump through arbitrary hoops in a prescribed time and format, which is one way of looking at the legal profession as a whole. It's entirely possible that as the vendor you may end up caring more about the exam than the actual test-takers themselves.

ponker 6y ago

what's the alternative to do the best they can to thwart cheating?

colejohnson66 6y ago
In person proctoring at a library?
- manquer 6y ago
  There is an pandemic going on , that would not be feasible today ?
  - deleted 6y ago
    [deleted]
10000truths 6y ago
The solution is to not rely on remote proctoring at all. You ultimately cannot fully monitor an environment you don’t physically control, and in trying to do so, you’ll only end up pissing a lot of people off without doing anything to solve the problem. If potential cheating is such a critically important issue that proctors are ABSOLUTELY necessary, then don’t administer the tests online in the first place. Just augment the testing facility with COVID-19 related precautions, like required masks, hand sanitizer and spaced out desks.
- manquer 6y ago
  Remote proctoring does not have to be automated. Plenty of tools do it with proctors monitoring the candidate. You don't need it to be at location , you need human proctors . Human proctors are expensive the bar exams are to blame here, if their question papers banks also too narrow so they have to do it the same time. Good exams will give you slots you can choose from , spreads the load. No provider can come up with 30,000 proctors for just a day , even if bar exam boards were ready to pay, which they don't seem to be