38 ms·

Students have to jump through absurd hoops to use exam monitoring software

238 points by elsewhen 6y ago

908B64B197 6y ago
Folks are doing take-home exam wrong. Caltech had tremendous success with a culture of honesty and take-home exams.
- eben-ezer 6y ago
  I'm currently taking online classes at a program which has had entirely online offerings for over a decade. Out of the 10+ classes I've taken so far, I'm currently enrolled in the only one that has required a lockdown browser (I haven't taken any tests yet, so I'm not sure what that will be like). Up until now, all the classes just had me sign something saying I didn't cheat. Also, usually the test was designed in such a way as to make it more difficult to cheat. About half of the classes, the tests are open-note. They combat the effectiveness of cheating by making the time of the test short enough to where if you don't know the information, you won't b able to look up more than 25% of the questions before time runs out. I can see how that wouldn't work for all classes, and admittedly I haven't taken any Math or Science courses so it may be different in that arena, but it seems like they have decided to take a different route than the schools mentioned in the article, and presumably haven't had any issues.
  - 5mk 6y ago
    I'm an undergrad studying biochem, and my professors have also made their exams open-notes but timed. There are more short free-response questions than previous years. Also, the questions are focused on synthesizing the material rather than recalling minutiae. I think it's a good compromise, especially since you can't rely on Ctrl+F when you have to reason beyond the material in your own words.
- _8091149529 6y ago
  I spent a few years at Caltech in an academic role (not as a student). I never understood the rationale behind the unsupervised take-home exams. Could you elaborate what is it that makes it a 'success', in your opinion? The shortcomings that seem obvious to me are: - Penalizes honesty - Implied notion that Caltech students are "honourable" and honest. How is this achieved in practice?
  - beisner 6y ago
    The idea is that you create questions that are ungoogleable. Most of my upper-level CS courses had take-homes that were open-book, open-internet, and the professor basically said “good luck finding the answer to this problem online; you will not”. This is harder for professors to do, and maybe fails in intro courses, but generally is quite effective.
    - 908B64B197 6y ago
      This. Else you might as well call the degree "rote" because that's what you measure. > maybe fails in intro courses If someone cheats their way through intro courses they are up for a bad surprise since the upper level courses assumes a mastery of the intro classes!
      - _8091149529 6y ago
        Interesting, thanks for the replies. I'd still point out that the system is not robust against: - Working in groups. (Bad if tests are supposed to assess individual performance.) - Asking outsiders for help. Also, your downplaying of "rote" learning feels misguided, no matter how advanced/abstract/high-level the domain in question is. Cue the Bruce Lee quote about 10,000 kicks..
        turbinerneiter 6y ago
        Just imagine a world were solving problems in groups and asking for outside help is bad. We are so used to exams being about doing stuff that literally nobody in real life would ever even consider.
- CamperBob2 6y ago
  Caltech had tremendous success with a culture of honesty and take-home exams. "Can I use Feynman?"
deleted 6y ago
[deleted]
bitwize 6y ago
Makes me glad for things like the Stevens Honor System, which was how exams were administered at Stevens Institute of Technology. Basically, all work submitted, whether tests or homework, require a pledge with a signature that the work is your own and you didn't cheat. Violations of the honor system -- cheating, plagiarism, etc. could be reported by students or faculty and would be investigated by the Honor Board with disciplinary sanctions for violators. One upside to this was unproctored exams -- although if cheating became widespread, the Honor Board would threaten to bring proctors back! The idea in general was that students were expected to become professional engineers, who are expected to have integrity enough when working in the field not to deliver/sign off on stolen or slipshod work. Proctored exams keep everybody honest while the proctor is watching, but do not address the concerns of personal integrity when you might believe you can get away with cheating.
- aaplok 6y ago
  People are doing that at my institution. The average GPA increased by several points. The thing is, if it's perceived that it's easier to cheat than to learn, people will do that, honor system or not. I admit that there is a difference between an institutionalised honor system, designed with respect toward and in collaboration with students, and a last-minute arrangement by an unprepared organisation. Students have been putting up with a lot of nonsense from universities this year. Their trust might have eroded. If universities are not perceived as honorable themselves, a honors system is not the answer here unfortunately.
  - 908B64B197 6y ago
    > The average GPA increased by several points. The credibility of your school went down by the same amount. If cheating is rampant, either the culture is broken or you are selecting students wrong.
    - aaplok 6y ago
      > The credibility of your school went down by the same amount. That was the point I was trying to make. This is not a problem of a single school, or a single university. This is happening across the sector. The success of contract cheating websites, and the inability of universities worldwide to deal with them, attest to that. > If cheating is rampant, either the culture is broken or you are selecting students wrong. Sure. People are willing to risk jail to get their kids into top universities. A kid whose family borrowed money to send them study in a Western institution might not see honor as their top priority. It is not a student selection problem: the culture is broken all the way down. Lest I come across as pessimistic, I am not. Academics may not be able to fix the structural problems, but we have more control over our own little part of our world than in the private sector. I, for one, don't rely on exams anymore for the time being and have not seen a GPA increase in my course. A honor-based exam system is not something I rely on, however.
cosmie 6y ago
Back in the good ol' days, you could just take the exam inside of a VM. The VM is locked down, but the host isn't. Looks like from ProctorU's docs[1] that they got around to probing for the (blatantly obvious[2]) signatures of a VM and stopped that trick. I wonder if those signatures exist in a Windows Sandbox[3] instance, or if you can detect/block the Enhanced RDP Session Windows leverages when running a Sandbox... [1] Under "Not Supported" at https://support.proctoru.com/hc/en-us/articles/115011772748-Equipment-Requirements https://support.proctoru.com/hc/en-us/articles/115011772748-... [2] https://bannedit.github.io/Virtual-Machine-Detection-In-The-Browser.html https://bannedit.github.io/Virtual-Machine-Detection-In-The-... [3] https://docs.microsoft.com/en-us/windows/security/threat-protection/windows-sandbox/windows-sandbox-overview https://docs.microsoft.com/en-us/windows/security/threat-pro...
- ColanR 6y ago
  I wonder how feasible it is to remove such fingerprints - find a way to make the VM be functionally indistinguishable from the host.
  - conductor 6y ago
    This is an old problem for reverse engineers, and there are some solutions, for example VBoxHardenedLoader [0]. As per usual with such things, these tools are in cat/mouse category. [0] https://github.com/hfiref0x/VBoxHardenedLoader https://github.com/hfiref0x/VBoxHardenedLoader
    - ColanR 6y ago
      That's interesting - I would have guessed that after a certain point, a VM and a physical computer are indistinguishable from the POV of software running inside.
      - lights0123 6y ago
        It's easy to make the core device indistinguishable. It's the additional hardware that becomes a problem: QEMU/libvirt hardcodes its hard drive model to include the word "QEMU" (although there is an unmerged patch to make that configurable), and although I don't know how graphics work in a VM, it's far easier to create a higher-level graphics device than e.g. emulating Intel graphics.
type0 6y ago
Proctored exams are as smooth as having proctologist to look remotely through your webcam
lxe 6y ago
I took a remote amateur radio exam twice. The coordinator (proctor) asked to share the whole screen, scan the room with another device, and that's it! Just enough steps to have a degree of assurance that I'm not cheating, but not going overboard.
- usrusr 6y ago
  Examiners don't need to absolutely rule out a cheat, they just need to be sure that a cheating examinee would not be able to absolutely rule out cheat detection. Demonstrating absence of cheats isn't like some captcha that you just take again and again if you fail, as often as you like.
aliceryhl 6y ago
My only computer has Linux installed on it. I wouldn't be able to take an exam that required a windows computer. If you make exams online, you should change the format to one that is better suited to people having access to the internet. They did that with our exams, and it worked perfectly fine, sidestepping the issues with these exams.
- Zeturic 6y ago
  This is a good point. My Organic Chemistry class had tests that were open-book, open-note, open-internet. Pretty much the only thing you couldn't do was talk to other students. Even if you tried to contact other students by the internet (which they didn't even try to police), there were several versions of each exam, so you were unlikely to find help. They were still by far the hardest exams I took at University, because the questions were crafted such that you couldn't easily find the answers. Trying to look up a question in the book or online was more likely to waste test time than get you the answer.
  - asdff 6y ago
    That's the thing with ochem, if you don't know it right away it could take you 4 hours even with the book in front of you. 50% got me a B+ in that class with the curve. Humbling.
askvictor 6y ago
A solution for both this, and the problems with exams in general, is oral exams. Student has to answer questions from the teacher, in real time, one-on-one. Takes longer, but is much more authentic.
- chrisseaton 6y ago
  I think in the US some class sizes are in the hundreds, so this does not seem scalable for their case.
  - space_fountain 6y ago
    Some but for my college at least very few. I think I took one class with more than 100 people in it
    - deleted 6y ago
      [deleted]
  - nkrisc 6y ago
    If they're having trouble proctoring their exams, perhaps they're admitting too many students.
    - chrisseaton 6y ago
      I think proctoring in-person was fine. The issue is only because it's remote.
  - asdff 6y ago
    In all my 300 student classes, we had recitation sections with maybe 20 students to a TA or teaching professor to go over the homework/take quizzes/serve as extended office hours beyond the time the professor and this TA already made available themselves every week. It wouldn't be much more work for the TA to give each student in their section a 1 on 1 hour meeting over the course of a week or two.
- 6y ago
poulsbohemian 6y ago
I just went through this kind of experience on Friday, taking my Oregon real estate license exam (I passed, now will be dual licensed in WA and OR!). The having to scan the room, scan my ears, scan my "desk" (or in this case my crotch because I was using a laptop) was all very silly, but didn't really bother me. The worst parts were all very human - they sent me the wrong link for the test, shared incorrect information about the allowed materials (IE: scratch paper, a 4 function calculator, etc), and then I waited for more than 20 minutes for the proctor to finally show up. Then of course there's the problem that the test provider has wonky weird software, including forcing you to install their "secure" test software. Which then yells at you for having a browser open that it necessitated to start the install process. I guess what I'm really saying is - I find it funny that this is deemed "secure" when really I suspect most people taking a test have the same emotions as myself - we just wanna get to it and get it done.
- reustle 6y ago
  Congrats on passing, by the way. Nice work.
cloud5ir 6y ago
To me, this underscores notion that schools continue to test for the wrong thing. With the exception of professions/trades that require impulse application of knowledge, I don't know why I would want/need someone to memorize a concept in order to apply it. Productive members of our society/workforce think critically, they ask good questions, and they research and leverage the information at their disposal to make data-driven decisions. Let's test for that.
- Enginerrrd 6y ago
  Thinking critically on the subject: it also prevents the problem of having someone take the test for you by feeding you information through an earpiece or chatbox on a browser tab. In places where testing fraud is rampant, this is a common form of fraud actually. So even a well designed
- bawolff 6y ago
  From what i remember of uni, most of the tests i took were not testing if you memorized a concept, but if you could apply it. Sure the situation was still extremely contrived, but it wasn't simply regurgitating facts
  - kenniskrag 6y ago
    Same in university of applied science in switzerland. For example you learn a depth first search and in the exam you have to store some information on each step to print it in the end. You have to learn different proof schemes and use the easiest one for a given hypothesis. Sometimes you get with the wrong one in an endless recursion for example.
- bllguo 6y ago
  while this sounds good on the surface, i think memorization is much more important than you give it credit. imo you absolutely should have key concepts memorized if you have any real understanding of a subject. and as you become more of a specialist, your bar for what constitutes a "key concept" should raise. Being able to search things up is well and good, but dont you run into situations where you don't even know what to search for in the first place?
maerF0x0 6y ago
I had a teacher that solved this by saying all exams were open book, and then put so many questions that if you didnt know the material you'd just run out of time. Basically you could look up a very low number of things in the amount of time per question allotted
- chillwaves 6y ago
  Some people just work more slowly than others. Doesn't seem like a great solution.
  - deleted 6y ago
    [deleted]
  - hombre_fatal 6y ago
    Most of my exams were like that in lower school and there was always enough time if you knew the material. It wasn’t open ended essay. An exam that has too many people timing out just isn’t a well balanced exam.
  - smabie 6y ago
    And the real world penalizes people for that, for good reason. As such, I don't think there's anything wrong with time pressure on tests.
    - chillwaves 6y ago
      That's not what the test is supposed to measure.
  - ntsplnkv2 6y ago
    Some people are slower runners, that doesn't mean we should slow down everyone else on a ball field.
    - chillwaves 6y ago
      A person taking a test more slowly has no effect on the people who finish more quickly.
bArray 6y ago
Can we not just replace exams for meaningful coursework? We already have pretty decent software for testing uniqueness, etc. This does of course put the burden back onto the teaching staff, give them a nice bonus to mark the work. Given school can shutdown their buildings they can save on costs such as heating, air conditioning, cleaning, electric, gas, maintenance, etc - there should be a little extra budget left-over for a marking bonus.
6gvONxR4sf7o 6y ago
We must do all things at scale and perfectly standardized! Only multiple choice tests that can be graded by a machine! Make sure the teachers never have a one on one conversation with students! It’s easy if we make sure they don’t have time and resources for that. Imagine how much harder interviewing would be without having a one on one conversation. And if it had to be perfectly standardized nationwide. It would have a ton less bias, but you’d have a terrible signal and have to do all these crazy things to prevent cheating. In other words, these seem even worse than the state of tech interviewing.
musicale 6y ago
Dealing with cheating is a complete drag, but I would still never use any of this nonsense if I were teaching a class. It's horribly intrusive and dedicated cheaters are still likely to figure out a workaround. I'd much rather just assign take-home exams and final projects, or use a synchronous 1-hour exam that everyone takes at the same time.
aarong11 6y ago
With a lot more people taking courses remotely, I expect some universities will take the approach Amazon does and have "test centers" with proctors to make sure there are no shenanigans. That, and a lot more coursework / open book exams instead of traditional tests.
Green_man 6y ago
Online classes really emphasize how irrelevant closed book/notes/neighbor tests have always been. In cute terms, this environment is the strongest possible counterexample to "you won't always have a calculator with you!". All students are literally doing the test on a computer with internet access in the present tense. Their connectivity is considered so robust that a student may lose points or time if their internet drops during the test, and yet open internet access (or even just textbook and notes!) is somehow not an obvious resource to allow students during tests. Professors frequently copy tests from other sources or previous semesters, and this obviously saves an enormous amount of effort, but ironically their copied work is what makes cheating on their tests so much more accessible. Professors make loud noises about cheating, and some are so hyperfixated on preventing cheating that they demand ridiculous requirements like proctoring software. They clearly aren't really prioritizing cheating prevention in their efforts, otherwise they'd write new tests more frequently (or at all!). Cheating prevention measures that don't require greater instructor effort are far more commonplace. When a professor literally copies a test, they shouldn't be surprised to see their students doing the same. Tests that are suddenly ineffective with open internet access are already complete garbage for 3 reasons: 1. Some students will always cheat, and as long as teaching is remote, some students will always have internet access. Open access means that honesty is no longer discouraged. 2. Verifying granular details/vocab on the internet is completely normal in real work, even among experts. 3. Any test without resources like notes or internet relies far too heavily on memorization. This might be justifiable if there were evidence that standard test studying strategies (cramming) were effective at guaranteeing long term memorization. Anecdotally, I would fail the majority of my finals from previous semesters without time to (re)prepare. Students should be encouraged to understand the inner workings of the topic, in terms of analysis and potential states of that system, not to memorize dozens of details and vocabulary terms. Within my Computer Science curriculum, the most effective and important content (between tests, projects, and lectures) has been programming projects by far. I'm dubious that tests are effective for any curriculum with relevant real world projects, though CS may be better than other subjects at making projects the primary measurement of understanding, and replacing the rigor that testing creates.
- 3np 6y ago
  Yeah, if anything the first thing this makes me think is "how could I circumvent this?", on a test I otherwise would have just gone with and not even considered cheating. Maybe it's the hacker mind in me, or maybe these things actually induce more cheating than they prevent...
Avi-D-coder 6y ago
At what point can we stop pretending most universities prepare you for the job market? If your a professor that cares about Google give a rapid oral exam. If your a college professor giving multiple choice or other memorization based formats, what value are you providing over free online courses? Frankly, if your institution "has" to use this sort of software, it's not worth attending.
- notRobot 6y ago
  While I agree with you that most universities don't generally prepare you for the job market (especially when it comes to CS), this is not an option for almost all students: > if your institution "has" to use this sort of software, it's not worth attending Because of the sheer number of applicants in many places, whether a person has a relevant college degree and how much they scored in college is often the first "filter" jobs apply while sorting candidates, and if you can't get past that, it's essentially impossible for you to get a job. Students will stop going to college for courses that don't prepare them for the job market when the job market stops requiring them to do so, which definitely isn't the case right now.
  - marcthe12 6y ago
    Maybe we could Split up degrees into courses allow them fully independent. At least that can help alot because now it allow mixing and matching
nerdbaggy 6y ago
Here if a fun thing. In chrome whenever you grant we website to use your camera and mic you now have that web site access to view all cameras and listen to all mics on your computer at the same time
olyjohn 6y ago
I feel so bad for the students that have to deal with this. I worked at a Community College in IT for a while. Students would come into a lab that was specifically set up for testing. The anti-cheating software was such total trash, it would crash, crash the browser, crash the whole system. Students would have to start testing over... At least twice a month, their systems would be down, and nobody could take tests at all. We'd have to send home 30-60 students at a time, and reschedule their tests for another day. These were students who were also working jobs, and took time off of work, likely unpaid, to come in and take the test. So we wasted their time, and lost them money. Not to mention the constant waste of IT's time dealing with this trash software. It's totally fucking criminal that these companies sell such garbage to our schools and get away with it. It doesn't matter how bad it is, it checks a box on some requirements sheet so we will never switch to something else... Not that the other options are any better.
- danmur 6y ago
  This could be describing plenty of enterprise software too. Boxticking crapware.
  - reustle 6y ago
    Sounds like we just need to push for an "uptime" checkbox, where uptime means tests can be taken.
- powersnail 6y ago
  What kind of cheat does it prevent? I imagine it wouldn't be terribly hard to run the machines so that only one full-screen window is shown, or a user for whom the internet is disabled. Is that not enough?
  - Shared404 6y ago
    > I imagine it wouldn't be terribly hard to run the machines so that only one full-screen window is shown, or a user for whom the internet is disabled. Is that not enough? Many of the anti-cheating software will also run the webcam and yell at you if it thinks you aren't paying attention to only the screen, while recording everything on your screen. And students are being forced to install this on their home machines.
peeters 6y ago
> one WLU professor wrote that anyone who wished to use foam noise-cancelling ear plugs must “in plain view of your webcam … place the ear plugs on your desk and use a hard object to hit each ear plug before putting it in your ear—if they are indeed just foam ear plugs they will not be harmed.” Nitpick but noise-cancelling is by definition an active electronic process, and so will never be a purely foam ear plug. This is Vice misreporting what the professor said (professor's words were "ear plugs for noise reduction").
ehvatum 6y ago
Jesus Christ, as a student, I am absolutely positive that I would spend 100% of my effort fucking with this software. This is “The lessons being taught are not the ones being learned” in extremis. Do we really want our best and brightest to have such practical education in subversive techno-syndicatetry?! I would be proud as a parent, but I’m also a bad parent. Edit: my plan is to home school in a lumber mill / cnc machine shop. The strong will survive.
liminalsunset 6y ago
Hypothetical question: What if the client side of this program was open-source, and all it did was record the screen and web camera, and upload a copy to a server that was run by the school on premises. Would this change your view on the software or make you more likely to agree, however begrudgingly, with the use of this kind of program?
lakecresva 6y ago
As bad as all of the proctoring software seems to be, I wish we had it. I'm a 1L at an American law school that's doing entirely unproctored exams. I didn't come to the state with any connections, nor am I friends with any upperclassmen. I already know I'm going to get fucking annihilated by people who are cheating with their 2L/3L buddies, because there's literally zero risk of getting caught. In light of how heavily your 1L grades affect your career prospects, it seems insane that they're going to basically sort people by their performance on unproctored take-home exams with no integrity whatsoever.
- aaplok 6y ago
  I am really sorry this is happening to you. The situation is deeply unfair, and it affects the fair and square overwhelmingly. If that is any consolation, keep in mind that both you and they have a long way to go before graduation. Exams at your level are meant to assess whether students are ready for advanced classes. Cheating on them does little more than removing that safeguard, which can very well backfire. You're doing the right thing if you focus on learning the basics.
bakuninsbart 6y ago
I recently wrote a coding exam at uni, the software would randomly delete 'unnecessary' whitespace. We mostly code in python.
dmitrygr 6y ago
I know a few people in college now, and my advice to them is to run all this crap in a VM. That way it can be easily and harmlessly blown away without affecting the workability of their main learning tool (computer)
freshhawk 6y ago
I've heard some good stories of VMs and obs to circumvent all this insane stuff. It's good. I remember learning about networks to pirate stuff back in the day and that's much less of a thing now. Now kids have their own incredibly stupid invented problem they can easily work around if they learn some tech stuff, and a lot of them are familiar with obs from streaming anyway.
auscompgeek 6y ago
A random tidbit I've heard floating around: Examplify (yet another proctor) will think that you're running in a VM if you have WSL installed.
- monocasa 6y ago
  You sort of are. The main OS is virtualized too under WSL2. Both it and WSL2's kernels running on top of hyper-v.
  - tremon 6y ago
    That doesn't apply only to WSL2. As soon as you install the Hyper-V role (for WSL2 or to virtualize other VMs), your main OS becomes a VM with special hardware privileges (the Root Partition, in Hyper-V parlance), on top of the Hyper-V hypervisor kernel.
georgianar 6y ago
This Man brought my Lover back In less than 3 days, i saw wonders, my Lover came back to me and my life got back just like a completed puzzle… am so happy.. thanks to Robinson.buckler@ {{yahoo}} com,, for saving my life. Mr Robinson is the best I’ve ever met! He also cured us from Herpes with his herbal remedy thanks ……https:// https:// r o b i n b u c k l e r. com…………^" _____________________________
whywhywhywhy 6y ago
The older I get the more absurd the years of my life wasted in schooling become. Being tested by people who mostly have never even worked the job you're supposedly preparing for using methods that don't represent the way you'll actually be working seems a complete waste of time and money. Increasingly the more talented individuals I'm working with in my job I see more and more that never even went through any of that in the first place. I can imagine a way that it would be valuable but really you'd have to start from scratch, there is too much cruft.
- musicale 6y ago
  > The older I get the more absurd the years of my life wasted in schooling become The more time I spend working in my chosen field, the more I appreciate the time I spent studying unrelated subjects.
  - the_only_law 6y ago
    I dont think school would have done me much either way, but I really do wish I put more effort into learning certain subjects I wrote off as useless a teenager.

238 points by elsewhen 6y ago

908B64B197 6y ago

Folks are doing take-home exam wrong. Caltech had tremendous success with a culture of honesty and take-home exams.

eben-ezer 6y ago
I'm currently taking online classes at a program which has had entirely online offerings for over a decade. Out of the 10+ classes I've taken so far, I'm currently enrolled in the only one that has required a lockdown browser (I haven't taken any tests yet, so I'm not sure what that will be like). Up until now, all the classes just had me sign something saying I didn't cheat. Also, usually the test was designed in such a way as to make it more difficult to cheat. About half of the classes, the tests are open-note. They combat the effectiveness of cheating by making the time of the test short enough to where if you don't know the information, you won't b able to look up more than 25% of the questions before time runs out. I can see how that wouldn't work for all classes, and admittedly I haven't taken any Math or Science courses so it may be different in that arena, but it seems like they have decided to take a different route than the schools mentioned in the article, and presumably haven't had any issues.
- 5mk 6y ago
  I'm an undergrad studying biochem, and my professors have also made their exams open-notes but timed. There are more short free-response questions than previous years. Also, the questions are focused on synthesizing the material rather than recalling minutiae. I think it's a good compromise, especially since you can't rely on Ctrl+F when you have to reason beyond the material in your own words.
_8091149529 6y ago
I spent a few years at Caltech in an academic role (not as a student). I never understood the rationale behind the unsupervised take-home exams. Could you elaborate what is it that makes it a 'success', in your opinion? The shortcomings that seem obvious to me are: - Penalizes honesty - Implied notion that Caltech students are "honourable" and honest. How is this achieved in practice?
- beisner 6y ago
  The idea is that you create questions that are ungoogleable. Most of my upper-level CS courses had take-homes that were open-book, open-internet, and the professor basically said “good luck finding the answer to this problem online; you will not”. This is harder for professors to do, and maybe fails in intro courses, but generally is quite effective.
  - 908B64B197 6y ago
    This. Else you might as well call the degree "rote" because that's what you measure. > maybe fails in intro courses If someone cheats their way through intro courses they are up for a bad surprise since the upper level courses assumes a mastery of the intro classes!
    - _8091149529 6y ago
      Interesting, thanks for the replies. I'd still point out that the system is not robust against: - Working in groups. (Bad if tests are supposed to assess individual performance.) - Asking outsiders for help. Also, your downplaying of "rote" learning feels misguided, no matter how advanced/abstract/high-level the domain in question is. Cue the Bruce Lee quote about 10,000 kicks..
      - turbinerneiter 6y ago
        Just imagine a world were solving problems in groups and asking for outside help is bad. We are so used to exams being about doing stuff that literally nobody in real life would ever even consider.
CamperBob2 6y ago
Caltech had tremendous success with a culture of honesty and take-home exams. "Can I use Feynman?"

deleted 6y ago

[deleted]

bitwize 6y ago

Makes me glad for things like the Stevens Honor System, which was how exams were administered at Stevens Institute of Technology. Basically, all work submitted, whether tests or homework, require a pledge with a signature that the work is your own and you didn't cheat. Violations of the honor system -- cheating, plagiarism, etc. could be reported by students or faculty and would be investigated by the Honor Board with disciplinary sanctions for violators. One upside to this was unproctored exams -- although if cheating became widespread, the Honor Board would threaten to bring proctors back! The idea in general was that students were expected to become professional engineers, who are expected to have integrity enough when working in the field not to deliver/sign off on stolen or slipshod work. Proctored exams keep everybody honest while the proctor is watching, but do not address the concerns of personal integrity when you might believe you can get away with cheating.

aaplok 6y ago
People are doing that at my institution. The average GPA increased by several points. The thing is, if it's perceived that it's easier to cheat than to learn, people will do that, honor system or not. I admit that there is a difference between an institutionalised honor system, designed with respect toward and in collaboration with students, and a last-minute arrangement by an unprepared organisation. Students have been putting up with a lot of nonsense from universities this year. Their trust might have eroded. If universities are not perceived as honorable themselves, a honors system is not the answer here unfortunately.
- 908B64B197 6y ago
  > The average GPA increased by several points. The credibility of your school went down by the same amount. If cheating is rampant, either the culture is broken or you are selecting students wrong.
  - aaplok 6y ago
    > The credibility of your school went down by the same amount. That was the point I was trying to make. This is not a problem of a single school, or a single university. This is happening across the sector. The success of contract cheating websites, and the inability of universities worldwide to deal with them, attest to that. > If cheating is rampant, either the culture is broken or you are selecting students wrong. Sure. People are willing to risk jail to get their kids into top universities. A kid whose family borrowed money to send them study in a Western institution might not see honor as their top priority. It is not a student selection problem: the culture is broken all the way down. Lest I come across as pessimistic, I am not. Academics may not be able to fix the structural problems, but we have more control over our own little part of our world than in the private sector. I, for one, don't rely on exams anymore for the time being and have not seen a GPA increase in my course. A honor-based exam system is not something I rely on, however.

cosmie 6y ago

Back in the good ol' days, you could just take the exam inside of a VM. The VM is locked down, but the host isn't. Looks like from ProctorU's docs[1] that they got around to probing for the (blatantly obvious[2]) signatures of a VM and stopped that trick. I wonder if those signatures exist in a Windows Sandbox[3] instance, or if you can detect/block the Enhanced RDP Session Windows leverages when running a Sandbox... [1] Under "Not Supported" at https://support.proctoru.com/hc/en-us/articles/115011772748-Equipment-Requirements https://support.proctoru.com/hc/en-us/articles/115011772748-... [2] https://bannedit.github.io/Virtual-Machine-Detection-In-The-Browser.html https://bannedit.github.io/Virtual-Machine-Detection-In-The-... [3] https://docs.microsoft.com/en-us/windows/security/threat-protection/windows-sandbox/windows-sandbox-overview https://docs.microsoft.com/en-us/windows/security/threat-pro...

ColanR 6y ago
I wonder how feasible it is to remove such fingerprints - find a way to make the VM be functionally indistinguishable from the host.
- conductor 6y ago
  This is an old problem for reverse engineers, and there are some solutions, for example VBoxHardenedLoader [0]. As per usual with such things, these tools are in cat/mouse category. [0] https://github.com/hfiref0x/VBoxHardenedLoader https://github.com/hfiref0x/VBoxHardenedLoader
  - ColanR 6y ago
    That's interesting - I would have guessed that after a certain point, a VM and a physical computer are indistinguishable from the POV of software running inside.
    - lights0123 6y ago
      It's easy to make the core device indistinguishable. It's the additional hardware that becomes a problem: QEMU/libvirt hardcodes its hard drive model to include the word "QEMU" (although there is an unmerged patch to make that configurable), and although I don't know how graphics work in a VM, it's far easier to create a higher-level graphics device than e.g. emulating Intel graphics.

type0 6y ago

Proctored exams are as smooth as having proctologist to look remotely through your webcam

lxe 6y ago

I took a remote amateur radio exam twice. The coordinator (proctor) asked to share the whole screen, scan the room with another device, and that's it! Just enough steps to have a degree of assurance that I'm not cheating, but not going overboard.

usrusr 6y ago
Examiners don't need to absolutely rule out a cheat, they just need to be sure that a cheating examinee would not be able to absolutely rule out cheat detection. Demonstrating absence of cheats isn't like some captcha that you just take again and again if you fail, as often as you like.

aliceryhl 6y ago

My only computer has Linux installed on it. I wouldn't be able to take an exam that required a windows computer. If you make exams online, you should change the format to one that is better suited to people having access to the internet. They did that with our exams, and it worked perfectly fine, sidestepping the issues with these exams.

Zeturic 6y ago
This is a good point. My Organic Chemistry class had tests that were open-book, open-note, open-internet. Pretty much the only thing you couldn't do was talk to other students. Even if you tried to contact other students by the internet (which they didn't even try to police), there were several versions of each exam, so you were unlikely to find help. They were still by far the hardest exams I took at University, because the questions were crafted such that you couldn't easily find the answers. Trying to look up a question in the book or online was more likely to waste test time than get you the answer.
- asdff 6y ago
  That's the thing with ochem, if you don't know it right away it could take you 4 hours even with the book in front of you. 50% got me a B+ in that class with the curve. Humbling.

askvictor 6y ago

A solution for both this, and the problems with exams in general, is oral exams. Student has to answer questions from the teacher, in real time, one-on-one. Takes longer, but is much more authentic.

chrisseaton 6y ago
I think in the US some class sizes are in the hundreds, so this does not seem scalable for their case.
- space_fountain 6y ago
  Some but for my college at least very few. I think I took one class with more than 100 people in it
  - deleted 6y ago
    [deleted]
- nkrisc 6y ago
  If they're having trouble proctoring their exams, perhaps they're admitting too many students.
  - chrisseaton 6y ago
    I think proctoring in-person was fine. The issue is only because it's remote.
- asdff 6y ago
  In all my 300 student classes, we had recitation sections with maybe 20 students to a TA or teaching professor to go over the homework/take quizzes/serve as extended office hours beyond the time the professor and this TA already made available themselves every week. It wouldn't be much more work for the TA to give each student in their section a 1 on 1 hour meeting over the course of a week or two.
6y ago

poulsbohemian 6y ago

I just went through this kind of experience on Friday, taking my Oregon real estate license exam (I passed, now will be dual licensed in WA and OR!). The having to scan the room, scan my ears, scan my "desk" (or in this case my crotch because I was using a laptop) was all very silly, but didn't really bother me. The worst parts were all very human - they sent me the wrong link for the test, shared incorrect information about the allowed materials (IE: scratch paper, a 4 function calculator, etc), and then I waited for more than 20 minutes for the proctor to finally show up. Then of course there's the problem that the test provider has wonky weird software, including forcing you to install their "secure" test software. Which then yells at you for having a browser open that it necessitated to start the install process. I guess what I'm really saying is - I find it funny that this is deemed "secure" when really I suspect most people taking a test have the same emotions as myself - we just wanna get to it and get it done.

reustle 6y ago
Congrats on passing, by the way. Nice work.

cloud5ir 6y ago

To me, this underscores notion that schools continue to test for the wrong thing. With the exception of professions/trades that require impulse application of knowledge, I don't know why I would want/need someone to memorize a concept in order to apply it. Productive members of our society/workforce think critically, they ask good questions, and they research and leverage the information at their disposal to make data-driven decisions. Let's test for that.

Enginerrrd 6y ago
Thinking critically on the subject: it also prevents the problem of having someone take the test for you by feeding you information through an earpiece or chatbox on a browser tab. In places where testing fraud is rampant, this is a common form of fraud actually. So even a well designed
bawolff 6y ago
From what i remember of uni, most of the tests i took were not testing if you memorized a concept, but if you could apply it. Sure the situation was still extremely contrived, but it wasn't simply regurgitating facts
- kenniskrag 6y ago
  Same in university of applied science in switzerland. For example you learn a depth first search and in the exam you have to store some information on each step to print it in the end. You have to learn different proof schemes and use the easiest one for a given hypothesis. Sometimes you get with the wrong one in an endless recursion for example.
bllguo 6y ago
while this sounds good on the surface, i think memorization is much more important than you give it credit. imo you absolutely should have key concepts memorized if you have any real understanding of a subject. and as you become more of a specialist, your bar for what constitutes a "key concept" should raise. Being able to search things up is well and good, but dont you run into situations where you don't even know what to search for in the first place?

maerF0x0 6y ago

I had a teacher that solved this by saying all exams were open book, and then put so many questions that if you didnt know the material you'd just run out of time. Basically you could look up a very low number of things in the amount of time per question allotted

chillwaves 6y ago
Some people just work more slowly than others. Doesn't seem like a great solution.
- deleted 6y ago
  [deleted]
- hombre_fatal 6y ago
  Most of my exams were like that in lower school and there was always enough time if you knew the material. It wasn’t open ended essay. An exam that has too many people timing out just isn’t a well balanced exam.
- smabie 6y ago
  And the real world penalizes people for that, for good reason. As such, I don't think there's anything wrong with time pressure on tests.
  - chillwaves 6y ago
    That's not what the test is supposed to measure.
- ntsplnkv2 6y ago
  Some people are slower runners, that doesn't mean we should slow down everyone else on a ball field.
  - chillwaves 6y ago
    A person taking a test more slowly has no effect on the people who finish more quickly.

bArray 6y ago

Can we not just replace exams for meaningful coursework? We already have pretty decent software for testing uniqueness, etc. This does of course put the burden back onto the teaching staff, give them a nice bonus to mark the work. Given school can shutdown their buildings they can save on costs such as heating, air conditioning, cleaning, electric, gas, maintenance, etc - there should be a little extra budget left-over for a marking bonus.

6gvONxR4sf7o 6y ago

We must do all things at scale and perfectly standardized! Only multiple choice tests that can be graded by a machine! Make sure the teachers never have a one on one conversation with students! It’s easy if we make sure they don’t have time and resources for that. Imagine how much harder interviewing would be without having a one on one conversation. And if it had to be perfectly standardized nationwide. It would have a ton less bias, but you’d have a terrible signal and have to do all these crazy things to prevent cheating. In other words, these seem even worse than the state of tech interviewing.

musicale 6y ago

Dealing with cheating is a complete drag, but I would still never use any of this nonsense if I were teaching a class. It's horribly intrusive and dedicated cheaters are still likely to figure out a workaround. I'd much rather just assign take-home exams and final projects, or use a synchronous 1-hour exam that everyone takes at the same time.

aarong11 6y ago

With a lot more people taking courses remotely, I expect some universities will take the approach Amazon does and have "test centers" with proctors to make sure there are no shenanigans. That, and a lot more coursework / open book exams instead of traditional tests.

Green_man 6y ago

Online classes really emphasize how irrelevant closed book/notes/neighbor tests have always been. In cute terms, this environment is the strongest possible counterexample to "you won't always have a calculator with you!". All students are literally doing the test on a computer with internet access in the present tense. Their connectivity is considered so robust that a student may lose points or time if their internet drops during the test, and yet open internet access (or even just textbook and notes!) is somehow not an obvious resource to allow students during tests. Professors frequently copy tests from other sources or previous semesters, and this obviously saves an enormous amount of effort, but ironically their copied work is what makes cheating on their tests so much more accessible. Professors make loud noises about cheating, and some are so hyperfixated on preventing cheating that they demand ridiculous requirements like proctoring software. They clearly aren't really prioritizing cheating prevention in their efforts, otherwise they'd write new tests more frequently (or at all!). Cheating prevention measures that don't require greater instructor effort are far more commonplace. When a professor literally copies a test, they shouldn't be surprised to see their students doing the same. Tests that are suddenly ineffective with open internet access are already complete garbage for 3 reasons: 1. Some students will always cheat, and as long as teaching is remote, some students will always have internet access. Open access means that honesty is no longer discouraged. 2. Verifying granular details/vocab on the internet is completely normal in real work, even among experts. 3. Any test without resources like notes or internet relies far too heavily on memorization. This might be justifiable if there were evidence that standard test studying strategies (cramming) were effective at guaranteeing long term memorization. Anecdotally, I would fail the majority of my finals from previous semesters without time to (re)prepare. Students should be encouraged to understand the inner workings of the topic, in terms of analysis and potential states of that system, not to memorize dozens of details and vocabulary terms. Within my Computer Science curriculum, the most effective and important content (between tests, projects, and lectures) has been programming projects by far. I'm dubious that tests are effective for any curriculum with relevant real world projects, though CS may be better than other subjects at making projects the primary measurement of understanding, and replacing the rigor that testing creates.

3np 6y ago
Yeah, if anything the first thing this makes me think is "how could I circumvent this?", on a test I otherwise would have just gone with and not even considered cheating. Maybe it's the hacker mind in me, or maybe these things actually induce more cheating than they prevent...

Avi-D-coder 6y ago

At what point can we stop pretending most universities prepare you for the job market? If your a professor that cares about Google give a rapid oral exam. If your a college professor giving multiple choice or other memorization based formats, what value are you providing over free online courses? Frankly, if your institution "has" to use this sort of software, it's not worth attending.

notRobot 6y ago
While I agree with you that most universities don't generally prepare you for the job market (especially when it comes to CS), this is not an option for almost all students: > if your institution "has" to use this sort of software, it's not worth attending Because of the sheer number of applicants in many places, whether a person has a relevant college degree and how much they scored in college is often the first "filter" jobs apply while sorting candidates, and if you can't get past that, it's essentially impossible for you to get a job. Students will stop going to college for courses that don't prepare them for the job market when the job market stops requiring them to do so, which definitely isn't the case right now.
- marcthe12 6y ago
  Maybe we could Split up degrees into courses allow them fully independent. At least that can help alot because now it allow mixing and matching

nerdbaggy 6y ago

Here if a fun thing. In chrome whenever you grant we website to use your camera and mic you now have that web site access to view all cameras and listen to all mics on your computer at the same time

olyjohn 6y ago

I feel so bad for the students that have to deal with this. I worked at a Community College in IT for a while. Students would come into a lab that was specifically set up for testing. The anti-cheating software was such total trash, it would crash, crash the browser, crash the whole system. Students would have to start testing over... At least twice a month, their systems would be down, and nobody could take tests at all. We'd have to send home 30-60 students at a time, and reschedule their tests for another day. These were students who were also working jobs, and took time off of work, likely unpaid, to come in and take the test. So we wasted their time, and lost them money. Not to mention the constant waste of IT's time dealing with this trash software. It's totally fucking criminal that these companies sell such garbage to our schools and get away with it. It doesn't matter how bad it is, it checks a box on some requirements sheet so we will never switch to something else... Not that the other options are any better.

danmur 6y ago
This could be describing plenty of enterprise software too. Boxticking crapware.
- reustle 6y ago
  Sounds like we just need to push for an "uptime" checkbox, where uptime means tests can be taken.
powersnail 6y ago
What kind of cheat does it prevent? I imagine it wouldn't be terribly hard to run the machines so that only one full-screen window is shown, or a user for whom the internet is disabled. Is that not enough?
- Shared404 6y ago
  > I imagine it wouldn't be terribly hard to run the machines so that only one full-screen window is shown, or a user for whom the internet is disabled. Is that not enough? Many of the anti-cheating software will also run the webcam and yell at you if it thinks you aren't paying attention to only the screen, while recording everything on your screen. And students are being forced to install this on their home machines.

peeters 6y ago

> one WLU professor wrote that anyone who wished to use foam noise-cancelling ear plugs must “in plain view of your webcam … place the ear plugs on your desk and use a hard object to hit each ear plug before putting it in your ear—if they are indeed just foam ear plugs they will not be harmed.” Nitpick but noise-cancelling is by definition an active electronic process, and so will never be a purely foam ear plug. This is Vice misreporting what the professor said (professor's words were "ear plugs for noise reduction").

ehvatum 6y ago

Jesus Christ, as a student, I am absolutely positive that I would spend 100% of my effort fucking with this software. This is “The lessons being taught are not the ones being learned” in extremis. Do we really want our best and brightest to have such practical education in subversive techno-syndicatetry?! I would be proud as a parent, but I’m also a bad parent. Edit: my plan is to home school in a lumber mill / cnc machine shop. The strong will survive.

liminalsunset 6y ago

Hypothetical question: What if the client side of this program was open-source, and all it did was record the screen and web camera, and upload a copy to a server that was run by the school on premises. Would this change your view on the software or make you more likely to agree, however begrudgingly, with the use of this kind of program?

lakecresva 6y ago

As bad as all of the proctoring software seems to be, I wish we had it. I'm a 1L at an American law school that's doing entirely unproctored exams. I didn't come to the state with any connections, nor am I friends with any upperclassmen. I already know I'm going to get fucking annihilated by people who are cheating with their 2L/3L buddies, because there's literally zero risk of getting caught. In light of how heavily your 1L grades affect your career prospects, it seems insane that they're going to basically sort people by their performance on unproctored take-home exams with no integrity whatsoever.

aaplok 6y ago
I am really sorry this is happening to you. The situation is deeply unfair, and it affects the fair and square overwhelmingly. If that is any consolation, keep in mind that both you and they have a long way to go before graduation. Exams at your level are meant to assess whether students are ready for advanced classes. Cheating on them does little more than removing that safeguard, which can very well backfire. You're doing the right thing if you focus on learning the basics.

bakuninsbart 6y ago

I recently wrote a coding exam at uni, the software would randomly delete 'unnecessary' whitespace. We mostly code in python.

dmitrygr 6y ago

I know a few people in college now, and my advice to them is to run all this crap in a VM. That way it can be easily and harmlessly blown away without affecting the workability of their main learning tool (computer)

freshhawk 6y ago

I've heard some good stories of VMs and obs to circumvent all this insane stuff. It's good. I remember learning about networks to pirate stuff back in the day and that's much less of a thing now. Now kids have their own incredibly stupid invented problem they can easily work around if they learn some tech stuff, and a lot of them are familiar with obs from streaming anyway.

auscompgeek 6y ago

A random tidbit I've heard floating around: Examplify (yet another proctor) will think that you're running in a VM if you have WSL installed.

monocasa 6y ago
You sort of are. The main OS is virtualized too under WSL2. Both it and WSL2's kernels running on top of hyper-v.
- tremon 6y ago
  That doesn't apply only to WSL2. As soon as you install the Hyper-V role (for WSL2 or to virtualize other VMs), your main OS becomes a VM with special hardware privileges (the Root Partition, in Hyper-V parlance), on top of the Hyper-V hypervisor kernel.

georgianar 6y ago

This Man brought my Lover back In less than 3 days, i saw wonders, my Lover came back to me and my life got back just like a completed puzzle… am so happy.. thanks to Robinson.buckler@ {{yahoo}} com,, for saving my life. Mr Robinson is the best I’ve ever met! He also cured us from Herpes with his herbal remedy thanks ……https:// https:// r o b i n b u c k l e r. com…………^" _____________________________

whywhywhywhy 6y ago

The older I get the more absurd the years of my life wasted in schooling become. Being tested by people who mostly have never even worked the job you're supposedly preparing for using methods that don't represent the way you'll actually be working seems a complete waste of time and money. Increasingly the more talented individuals I'm working with in my job I see more and more that never even went through any of that in the first place. I can imagine a way that it would be valuable but really you'd have to start from scratch, there is too much cruft.

musicale 6y ago
> The older I get the more absurd the years of my life wasted in schooling become The more time I spend working in my chosen field, the more I appreciate the time I spent studying unrelated subjects.
- the_only_law 6y ago
  I dont think school would have done me much either way, but I really do wish I put more effort into learning certain subjects I wrote off as useless a teenager.