7 ms·

I still have yet to hear anyone propose a workable solution for the ultimate problem of remote assessment in CS: hiring someone to just do your work for you. Ev

• Frondo 6y ago
  Could something like pair programming work to reduce the risk of hiring out the whole project?
  • acbart 6y ago
    Reduces the risk, but doesn't prevent it.
• GhostVII 6y ago
  You could make them give a video presentation about their project to show that they actually know how it works, not foolproof but it helps. I think it is pretty much impossible to completely remove that possibility though. I've had lots of courses where the majority of my mark is assignments and projects, and I'm sure some students pay others to do it for them, and the professors just accept that as a possibility. I don't think it is worth providing a worse experience to the majority of honest students to prevent a small minority of dishonest students from cheating.
• deleted 6y ago
  [deleted]
• vulcan01 6y ago
  An interesting way to assess your students, if you have time and a reasonable amount of students, is to do what the IB organisation required my Spanish teacher to do: a one-on-one oral exam with presentation and follow-up questions from the teacher, all recorded and sent to the IBO grading board.
  • acbart 6y ago
    I'm 1-on-1 interviewing my 59 Algorithms students this semester. 20 minute interviews, and I have 2.5 TAs to help offset the workload. Starts on Wednesday, I'm absolutely dreading it. This would never scale to my 180 Intro students that I'll be getting this fall.
    • wombatpm 6y ago
      But doesn't the whole 101 CS classes really not scale in general. I remember having code submission submitted to to some anti-plagiarism system back in the 90's.
      • acbart 6y ago
        I like to the curriculum I'd been developing for the past few years was scaling pretty well. Most of the technology is reaching enough stability that I can give a ton of automated feedback without killing myself, I had a pretty low DFW rate, most students could complete my final programming problems in a proctored exam situation, and evaluations came back high. I wasn't happy with my project rubric scores, but I think I was on the way to progress with that and it was partially just too high expectations (I just want students to test and decompose more). I'm interested in how we can scale introductory computing experiences, and I think I was on a good path before they yanked out my proctored exams out from under me :)
• chrisseaton 6y ago
  > I still have yet to hear anyone propose a workable solution for the ultimate problem of remote assessment in CS What about 1-on-1 interviews? Ask people the questions face-to-face, ask them to talk you through their answers. That's how advanced degrees like PhDs are assessed.
  • NullPrefix 6y ago
    1on1's take too long and don't InternetScale^tm
    • deleted 6y ago
      [deleted]
  • indigochill 6y ago
    This is also what my team does for professional interviews. We give them a take-home test with some simple problems, but we actually care much more about how they talk through their solution with us than the solution itself. We don't even care if they copy/pasted stuff from Stack Overflow. What we care about is that they can demonstrate they understand the problem and solution. The code is just the starting point for that conversation. We have hired people who submitted failing solutions because they were able to think through the problem on the spot when we told them it failed and asked why.
  • acbart 6y ago
    I'm 1-on-1 interviewing my 59 Algorithms students this semester. 20 minute interviews, and I have 2.5 TAs to help offset the workload. Starts on Wednesday, I'm absolutely dreading it. This would never scale to my 180 Intro students that I'll be getting this fall.
    • turndown 6y ago
      What are the 59 algorithms?
      • chrisseaton 6y ago
• whymauri 6y ago
  Some companies, like Pinterest, use systems like "lytmus" to prevent cheating. It's a monitored VM box where you do the entire assignment inside the GUI VM. I'm sure certain trust factor guarantees prevent you from outsourcing the assignment overseas. All that said, it's an awful system and I ragequit the take-home midway through. It's buggy, slow, lacks useful hotkeys, and basically requires you to become accustomed to a wiped Linux box without any of the tools you would typically use. Sure, you can install them - but it's a timed assessment, for crying out loud. Finally, I suppose nothing stops you from logging in with your credentials on your authorized machine and then physically handing that machine to a paid agent.
  • derefr 6y ago
    > I'm sure certain trust factor guarantees prevent you from outsourcing the assignment overseas. Who said anything about overseas? Hire your classmate to sit down at your own PC to do it. That's the Hard Problem.
    • marcinzm 6y ago
      I heard some companies, possibly only in India, required a webcam to be on the whole time and for you to rotate it around the room beforehand.
      • noodlesUK 6y ago
        Yeah and this stops me pre-recording it how? I have a USB capture card that appears to the host machine just as a UVC webcam.
        • 9nGQluzmnq3M 6y ago
          Ask a few questions, including at least one that's unpredictable.
        • ceejayoz 6y ago
          "Touch your left cheek. Now show me the ceiling. Smile at the camera. Blink three times quickly." If you can pre-record stuff like that without knowing what the sequence of requested events will be, I'm impressed.
• jackson1442 6y ago
  It's definitely a problem, even on CollegeBoard's current platform. Normal AP exams/SATs require photo ID validation at the exam site, and are run by school admins who will notice if someone is out of place. I work as a tutor on Wyzant and have been asked to "tutor" during the exam as well as to just do CS Principles students' portfolio assignments. While some might argue my approach is too aggressive, I report every account that messages me asking to cheat to Wyzant as well as the appropriate organization. My goal is to make it simply not worth the attempt to cheat (obviously only if they explicitly ask; I don't report when it's a possible miscommunication). Just about all of them have had their accounts deactivated. If technology access could be assumed, I don't think it would be unreasonable to ask students to upload a picture of themselves with their photo ID, as well as some sort of validity check, like a unique code only provided to the device they're taking the exam on. Obviously, that can't be implemented because of equity issues. Ideally, a secured assessment would be done on something similar to Pearson Vue, ProctorU, or the multitude of other online proctoring services. Unfortunately, that's not an option due to both scale and technology access (not everyone has a compatible device, I'm sure many students are taking the exam on their phones).
• elliekelly 6y ago
  Do you have them upload the source code of their projects? If so, after the upload could you display the project’s file tree along with a last minute change their hypothetical client has requested and give them a minute or two to select the file(s) that would need a pull request in order to make the change? They don’t even need to make the change, just identify where the change would happen. Someone who wrote the code themselves will know right away and someone who purchased a project won’t have enough time to sift through the code to figure out the answer.
  • acbart 6y ago
    Well, it's not an easy suggestion, but certainly an interesting one. I'm going to think more about this one. Thanks for providing a new thought on a hard problem!
    • elliekelly 6y ago
      The problem reminds me of that old story about the group of kids who missed their final exam with a flat tire. If you can figure out the modern CS equivalent(s) of asking your students “which tire” about their project there’s probably an easier implementation. Good luck!
• letitbeirie 6y ago
  When your students submit the project, have the landing page give them 10-15 minutes to make one tiny change and resubmit. Pick something that would be trivial to do in seconds for a student who did his own work, but would cause a lot of stress for someone who outsourced. Save the 1:1 interviews for students whose original program was great but whose modification went down in flames and you'll likely be talking to your outsourcers, plus identifying students who maybe could use some extra help even if they did everything themselves.
  • true_religion 6y ago
    This will work for one semester. The next semester everyone will know about it, and keep their outsources on staff till after grades are given.
  • wdevanny 6y ago
    Are you prepared on the basis of a student not successfully making the change to fail that student and/or expel them from their university? Sure you can invite them in for an interview, but what happens if the student just denies any accusations. I am also skeptical you could find appropriate trivial changes to request. Students, and particularly beginner students, may write very unconventional code that may not be as easily modified as a well-thought-out solution.
    • saagarjha 6y ago
      I guess you could take those students and review their cases personally by taking to them.
    • jdsully 6y ago
      If its something like modifying a string literal then I can’t see the problem. If the program is too obtuse to permit that then its a major concern on its own.
      • jgwil2 6y ago
        If it's just modifying a string literal I suspect most would just search for that string and have little trouble even if the work is outsourced. Perhaps within a certain set of requirements there are modifications you could ask for that are not trivial and could be done in fifteen minutes but I think this is pretty hard in the general case and depends a lot on code quality.
• dntbnmpls 6y ago
  The only way is in-person student code review or in-class projects. You could go over their code with each student and ask poignant questions that can only be answered if the student wrote the code themselves. But I'm not sure how feasible that or the in-class project is for most teachers. Also, anyone remember the story of the programmer who subcontracted his work to some offshore freelancer for a fraction of his salary? He just sat at his cubicle goofing off all day while the freelancer did all the work.