6 ms·

Folks are doing take-home exam wrong. Caltech had tremendous success with a culture of honesty and take-home exams.

collegeburner 6y ago
As a current student, I can maybe offer a bit of perspective on this. There are a few different options I've seen tried: 1. Project-based assignments. These work well. In certain cases a tool to check for plagiarism may be helpful, provided that a human review is used to double-check flagged assignments. 2. Essays. These can be given as take-home assignments, in which case they are similar to 1. They can also be given in an exam format, which is generally difficult to cheat and does a good job of testing understanding. 3. Traditional exams. These work poorly in a take-home format. They typically require invasive solutions. The least-bad that I've seen are those in which teachers require students to get on zoom with a camera pointing at the student's face, workspace, and screen. The other big downside is that they generally make students more nervous about the test. To any profs that read this: please use options one or two if possible. Your students will thank you and may even do better for it.
- 908B64B197 6y ago
  3. favors rote. But is cheaper and easier to grade.
- jholman 6y ago
  I think you're missing the point completely. Start by supposing, for the sake of argument, that you think that grades matter. Like for example for determining access to scholarships or graduate programs. Also suppose, for the sake of argument, that some percentage of the students are going to cheat, and suppose that it matters to the aforementioned grades. Also I'm assuming a distance-learning option, perhaps due to quarantine, but the question generalizes to other reasons. How do options 1 and 2 do anything at all to fight the alleged cheating problem? Remote essays are very easy to cheat, just as easy as any other kind of remote exam. Anything take-home is even easier to cheat. The parent commenter is simply claiming that cheating just doesn't happen enough to matter (at least not at Caltech, they claim). That's possible, I have no idea. But it's quite different from saying "remote exams are bad". --- As for why professors administer exams, btw... a professor is going to spend such-and-such an amount of time on you and your class. Maybe it's 5 hours per week, maybe it's 20. Would you rather they invest that time into teaching, or marking? Speaking as a former student and a current teacher, I tend to prefer it be spent on teaching. If you agree that the teaching is more valuable then the marking, teachers should choose the least labour-demanding form of assessment, so they can get back to investing energy in teaching (and/or get back to other responsibilities). Exams are easier to grade than projects and essays. A professor who chooses exams is usually saying "marking is not a good use of my time". (A professor who chooses essays or projects is usually saying "marking is a teaching opportunity so I hate this but oh well").
  - 908B64B197 6y ago
    > Remote essays are very easy to cheat Automated plagiarism detection tools should be good enough these days. Of course, there's always the possibility one could buy an essay but then there's always the risk the seller sold the same essay to several students!
    - ev1 6y ago
      I used to TA and have seen several of these essay farms in the most amusing ways. - Embedded metadata in the PDF of a long, foreign name that was not the student's (as Author:) - An email to several TAs + prof for the course with the student's personal information sent by the essay farm because the student was harassing the essay farm over payment/cost/speed, including live chat logs of the student calling them "pakis" (the essay farm was based in the Philippines) - Two students purchased and turned in an identical essay except the words were re-spun (bullshit synonym replacement, automated) to the point that both were unreadable to a native English speaker. It was clear neither of them had even read through it before submitting it at 23:59 A fair chunk of bought essays (under $100-150, probably, not the truly bespoke "pay2win" type) are nearly just the Wikipedia article regurgitated with irrelevant citations or sources that just vaguely match the keywords.
      - 908B64B197 6y ago
        > An email to several TAs + prof for the course with the student's personal information sent by the essay farm because the student was harassing the essay farm over payment/cost/speed, including live chat logs of the student calling them "pakis" (the essay farm was based in the Philippines) Wow. What ended up happening? The truly bespoke "pay2win" (love the name!) will always work sadly. But the price point means you don't have to be worried about 99% of the student population. It's an acceptable risk I suppose.
        ev1 6y ago
        Nothing major unfortunately, on paper that should have been expulsion/suspension but in reality he got a zero and had to retake, and since it was a prerequisite for other courses it pretty much had the side effect of adding an extra year of tuition.
- jojobas 6y ago
  Options 1 and 2 pretend there's no "Assignment as a service" market. In reality there's not just a market, there are Ubers on the market, making it easy to match cheaty students with industrious graduates.
thekyle 6y ago
> Caltech had tremendous success with a culture of honesty and take-home exams. Could you be a little more specific?
- tzs 6y ago
  Most exams at Caltech are take home. Typically the exam is made available sometime near the end of the term, with instructions that tell you when it is due, how long you are allowed to work on it, and what you can use during it (ranging from nothing to your own notes to full open book). Due dates are typically a week or two from when it becomes available. There is no one watching, in person or remotely, when you take an exam unless you happen to take it somewhere there are other people.
7thaccount 6y ago
I had engineering tests as take-home before. It wasn't super common, but some professors did do that for seniors on occasion for small classes where cheating wouldn't be a problem. They were so stupidly hard that even with multiple text books and the internet, the best I could often do was a "B". There's simply no way to google an answer for something that requires multiple pages of analysis and so forth per problem. I think this was a fairer representation of the real world too. I think a take home test would be harder for those subjects that are more about rote memorization.
- LASR 6y ago
  > something that requires multiple pages of analysis and so forth per problem. I think this was a fairer representation of the real world too. Exactly. This is exactly what engineering is about, isn't it? If any one thinks they're being smart/sneaky by googling to "cheat" in their take-home exams, I don't think they understand what exactly they're being tested on. Said another way, I wouldn't drive on a bridge designed by a civil engineer who didn't "cheat" by googling everything everwhere.
- stordoff 6y ago
  My university had an interesting solution for a couple of modules for computer science - there were six programming assignments spread throughout the term, and you do them at home and submitted online. It'd be (obviously) very easy to cheat, and being programming a certain degree of similarity would be expected. However, for the final assignment, you had to go for a viva. You had to explain how it worked and why you'd chosen to do things in a certain way, and that formed the bulk of your assessment.
- raxxorrax 6y ago
  For some math problems that works, but the set of possible questions is shrinking every time such questions are posed. But for programming? I think having to explain the solution is the best was to determine possible cheating.
meekrohprocess 6y ago
Most tests that I took in college were essentially open-book. Not all of them, but definitely more than half. They could have been take-home if(f) the professors trusted students not to use the internet during an exam. In the math and computing classes, professors structured the questions in layers that tested different concepts. You could always have a double-sided "cheat sheet", and some professors let students bring in the whole textbook for time-constrained exams. It took far too long to "brute-force" more than one or two questions. In the liberal arts classes, you were tested more on critical thinking and contextual questions. You needed to understand concepts at more than a surface level to make compelling arguments about how they related to each other, and you wouldn't be able to develop that understanding for many topics over the course of an hour, even if you have the primary material in front of you. We weren't allowed to bring anything with a radio, but forcing students to take exams in complete isolation always felt odd to me. It's pretty rare to need to do actual work in a "clean-room" environment.
vondur 6y ago
My P Chem professor let us use whatever notes/books on our exams. Of course the exam was very long and hard, so you really didn’t have enough time to go through your notes quickly enough to help.

As a current student, I can maybe offer a bit of perspective on this. There are a few different options I've seen tried: 1. Project-based assignments. These work well. In certain cases a tool to check for plagiarism may be helpful, provided that a human review is used to double-check flagged assignments. 2. Essays. These can be given as take-home assignments, in which case they are similar to 1. They can also be given in an exam format, which is generally difficult to cheat and does a good job of testing understanding. 3. Traditional exams. These work poorly in a take-home format. They typically require invasive solutions. The least-bad that I've seen are those in which teachers require students to get on zoom with a camera pointing at the student's face, workspace, and screen. The other big downside is that they generally make students more nervous about the test. To any profs that read this: please use options one or two if possible. Your students will thank you and may even do better for it.

908B64B197 6y ago
3. favors rote. But is cheaper and easier to grade.
jholman 6y ago
I think you're missing the point completely. Start by supposing, for the sake of argument, that you think that grades matter. Like for example for determining access to scholarships or graduate programs. Also suppose, for the sake of argument, that some percentage of the students are going to cheat, and suppose that it matters to the aforementioned grades. Also I'm assuming a distance-learning option, perhaps due to quarantine, but the question generalizes to other reasons. How do options 1 and 2 do anything at all to fight the alleged cheating problem? Remote essays are very easy to cheat, just as easy as any other kind of remote exam. Anything take-home is even easier to cheat. The parent commenter is simply claiming that cheating just doesn't happen enough to matter (at least not at Caltech, they claim). That's possible, I have no idea. But it's quite different from saying "remote exams are bad". --- As for why professors administer exams, btw... a professor is going to spend such-and-such an amount of time on you and your class. Maybe it's 5 hours per week, maybe it's 20. Would you rather they invest that time into teaching, or marking? Speaking as a former student and a current teacher, I tend to prefer it be spent on teaching. If you agree that the teaching is more valuable then the marking, teachers should choose the least labour-demanding form of assessment, so they can get back to investing energy in teaching (and/or get back to other responsibilities). Exams are easier to grade than projects and essays. A professor who chooses exams is usually saying "marking is not a good use of my time". (A professor who chooses essays or projects is usually saying "marking is a teaching opportunity so I hate this but oh well").
- 908B64B197 6y ago
  > Remote essays are very easy to cheat Automated plagiarism detection tools should be good enough these days. Of course, there's always the possibility one could buy an essay but then there's always the risk the seller sold the same essay to several students!
  - ev1 6y ago
    I used to TA and have seen several of these essay farms in the most amusing ways. - Embedded metadata in the PDF of a long, foreign name that was not the student's (as Author:) - An email to several TAs + prof for the course with the student's personal information sent by the essay farm because the student was harassing the essay farm over payment/cost/speed, including live chat logs of the student calling them "pakis" (the essay farm was based in the Philippines) - Two students purchased and turned in an identical essay except the words were re-spun (bullshit synonym replacement, automated) to the point that both were unreadable to a native English speaker. It was clear neither of them had even read through it before submitting it at 23:59 A fair chunk of bought essays (under $100-150, probably, not the truly bespoke "pay2win" type) are nearly just the Wikipedia article regurgitated with irrelevant citations or sources that just vaguely match the keywords.
    - 908B64B197 6y ago
      > An email to several TAs + prof for the course with the student's personal information sent by the essay farm because the student was harassing the essay farm over payment/cost/speed, including live chat logs of the student calling them "pakis" (the essay farm was based in the Philippines) Wow. What ended up happening? The truly bespoke "pay2win" (love the name!) will always work sadly. But the price point means you don't have to be worried about 99% of the student population. It's an acceptable risk I suppose.
      - ev1 6y ago
        Nothing major unfortunately, on paper that should have been expulsion/suspension but in reality he got a zero and had to retake, and since it was a prerequisite for other courses it pretty much had the side effect of adding an extra year of tuition.
jojobas 6y ago
Options 1 and 2 pretend there's no "Assignment as a service" market. In reality there's not just a market, there are Ubers on the market, making it easy to match cheaty students with industrious graduates.

thekyle 6y ago

> Caltech had tremendous success with a culture of honesty and take-home exams. Could you be a little more specific?

tzs 6y ago
Most exams at Caltech are take home. Typically the exam is made available sometime near the end of the term, with instructions that tell you when it is due, how long you are allowed to work on it, and what you can use during it (ranging from nothing to your own notes to full open book). Due dates are typically a week or two from when it becomes available. There is no one watching, in person or remotely, when you take an exam unless you happen to take it somewhere there are other people.

7thaccount 6y ago

I had engineering tests as take-home before. It wasn't super common, but some professors did do that for seniors on occasion for small classes where cheating wouldn't be a problem. They were so stupidly hard that even with multiple text books and the internet, the best I could often do was a "B". There's simply no way to google an answer for something that requires multiple pages of analysis and so forth per problem. I think this was a fairer representation of the real world too. I think a take home test would be harder for those subjects that are more about rote memorization.

LASR 6y ago
> something that requires multiple pages of analysis and so forth per problem. I think this was a fairer representation of the real world too. Exactly. This is exactly what engineering is about, isn't it? If any one thinks they're being smart/sneaky by googling to "cheat" in their take-home exams, I don't think they understand what exactly they're being tested on. Said another way, I wouldn't drive on a bridge designed by a civil engineer who didn't "cheat" by googling everything everwhere.
stordoff 6y ago
My university had an interesting solution for a couple of modules for computer science - there were six programming assignments spread throughout the term, and you do them at home and submitted online. It'd be (obviously) very easy to cheat, and being programming a certain degree of similarity would be expected. However, for the final assignment, you had to go for a viva. You had to explain how it worked and why you'd chosen to do things in a certain way, and that formed the bulk of your assessment.
raxxorrax 6y ago
For some math problems that works, but the set of possible questions is shrinking every time such questions are posed. But for programming? I think having to explain the solution is the best was to determine possible cheating.

meekrohprocess 6y ago

Most tests that I took in college were essentially open-book. Not all of them, but definitely more than half. They could have been take-home if(f) the professors trusted students not to use the internet during an exam. In the math and computing classes, professors structured the questions in layers that tested different concepts. You could always have a double-sided "cheat sheet", and some professors let students bring in the whole textbook for time-constrained exams. It took far too long to "brute-force" more than one or two questions. In the liberal arts classes, you were tested more on critical thinking and contextual questions. You needed to understand concepts at more than a surface level to make compelling arguments about how they related to each other, and you wouldn't be able to develop that understanding for many topics over the course of an hour, even if you have the primary material in front of you. We weren't allowed to bring anything with a radio, but forcing students to take exams in complete isolation always felt odd to me. It's pretty rare to need to do actual work in a "clean-room" environment.

vondur 6y ago

My P Chem professor let us use whatever notes/books on our exams. Of course the exam was very long and hard, so you really didn’t have enough time to go through your notes quickly enough to help.