5 ms·

Design better tests. If you can "cheat" on a test by just googling things/looking at notes then the test isn't testing anything but rote memorization anyway.

KingOfCoders 4y ago
A math professor (granted easier with math) at university always said, you can bring anything you want to the exam, if you can answer it with help he made a mistake designing the test. I see the same when I interview people online and I hear them frantically click their keyboard - but no help, I know before what Google returns.
- PeterStuer 4y ago
  Did any student show up with a 5G tablet and an earpiece?
  - KingOfCoders 4y ago
    No but there were some were I suspected someone was listening in and answering questions.
- p1necone 4y ago
  Trying to evaluate a software engineer without letting them use google strikes me as being similar to trying to evaluate a construction worker by making them dig holes with their bare hands.
  - KingOfCoders 4y ago
    I'm not asking things that can be solved by Google [1], because they can be solved by Google, and I only gain the knowledge "Candidate can Google", I see this as given. It doesn't mean that not a lot of work at the job involves googling Stackoverflow - I do it myself all the time. [1] I don't ask API knowledge, if someone doesn't know some API I tell them just to invent the call they need.
    - tpetry 4y ago
      Add 1-2 questions which can be googled. I have seen enough engineers not earning the „can efficiently google“ badge.
      - saalweachter 4y ago
        I got dinged on that test once. "Why didn't you just Google this question?" they asked, "Because I was taking a test," I said.
  - zimpenfish 4y ago
    Best technical interview I ever had, they sat me down in front of an internet-connected laptop, gave me some problems on a piece of paper, and said "we'll be back in an hour, do your best". Just left me alone with Google et al. (Then I spent an hour-ish talking to swardley about 3D printing and its future impact on IP law, etc. -- this was back in early 2007, before it was generally available. That was the weirdest non-technical interview I've had.)
IshKebab 4y ago
Ok how do you design a test that's so good I can't just pay someone else to take it for me?
- deleted 4y ago
  [deleted]
rtev 4y ago
While preparing for a certification, I had someone DM me on the prep forums and offer to ship me an in-ear monitor and physical HDMI mole to coach me in real-time. Cheaters are nuts, there are few test formats that would survive an attack like that.
gommm 4y ago
Exactly, I strongly believe that any test at a secondary education level should be open book. We should be testing for reasoning and capacity to apply knowledge not rote-learning. It's also why I believe that in most cases multiple choice questions have no place in any well designed tests because it's extremely hard to create multiple choice questions that actually test the application of knowledge versus the regurgitation of rote learning. When I was in university (grande école) in France, luckily all tests followed that format, they were open book, very few questions that needed deep detailed answers and were about interesting problems. Even finding tests from previous years and doing them was actually fun (tests were never reused). I did an exchange in the US at RIT and most tests had very boring rote learning questions. For example in a computer graphics class: Which company were voting members of the OpenGL Architecture Review Board? Which year was the first OpenGl specification released?. I still remember those questions because I was shocked by how utterly stupid it was to have those on a test but, in general, 25 to 50% of the points of any tests was on those completely irrelevant, useless questions that anyone can google for. Cheating is a problem for universities that never had proper testing standards. But that's a symptom of incompetence on the part of professors and of universities not making the choice to ensure that tests have to be well designed.
- p1necone 4y ago
  I've seen some very hairy multi choice questions in Math/Computer science (think testing order of operations, which function gets called due to complicated polymorphism etc). I think it depends on the subject. Although I also encountered a few where multiple answers were technically correct. Then you have to guess at which answer the professor thinks is "more correct" which kinda sucks.
  - gommm 4y ago
    Yeah, another decent alternative I've seen was a test with a few multiple choice questions where each question needed quite a bit of reasoning and where you had to give the reasoning to get the points. The multiple choice question was a way for the teacher to quickly know if you had the correct answer and for the student to verify their answer against possible answers and see if they might have made a mistake but the teacher would still need to review the half a page of reasoning for each question. It can be done decently but it's much easier to write and much more common to see bad multiple choice questions.
PeterStuer 4y ago
It's not just googling . It's communicating with others. In a no checks environment you can have an a la carte team of paid consultants and helpfull friends solving the test with you.
- janekm 4y ago
  and... where's the problem? That's exactly the kind of group problem-solving that is actually required in the real world.
  - blitzar 4y ago
    apparently 'im really good at google search' wasnt the answer someone was looking for in an interview
  - detaro 4y ago
    that if I hire someone for skills with X, I dont want their skill to be "I can ask people with skills with X what to write".
- klabb3 4y ago
  Bingo! [Surprised I had to scroll so far to find this.] You can discuss the merits of rote learning and test design all day, but cheating exists within any pedagogic paradigm. Back in my day, most cheating was between students doing each other favors or taking a small commission. The way some teachers solved this was a 1:1 oral complementary exam, where students were asked to explain a random set of their solutions. This is drastically more difficult to cheat in real time, and you will almost certainly look like an idiot if you come unprepared. It's a stupidly simple solution, disincentivizing cheating through shame. Doesn't need any proctoring rootkits and rectal cameras, a simple identity check is enough.
tpetry 4y ago
One of the bests I had written in university allowed you to bring every non-digital material you want to have. All slides? The books? The former exercises? No problem! The test was designed to be built on 2 primitives: 1. If you need to lookup stuff in your waste of material you won‘t be able to finish everything. The time constraint was very hard. 2. It was not just repeating former things, you had to have understood them. Everything was a transfer knowledge exercise.
- NotOscarWilde 4y ago
  > If you need to lookup stuff in your waste of material you won‘t be able to finish everything. The time constraint was very hard. Did you enjoy this exam? I am asking as I had a very similar experience in my undergrad: Our university allowed some basic courses to be studied at different departments, if they were equivalent. This means that there was a CS Major Algebra 1, but also Math Major Algebra 1, and you can take either. I took the Math Major Algebra 1, being a CS student, and I have passed everything up to the final exam, where I learned that the exam is literally what you describe. All the proofs and arguments needed follow from basic principles (as things in Algebra 1 tend to do), but there is a lot of questions and the time limit is strict. Thus, you have no time to actually do any deliberate thinking, you have to memorize and understand the basics so well that you can develop the arguments essentially in real time. (And it was not an open book exam, even, but as you describe, it would not have helped you too much.) I hated that exam. I never took it out of principle, instead just quitting the course and doing it with a great grade a year later, in CS. I still dislike the idea of forcing knowledge acquisition through strict time limits. What if somebody is smart but has a slow start due to stress? Ugh.
- matheusmoreira 4y ago
  > If you need to lookup stuff in your waste of material you won‘t be able to finish everything. The time constraint was very hard. Reminds me of my school days. Tests with hundreds of questions and less than a minute to solve each question. They were essentially testing the student's time management skills as well as their physical and mental fortitude in addition to their knowledge. Extremely exhausting and anxiety inducing.
- 4y ago
jonfw 4y ago
“Design better tests” is not useful feedback. It’s just kicking the can down the road and blaming professors
Alex7845 4y ago
Students typically don't cheat in online tests by looking at their notes. What they do is ask other students or have someone else do the exam for them.

KingOfCoders 4y ago

A math professor (granted easier with math) at university always said, you can bring anything you want to the exam, if you can answer it with help he made a mistake designing the test. I see the same when I interview people online and I hear them frantically click their keyboard - but no help, I know before what Google returns.

PeterStuer 4y ago
Did any student show up with a 5G tablet and an earpiece?
- KingOfCoders 4y ago
  No but there were some were I suspected someone was listening in and answering questions.
p1necone 4y ago
Trying to evaluate a software engineer without letting them use google strikes me as being similar to trying to evaluate a construction worker by making them dig holes with their bare hands.
- KingOfCoders 4y ago
  I'm not asking things that can be solved by Google [1], because they can be solved by Google, and I only gain the knowledge "Candidate can Google", I see this as given. It doesn't mean that not a lot of work at the job involves googling Stackoverflow - I do it myself all the time. [1] I don't ask API knowledge, if someone doesn't know some API I tell them just to invent the call they need.
  - tpetry 4y ago
    Add 1-2 questions which can be googled. I have seen enough engineers not earning the „can efficiently google“ badge.
    - saalweachter 4y ago
      I got dinged on that test once. "Why didn't you just Google this question?" they asked, "Because I was taking a test," I said.
- zimpenfish 4y ago
  Best technical interview I ever had, they sat me down in front of an internet-connected laptop, gave me some problems on a piece of paper, and said "we'll be back in an hour, do your best". Just left me alone with Google et al. (Then I spent an hour-ish talking to swardley about 3D printing and its future impact on IP law, etc. -- this was back in early 2007, before it was generally available. That was the weirdest non-technical interview I've had.)

IshKebab 4y ago

Ok how do you design a test that's so good I can't just pay someone else to take it for me?

deleted 4y ago
[deleted]

rtev 4y ago

While preparing for a certification, I had someone DM me on the prep forums and offer to ship me an in-ear monitor and physical HDMI mole to coach me in real-time. Cheaters are nuts, there are few test formats that would survive an attack like that.

gommm 4y ago

Exactly, I strongly believe that any test at a secondary education level should be open book. We should be testing for reasoning and capacity to apply knowledge not rote-learning. It's also why I believe that in most cases multiple choice questions have no place in any well designed tests because it's extremely hard to create multiple choice questions that actually test the application of knowledge versus the regurgitation of rote learning. When I was in university (grande école) in France, luckily all tests followed that format, they were open book, very few questions that needed deep detailed answers and were about interesting problems. Even finding tests from previous years and doing them was actually fun (tests were never reused). I did an exchange in the US at RIT and most tests had very boring rote learning questions. For example in a computer graphics class: Which company were voting members of the OpenGL Architecture Review Board? Which year was the first OpenGl specification released?. I still remember those questions because I was shocked by how utterly stupid it was to have those on a test but, in general, 25 to 50% of the points of any tests was on those completely irrelevant, useless questions that anyone can google for. Cheating is a problem for universities that never had proper testing standards. But that's a symptom of incompetence on the part of professors and of universities not making the choice to ensure that tests have to be well designed.

p1necone 4y ago
I've seen some very hairy multi choice questions in Math/Computer science (think testing order of operations, which function gets called due to complicated polymorphism etc). I think it depends on the subject. Although I also encountered a few where multiple answers were technically correct. Then you have to guess at which answer the professor thinks is "more correct" which kinda sucks.
- gommm 4y ago
  Yeah, another decent alternative I've seen was a test with a few multiple choice questions where each question needed quite a bit of reasoning and where you had to give the reasoning to get the points. The multiple choice question was a way for the teacher to quickly know if you had the correct answer and for the student to verify their answer against possible answers and see if they might have made a mistake but the teacher would still need to review the half a page of reasoning for each question. It can be done decently but it's much easier to write and much more common to see bad multiple choice questions.

PeterStuer 4y ago

It's not just googling . It's communicating with others. In a no checks environment you can have an a la carte team of paid consultants and helpfull friends solving the test with you.

janekm 4y ago
and... where's the problem? That's exactly the kind of group problem-solving that is actually required in the real world.
- blitzar 4y ago
  apparently 'im really good at google search' wasnt the answer someone was looking for in an interview
- detaro 4y ago
  that if I hire someone for skills with X, I dont want their skill to be "I can ask people with skills with X what to write".
klabb3 4y ago
Bingo! [Surprised I had to scroll so far to find this.] You can discuss the merits of rote learning and test design all day, but cheating exists within any pedagogic paradigm. Back in my day, most cheating was between students doing each other favors or taking a small commission. The way some teachers solved this was a 1:1 oral complementary exam, where students were asked to explain a random set of their solutions. This is drastically more difficult to cheat in real time, and you will almost certainly look like an idiot if you come unprepared. It's a stupidly simple solution, disincentivizing cheating through shame. Doesn't need any proctoring rootkits and rectal cameras, a simple identity check is enough.

tpetry 4y ago

One of the bests I had written in university allowed you to bring every non-digital material you want to have. All slides? The books? The former exercises? No problem! The test was designed to be built on 2 primitives: 1. If you need to lookup stuff in your waste of material you won‘t be able to finish everything. The time constraint was very hard. 2. It was not just repeating former things, you had to have understood them. Everything was a transfer knowledge exercise.

NotOscarWilde 4y ago
> If you need to lookup stuff in your waste of material you won‘t be able to finish everything. The time constraint was very hard. Did you enjoy this exam? I am asking as I had a very similar experience in my undergrad: Our university allowed some basic courses to be studied at different departments, if they were equivalent. This means that there was a CS Major Algebra 1, but also Math Major Algebra 1, and you can take either. I took the Math Major Algebra 1, being a CS student, and I have passed everything up to the final exam, where I learned that the exam is literally what you describe. All the proofs and arguments needed follow from basic principles (as things in Algebra 1 tend to do), but there is a lot of questions and the time limit is strict. Thus, you have no time to actually do any deliberate thinking, you have to memorize and understand the basics so well that you can develop the arguments essentially in real time. (And it was not an open book exam, even, but as you describe, it would not have helped you too much.) I hated that exam. I never took it out of principle, instead just quitting the course and doing it with a great grade a year later, in CS. I still dislike the idea of forcing knowledge acquisition through strict time limits. What if somebody is smart but has a slow start due to stress? Ugh.
matheusmoreira 4y ago
> If you need to lookup stuff in your waste of material you won‘t be able to finish everything. The time constraint was very hard. Reminds me of my school days. Tests with hundreds of questions and less than a minute to solve each question. They were essentially testing the student's time management skills as well as their physical and mental fortitude in addition to their knowledge. Extremely exhausting and anxiety inducing.
4y ago

jonfw 4y ago

“Design better tests” is not useful feedback. It’s just kicking the can down the road and blaming professors

Alex7845 4y ago

Students typically don't cheat in online tests by looking at their notes. What they do is ask other students or have someone else do the exam for them.