6 ms·

This has been eluded to by a few other comments already, but I think it's worth stressing: Maybe the problem is that our definition of "cheating" doesn't make s

This has been eluded to by a few other comments already, but I think it's worth stressing: Maybe the problem is that our definition of "cheating" doesn't make sense anymore. Our methods of teaching come from a worldview in which the information stored in your head is your only resource. That wasn't true in the past where you could consult colleagues, look things up in books or call experts - and it's certainly not true now that everyone carries the entire knowledge of humankind around in their pocket. So what is the point in artificially creating a test situation where the student is isolated from resources she'll have at her disposal in real life? How about instead, we create (whatever the plural of curriculum is) that teach you how to use these resources efficiently and test you on how well you perform using all the tools that will be at your disposal in your actual job as well?

lotsofpulp 4y ago
> So what is the point in artificially creating a test situation where the student is isolated from resources she'll have at her disposal in real life? I can envision many scenarios where a person needs to be quick on their feet and does not have time to look things up. Drivers, pilots, doctors, etc. Certification provides a lot of value to society. The other point of testing is to separate wheat from chaff. To see if someone is disciplined enough to learn the material or see a task through. There are issues of going beyond the ideal amount of testing here, but not totally useless in my opinion.
- wolframhempel 4y ago
  True - and there's certainly a line to be drawn between core knowledge and wider knowledge. A programmer, looking up what an if-statement is each time won't do well, regardless of the resources at their disposal. But I'd argue that realistic test conditions correct for this automatically. If your test is "write a program that does X within two hours" the programmer lacking core knowledge will fail. If your test is "recite 20 programming paradigms from memory" you will have to check for "cheating"
- antipurist 4y ago
  Those scenarios can be better tested through simulations. E.g. to pass a driving exam you are presented with a road situation, and given only a few seconds to respond — this completely eliminates the possibility of looking things up on the internet. A similar approach can be used for other disciplines. > separate wheat from chaff Wheat can demonstrate/use the knowledge, chaff can't. Why does it matter whether the knowledge is internal or external, assuming results are the same?
  - lotsofpulp 4y ago
    Because you do not know if the results are the same before making the decision to go with person A or person B. There are time costs in figuring out if someone can do something or is capable of learning how to do something, so test(s) of various kinds are used as a proxy.
tremon 4y ago
in the past where you could consult colleagues Perhaps tests are to make sure that you are a going to be colleague worth consulting then? Yes, pure memorization and recall of facts is less valuable now than before, but knowing how to apply the facts available to you is still a skill worth having, and similarly worth testing. Those skills can also be assessed using written tests, and the potential for cheating is just as on memory tests. what is the point in artificially creating a test situation where the student is isolated from resources she'll have at her disposal in real life Let's take language learning as an example: do you consider a foreigner with a dictionary to be just as expressive as a native speaker? Perhaps there is value in testing the immediate knowledge of the student rather than their lookup skills?
- wolframhempel 4y ago
  I want to test their ability to perform a task - rather than how well they memorized things. Which is not to say that memorization is not important, but it's just one piece of the Puzzle. So when it comes to language tests, rather than having the candidate fill in prepositions into blanks, let's have an actual conversation with them in which they apply their combined skills.
  - wolframhempel 4y ago
    I actually adhere to this philosophy when hiring candidates as well. Rather than having them explain how to reverse binary trees on a whiteboard, I have them sit down and do a mini version of the actual task they will have to do in their job later. And, unsurprisingly, knowing how well they do the actual task is a much better indicator of how well they perform at doing the actual task later.
    - nano9 4y ago
      I understand what the halting problem implies, what complexity is, what Turing machine means and why they are an abstraction for algorithms, and the basic common memes of programming such as using a stack and a heap, IEEE 754 floating point numbers, 2s complement binary numbering, and so on and so forth. Yet I don't know a lick of Go. If I were to open up a Hello World tutorial for Go during the interview, would I honestly still have a shot?
kumarvvr 4y ago
This could be done with tests, by having open book tests. The point of a test is to check if you are capable on your own, and not piggybacking on someone elses intellect or hard work. It is true, that in the real world, the resources available to use are infinitely larger than a classroom or a teaching setting. But your idea springs from lack of understanding about the true purpose of education, i.e, to groom children to think. Alas, modern education has become a cesspool of rote learning, atleast in my country, India.
comfypotato 4y ago
I believe the issue has more to do with the accreditation of learning institutions. “Untimed power tests” are the gold standard for testing, and in the modern world this includes own internet access. Everyone acknowledges that these are better at assessing knowledge, critical thinking, etc.. The problem is that these tests are very difficult to make and logistical nightmares concerning cheating. A timed, closely proctored, closed book/internet exam is much easy to create and administer, and it halfway (better than nothing) assesses whether or not the student has learned the material.
ookdatnog 4y ago
Even if all tests would be open-book, there's still a need to make sure that the person who is being tested is actually the one taking the test.
kaba0 4y ago
Well, it depends. A big part of the valuable experience comes from having memories of similar situations, and thus a shortcut to the solution. In IT it can be similar to “indexing”, like I may remember reading a relevant stackoverflow question and I know how to find it again, but I don’t remember the exact solution. But the catch is, to index something you first have to crawl through it. In medicine for example, there may not be much point in holding every single extreme-rare disease in memory, but having memorized them once and mostly forgotten them will lead a good doctor to know about it enough so that he/she can look it up on the odd chance it gets relevant. It will be only possible when he/she memorized it once back then. My point is, there is a point in memorization (especially that it improves memory all-around - one will remember facts much easily after extensive learning), what should be better distinguished is facts that are to be learnt for life vs those that are only needed for a given test. It’s like unknowns vs unknown unknowns - the latter can be much worse and you don’t want someone like that as a colleague.
- fn-mote 4y ago
  > [...] memories of similar situations [...] I'm going to claim that memory is only one part, maybe 1/3, of what I would like to examine for. Certainly in a job interview, but I also like university exams that include the parts below. There's another huge part, *problem solving skills*. Working long enough to figure out which pieces of knowledge that you have apply to the situation. There's another part, *making it work*. Software engineers deal with failures in practice (debugging) as well as theory (don't know how to do it). Sadly but interestingly, sometimes this is the point of failure for a student/examinee when they possess plenty of knowledge but don't know how to use it.
wil421 4y ago
Most if not all of my upper classes in school were open book. Even my beginning CS classes for Java was open book. There was no way you had enough time to finish the tests if you look up every answer. You would have to learn topics on the spot if you didn’t study, it was near impossible to do with the time constraints. Most of my upper classes were heavily weighted towards group projects anyway.
_steady 4y ago
Curricula
tarakat 4y ago
Being able to look things up, even in seconds, is not equivalent to knowing them. When someone suggests "let's do X", you cannot reply with "but that would cause Y", if Y is buried in some pdf file on your computer, or one Bing search away - how do you know you should search for Y, when there are infinite other things you could search for.
- 2OEH8eoCRo0 4y ago
  That's one of the main points of The Shallows by Nicholas Carr and I think that I agree. Long term subconscious memory is where a lot of the connections happen. If your long term memory only knows where to find things without knowing them then that ability is stunted.
- rntksi 4y ago
  The way I used to write/set-up tests when I was teaching is that you could use a limited set of online resources to answer, but if you have to search every question you won't have enough time to do it. So it also tested your ability to quickly locate the information you need should you not know it right away.
- forgotusername6 4y ago
  Back at university one of the first things we were told about being an a good engineer was: "You don't need to know everything, but you do need to know where to look it up". It's a statement that I agree with. There is just too much knowledge to know it all. These days there is a very real skill in knowing how to search i.e. a second order version of the above statement "you need to know where to find out how to look this up".
  - arcbyte 4y ago
    There's a HUGE difference between looking something up to reactivate sleepy memories and learning something new for the first time. There are some really technical things ive done in the pasy that I couldn't do without a reference right now, but once I've got the reference I don't need to be reminded about low level details or edge cases or gotchas - it's all still there in my head I just need to see the road map and it will all come back to me.
2OEH8eoCRo0 4y ago
Tests are supposed to test your knowledge of a concept. If you have to look it up then you don't know it.
flanderspost 4y ago
This works for concrete knowledge (declarative, procedural) where simply knowing a set of facts or a pre-existing process is sufficient to perform the job. But when you most into more abstract, creative work, this idea falls apart quickly. You can easily find an answer to "how do I enable file sharing on Windows." You cannot easily find an answer to "how do I design a website for an engineering firm which wants to increase it's inbound leads by 25%." The former is a simple search away. The latter requires the integration of numerous abstract concepts which the shallows of a search engine cannot return. Further, such an approach also limits our ability to create new knowledge.
Alex7845 4y ago
In online tests, students cheat by asking the answers to other people online (eg. chatrooms with other students in the same class) or by having someone else do the test. Teachers don't care about students looking into their books/pdf, what they want to prevent is them getting the answers from a snapchat group.
thayne 4y ago
As a physics student in college, almost all of my (physics) exams were open book to some degree. Some were even take home tests where I had a few days to complete it, and was allowed to use software like Mathematica and Matlab. I definitely think that is a better way to test.
jrpt 4y ago
That sounds nice in theory, but how would you write a test for, say, basic physics, when every type of question can be Googled? Even if you make brand new questions, it's still just generally available categories of problems, like inclined planes and stuff. It's even worse with algebra, because there's software that can both solve it for you and show the steps (which you can then copy onto a test). Unfortunately, in order to progress to more advanced topics, you need to understand the basics yourself. It's not sufficient to just be able to research it online. This is true in every field I can think of. That's why you can't just give people online access for every kind of test.
- Cerium 4y ago
  I had a basic physics teacher that allowed notes and gave us a chest sheet of his own which literally listed equations, described variables and explained the situation you might want the equation. People still failed. In a basic thermodynamics class the teacher said it was open everything but the person next to you: laptops, internet, the book etc. For fun I took the test without either book or laptop and did just fine. Others found it hard. In both cases the problems required thinking about the situation posed and which basic principles should be used to evaluate the problem. In my opinion, cheating from internet access or outside resources is not the problem, cheating by gogling exact problems is.
newbie2020 4y ago
It’s faster to look stuff up in the brain if it’s already there . Also having knowledge in the brain let’s it make connections you wouldn’t expect
denimnerd42 4y ago
I agree. We had open notes / open internet (no communication) exams at engineering school and they wouldn't help you pass the test if you didn't know the material. Maybe you could spend all exam period solving one of the questions if you weren't prepared but you'd be lucky to get a D.
As_You_Wish 4y ago
I agree. I dropped out of school in kindergarten. Why do I have to know my multiplication tables by heart? Why do I even have to know how to add two numbers together? I don't even know the alphabet and how to read and write - I'm just talking into a microphone and this is being written. I don't even know what country I live in or who the polical leader is, why should I? I can look it up online, I just don't have any desire to. As you can see, your argument is silly. It's important to know things, in your head. Of course I realize that you don't mean people shouldn't learn the alphabet or learn math, but where does one draw the line, where they don't have to learn stuff in their head? But there are always holes in what one is even able to look up. Maybe not for everyone, but for most. I just and a conversation the other day about this with someone and I was flabbergasted. The topic we discussed is that anyone can create a SQL database - anyone can. But do they normalize the database to 3rd or 4th normal form? He worked with a lot of corporations, and he said most companies don't have normalized databases. In university, that was a basic thing that they taught a semester course on that. An entire semester. Not to memorize anything, but just the general concepts. Someone can learn how to create an incorrectly structured database easily, but they don't even know about normalization, so how can you even look it up in the first place? And even if you do, somehow, learn what it is and look it up, I took the class for an entire semester. How is someone going to learn it in 15 minutes by looking it up? I looked up stuff in the book, too, and got info from the professor, and it took me an entire semester, and everyone else in the class, too. Unless you're calling us all dummies. There's all kinds of stuff like that. You don't know how to look up something if you don't even know it exists to look up, and it is not obvious.

lotsofpulp 4y ago

> So what is the point in artificially creating a test situation where the student is isolated from resources she'll have at her disposal in real life? I can envision many scenarios where a person needs to be quick on their feet and does not have time to look things up. Drivers, pilots, doctors, etc. Certification provides a lot of value to society. The other point of testing is to separate wheat from chaff. To see if someone is disciplined enough to learn the material or see a task through. There are issues of going beyond the ideal amount of testing here, but not totally useless in my opinion.

wolframhempel 4y ago
True - and there's certainly a line to be drawn between core knowledge and wider knowledge. A programmer, looking up what an if-statement is each time won't do well, regardless of the resources at their disposal. But I'd argue that realistic test conditions correct for this automatically. If your test is "write a program that does X within two hours" the programmer lacking core knowledge will fail. If your test is "recite 20 programming paradigms from memory" you will have to check for "cheating"
antipurist 4y ago
Those scenarios can be better tested through simulations. E.g. to pass a driving exam you are presented with a road situation, and given only a few seconds to respond — this completely eliminates the possibility of looking things up on the internet. A similar approach can be used for other disciplines. > separate wheat from chaff Wheat can demonstrate/use the knowledge, chaff can't. Why does it matter whether the knowledge is internal or external, assuming results are the same?
- lotsofpulp 4y ago
  Because you do not know if the results are the same before making the decision to go with person A or person B. There are time costs in figuring out if someone can do something or is capable of learning how to do something, so test(s) of various kinds are used as a proxy.

tremon 4y ago

in the past where you could consult colleagues Perhaps tests are to make sure that you are a going to be colleague worth consulting then? Yes, pure memorization and recall of facts is less valuable now than before, but knowing how to apply the facts available to you is still a skill worth having, and similarly worth testing. Those skills can also be assessed using written tests, and the potential for cheating is just as on memory tests. what is the point in artificially creating a test situation where the student is isolated from resources she'll have at her disposal in real life Let's take language learning as an example: do you consider a foreigner with a dictionary to be just as expressive as a native speaker? Perhaps there is value in testing the immediate knowledge of the student rather than their lookup skills?

wolframhempel 4y ago
I want to test their ability to perform a task - rather than how well they memorized things. Which is not to say that memorization is not important, but it's just one piece of the Puzzle. So when it comes to language tests, rather than having the candidate fill in prepositions into blanks, let's have an actual conversation with them in which they apply their combined skills.
- wolframhempel 4y ago
  I actually adhere to this philosophy when hiring candidates as well. Rather than having them explain how to reverse binary trees on a whiteboard, I have them sit down and do a mini version of the actual task they will have to do in their job later. And, unsurprisingly, knowing how well they do the actual task is a much better indicator of how well they perform at doing the actual task later.
  - nano9 4y ago
    I understand what the halting problem implies, what complexity is, what Turing machine means and why they are an abstraction for algorithms, and the basic common memes of programming such as using a stack and a heap, IEEE 754 floating point numbers, 2s complement binary numbering, and so on and so forth. Yet I don't know a lick of Go. If I were to open up a Hello World tutorial for Go during the interview, would I honestly still have a shot?

kumarvvr 4y ago

This could be done with tests, by having open book tests. The point of a test is to check if you are capable on your own, and not piggybacking on someone elses intellect or hard work. It is true, that in the real world, the resources available to use are infinitely larger than a classroom or a teaching setting. But your idea springs from lack of understanding about the true purpose of education, i.e, to groom children to think. Alas, modern education has become a cesspool of rote learning, atleast in my country, India.

comfypotato 4y ago

I believe the issue has more to do with the accreditation of learning institutions. “Untimed power tests” are the gold standard for testing, and in the modern world this includes own internet access. Everyone acknowledges that these are better at assessing knowledge, critical thinking, etc.. The problem is that these tests are very difficult to make and logistical nightmares concerning cheating. A timed, closely proctored, closed book/internet exam is much easy to create and administer, and it halfway (better than nothing) assesses whether or not the student has learned the material.

ookdatnog 4y ago

Even if all tests would be open-book, there's still a need to make sure that the person who is being tested is actually the one taking the test.

kaba0 4y ago

Well, it depends. A big part of the valuable experience comes from having memories of similar situations, and thus a shortcut to the solution. In IT it can be similar to “indexing”, like I may remember reading a relevant stackoverflow question and I know how to find it again, but I don’t remember the exact solution. But the catch is, to index something you first have to crawl through it. In medicine for example, there may not be much point in holding every single extreme-rare disease in memory, but having memorized them once and mostly forgotten them will lead a good doctor to know about it enough so that he/she can look it up on the odd chance it gets relevant. It will be only possible when he/she memorized it once back then. My point is, there is a point in memorization (especially that it improves memory all-around - one will remember facts much easily after extensive learning), what should be better distinguished is facts that are to be learnt for life vs those that are only needed for a given test. It’s like unknowns vs unknown unknowns - the latter can be much worse and you don’t want someone like that as a colleague.

fn-mote 4y ago
> [...] memories of similar situations [...] I'm going to claim that memory is only one part, maybe 1/3, of what I would like to examine for. Certainly in a job interview, but I also like university exams that include the parts below. There's another huge part, *problem solving skills*. Working long enough to figure out which pieces of knowledge that you have apply to the situation. There's another part, *making it work*. Software engineers deal with failures in practice (debugging) as well as theory (don't know how to do it). Sadly but interestingly, sometimes this is the point of failure for a student/examinee when they possess plenty of knowledge but don't know how to use it.

wil421 4y ago

Most if not all of my upper classes in school were open book. Even my beginning CS classes for Java was open book. There was no way you had enough time to finish the tests if you look up every answer. You would have to learn topics on the spot if you didn’t study, it was near impossible to do with the time constraints. Most of my upper classes were heavily weighted towards group projects anyway.

_steady 4y ago

Curricula

tarakat 4y ago

Being able to look things up, even in seconds, is not equivalent to knowing them. When someone suggests "let's do X", you cannot reply with "but that would cause Y", if Y is buried in some pdf file on your computer, or one Bing search away - how do you know you should search for Y, when there are infinite other things you could search for.

2OEH8eoCRo0 4y ago
That's one of the main points of The Shallows by Nicholas Carr and I think that I agree. Long term subconscious memory is where a lot of the connections happen. If your long term memory only knows where to find things without knowing them then that ability is stunted.
rntksi 4y ago
The way I used to write/set-up tests when I was teaching is that you could use a limited set of online resources to answer, but if you have to search every question you won't have enough time to do it. So it also tested your ability to quickly locate the information you need should you not know it right away.
forgotusername6 4y ago
Back at university one of the first things we were told about being an a good engineer was: "You don't need to know everything, but you do need to know where to look it up". It's a statement that I agree with. There is just too much knowledge to know it all. These days there is a very real skill in knowing how to search i.e. a second order version of the above statement "you need to know where to find out how to look this up".
- arcbyte 4y ago
  There's a HUGE difference between looking something up to reactivate sleepy memories and learning something new for the first time. There are some really technical things ive done in the pasy that I couldn't do without a reference right now, but once I've got the reference I don't need to be reminded about low level details or edge cases or gotchas - it's all still there in my head I just need to see the road map and it will all come back to me.

2OEH8eoCRo0 4y ago

Tests are supposed to test your knowledge of a concept. If you have to look it up then you don't know it.

flanderspost 4y ago

This works for concrete knowledge (declarative, procedural) where simply knowing a set of facts or a pre-existing process is sufficient to perform the job. But when you most into more abstract, creative work, this idea falls apart quickly. You can easily find an answer to "how do I enable file sharing on Windows." You cannot easily find an answer to "how do I design a website for an engineering firm which wants to increase it's inbound leads by 25%." The former is a simple search away. The latter requires the integration of numerous abstract concepts which the shallows of a search engine cannot return. Further, such an approach also limits our ability to create new knowledge.

Alex7845 4y ago

In online tests, students cheat by asking the answers to other people online (eg. chatrooms with other students in the same class) or by having someone else do the test. Teachers don't care about students looking into their books/pdf, what they want to prevent is them getting the answers from a snapchat group.

thayne 4y ago

As a physics student in college, almost all of my (physics) exams were open book to some degree. Some were even take home tests where I had a few days to complete it, and was allowed to use software like Mathematica and Matlab. I definitely think that is a better way to test.

jrpt 4y ago

That sounds nice in theory, but how would you write a test for, say, basic physics, when every type of question can be Googled? Even if you make brand new questions, it's still just generally available categories of problems, like inclined planes and stuff. It's even worse with algebra, because there's software that can both solve it for you and show the steps (which you can then copy onto a test). Unfortunately, in order to progress to more advanced topics, you need to understand the basics yourself. It's not sufficient to just be able to research it online. This is true in every field I can think of. That's why you can't just give people online access for every kind of test.

Cerium 4y ago
I had a basic physics teacher that allowed notes and gave us a chest sheet of his own which literally listed equations, described variables and explained the situation you might want the equation. People still failed. In a basic thermodynamics class the teacher said it was open everything but the person next to you: laptops, internet, the book etc. For fun I took the test without either book or laptop and did just fine. Others found it hard. In both cases the problems required thinking about the situation posed and which basic principles should be used to evaluate the problem. In my opinion, cheating from internet access or outside resources is not the problem, cheating by gogling exact problems is.

newbie2020 4y ago

It’s faster to look stuff up in the brain if it’s already there . Also having knowledge in the brain let’s it make connections you wouldn’t expect

denimnerd42 4y ago

I agree. We had open notes / open internet (no communication) exams at engineering school and they wouldn't help you pass the test if you didn't know the material. Maybe you could spend all exam period solving one of the questions if you weren't prepared but you'd be lucky to get a D.

As_You_Wish 4y ago

I agree. I dropped out of school in kindergarten. Why do I have to know my multiplication tables by heart? Why do I even have to know how to add two numbers together? I don't even know the alphabet and how to read and write - I'm just talking into a microphone and this is being written. I don't even know what country I live in or who the polical leader is, why should I? I can look it up online, I just don't have any desire to. As you can see, your argument is silly. It's important to know things, in your head. Of course I realize that you don't mean people shouldn't learn the alphabet or learn math, but where does one draw the line, where they don't have to learn stuff in their head? But there are always holes in what one is even able to look up. Maybe not for everyone, but for most. I just and a conversation the other day about this with someone and I was flabbergasted. The topic we discussed is that anyone can create a SQL database - anyone can. But do they normalize the database to 3rd or 4th normal form? He worked with a lot of corporations, and he said most companies don't have normalized databases. In university, that was a basic thing that they taught a semester course on that. An entire semester. Not to memorize anything, but just the general concepts. Someone can learn how to create an incorrectly structured database easily, but they don't even know about normalization, so how can you even look it up in the first place? And even if you do, somehow, learn what it is and look it up, I took the class for an entire semester. How is someone going to learn it in 15 minutes by looking it up? I looked up stuff in the book, too, and got info from the professor, and it took me an entire semester, and everyone else in the class, too. Unless you're calling us all dummies. There's all kinds of stuff like that. You don't know how to look up something if you don't even know it exists to look up, and it is not obvious.