5 ms·

To Really Learn, Quit Studying and Take a Test

112 points by bootload 16y ago

ionfish 16y ago
Here's one example of this kind of phenomenon. If you really want to understand a question, write a paper about it. It's easy enough to read five or ten research papers in an area, and if you do that thoroughly, and write extensive notes as you do so, you'll start to feel that you understand the area and the issues involved. However, if the way you learn is much like mine, when you try to start writing you will realise that you haven't grasped it with anything like the fullness that you had thought. Start writing anyway: you have a bibliography, and hopefully you have a rough idea of the thesis you want to defend. Write the literature review sections. Sketch outlines of arguments, and then try to take them apart. Find counterexamples. If your position has any merit, you will eventually figure out some convincing arguments. Flesh them out: write a first draft, as rough as you like. Improve it by smoothing out poor phrasing and removing extravagant, unsupported claims (it may only be obvious once you've written them that this is what they are). After some time you will have a reasonably presentable paper. More than that, though, you will actually understand the issues: how the various ideas in the field fit together, why you really disagree with some of the papers you've read, and perhaps even what your own take on it is. It's hard work! But writing is what makes the difference. If you just read a lot you can come away with the misapprehension that you understand the field, when in reality you just know a lot about what people have written.
- bartonfink 16y ago
  I think you're spot on. You don't really understand something unless you can explain it to an interested yet uninformed outsider.
  - mahmud 16y ago
    If you think writing prose solidifies your understanding of a subject, try formalizing its findings as knowledge .. in code. Most specifications can be turned into a database for some form of Prolog, play with it. At the very least, you might get yourself a type or model checker, or you might get lucky and actually get an expert system out of it.
  - arghnoname 16y ago
    I think the strength comes from being forced to defend your positions. Any paper worth its salt anticipates and defuses most likely objections. It really does force away any areas where one might have hand-waved. I think (as someone else mentioned too) teaching is generally good because one never knows where and when a challenge may present itself, but I'd suspect a paper gives deeper results, whereas teaching gives broader results, and both give more than study with neither.
  - ashearer 16y ago
    This is what I've found about programming as well. Computers are unforgiving of any gaps in your understanding, so explaining a process to a computer improves both you and the computer. Of course, not all ideas can be so neatly expressed in code. Constructing essays and explanations for people is similar, though it's easier to finesse gaps with rhetorical tricks.
- ojbyrne 16y ago
  Another way - if you really want to understand something, teach it. The preparation required to not embarrass yourself in front of students is huge.
- arghnoname 16y ago
  I never put two and two together, but I have long been amazed at how well I understand how the EU is put together, its history, etc, all just from having to write a 20 page paper on the topic once. This is true of other topics as well. More than once I've gone into it and came out the other side with the opposite opinion I started with. But despite the clear efficacy of this method, it never occurred to me to do it on my own, just to learn something. It's a lot of work and somewhat painful, but I suppose a more complete mastery always is.
- cema 16y ago
  And my experience is: I do not really understand it unless I can program it. To sum up this subthread: we all understand "it" better when we have experience of active involvement with it, not just passive studying.
- csomar 16y ago
  That works well, along with teaching; however, it takes extensively huge time and is very tiring (needs lot of efforts and patience).
kang 16y ago
Recalling information is not necessarily learning. That 'drawing-relation-diagrams' method contributes more towards understanding in my view. Also this is very limited in domain. As an example, here in India students learn English for an year or two before appearing in MBA examinations. One common observation is that many students do thousands of comprehensions and unseen passages but still stand nowhere in the 'reading between the lines' part, which is essentially, understanding. They spend all the time testing themselves, whereas 'testing' should follow after 'acquiring the knowledge'.
- aik 16y ago
  That's how it works in Japan too. Kids study English for 8 years and aren't able to speak a word or make basic linguistic connections. And it's all based on study and test taking.
  - jarek 16y ago
    Sadly, that's most of second language 'learning' in public schools in most places around the world.
    - me2i81 16y ago
      Almost everyone I've met from the Netherlands is fluent in English, and has a better grasp of English grammar than many Americans. I wonder why their result is so much better than the Japanese. There could be many reasons for this, including that English and Dutch are much closer than English and Japanese, that the Dutch have more contact with English-speaking people and media, or that the Dutch schools have a better way of teaching language than the Japanese schools. I've also found this to be the case with younger Germans.
- tygorius 16y ago
  Recalling information is not necessarily learning. That 'drawing-relation-diagrams' method contributes more towards understanding in my view. As far as I know, all definitions of learning include the ability to recall. As the article points out, the students that drew concept maps "felt" they understood the material, but later were actually worse at demonstrating it with the follow-up test on understanding of the material (ie, both recall of the previous facts and the ability to draw inferences from them). It's true that rote memorization is not the same as learning, but that's not what was being tested in the study. They specifically tested to determine both recall and understanding.
jacques_chester 16y ago
I'm not sure this is a new finding. I believe it's called the rehearsal or retrieval effect. I recall reading about it in a textbook about learning theory a decade ago. It's the basis of the study/summarise/recall/revise method.
drivebyacct2 16y ago
Man, ionfish beat me to the punch. Things I learn best, are things that I put into practice or write about. Things that I have an opinion about usually form some sort of dialog that I consider to be analogous to writing. This has been the case for me in every subject imaginable. Politics, math, science: all things that are conceptual with ideas. I know that history encompasses ideas, but rout memorization of facts has never appealed to me. Writing about a time period or such requires not just that I know the facts, but that I recall them, restructure them and express them in my own words. Even if it's just filtering through a couple extras areas of my brain, it makes a huge difference in my ability to remember important facts later. I've never tried to pay attention to my knowledge before and after a test, but I've taken too many tests that I didn't feel accurately represented the material to be leery of offering as much praise as the NYT does.
splat 16y ago
This is certainly true of physics or mathematics. You can read all the textbooks you want, but you can't claim to understand those subjects unless you sit down and actually solve some problems or prove some theorems.
- joubert 16y ago
  Ditto for programming.
s-phi-nl 16y ago
"What I cannot create, I do not understand." -- Richard Feynman
ghshephard 16y ago
ARGH. The title completely ignores the fact that the experiment-subjects that retained the information the best DIDN'T just take a test, they also retrieved it from memory and wrote what they knew. THEN they were tested on it. Horrible, horrible summary. A better title might have been "What they say is true, Read, Write, [Recite] - Then you'll test well." That is: "Read the Source Material - let's say a chapter of 10-15 pages. Close your book. Meditate on it for a minute or five. Now, without looking at the source material, write a one or two page summary of the material. Review your summary. Now, without looking at either the book, or your summary, verbally summarize the chapter." It's the foundation of every study plan I've ever seen - and the NYT article goes off on this "Just take a Test" tangent that isn't even supported by the observations they make.
- kenjackson 16y ago
  Isn't the one or two page summary the test? The Science paper refers to the "test" as retrieval practice. I guess I'm a little unclear on how this is different than what you state or what the NYTimes states.
- aik 16y ago
  This entire article is so full of that kind of crap - full of baseless insinuations and twisted meanings - it completely infuriates me. With the influx of recent NYT articles on education I'm nearly convinced the NYT is purposely trying to sabotage education (or rather they have a clear agenda, which I wish I knew where came from, that they are pushing through by twisting other people's words). As you say, retrieval practice is absolutely fundamental. Every person does it! Studying doesn't comprise of one method - concept mapping or repeated study - it's a combination! What kind of ridiculous study was this? Also, the article makes it seem like all experts purport 'concept mapping' as the "holy grail" of learning, that's it, nothing more. The test obviously assisted by assisting in organizing and structuring the information for them, in addition to highlighting key points from the readings. Also, the article never mentions what type of test they were taking. Multiple-choice? Essay? Short answer? Verbal? None of the above? That information would be the single most important piece of this article. To me this article seems to simply highlight how poor students are at understanding how they themselves learn and think. What NYT seems to want you to think: Test taking is the best way to learn. Forget in depth studying techniques or figuring out how you yourself learn. More of the truth: Tests (depending on how they are set up) are useful in assisting with learning in various ways, and super useful in providing feedback and assessing where a student is and what to focus on in the future.
binaryfinery 16y ago
Basically how I got a First from Imperial. Have fun 9 months of the year. Last three months before exams, get all the problems, tests and exams available and answer them all.
ericHosick 16y ago
We retain (1): 10% of what we read 20% of what we hear 30% of what we see 50% of what we hear and see 70% of what we say 90% of what we teach I used to lecture at a college. After learning about this the answer for me was like duh... I made my students teach my classes. (1) Edgar Dale, Audio-Visual Methods in Teaching 3rd Edition. Holt, Rinehart, and Winston (1969)
- joe_the_user 16y ago
  Actively learning and interacting with material is useful. I'm more trouble by glib and simplistic statistics supposedly illustrating this.
- barrkel 16y ago
  When I was a kid, whenever I was learning about some aspect of computing, I would have a running monologue in my head about how I would explain this to someone else, to the point that I would write out on a piece of paper an explanation of e.g. an adder composed out of boolean gates, or of how 2's complement made negative numbers work, out of sheer enthusiasm for what I was learning.
jcro41 16y ago
Here are a few links that I've found helpful in learning about this strategy, number 4 is my favorite. [1] Will that be on the Test? [2] The Power of Testing Memory [3] Practicing Information Retrieval is key to memory attention, study finds [4] Close the Book. Recall. Write It Down [1] http://www.linguistics.ucla.edu/people/hayes/Teaching/2006_Roediger_Karpicke_Review.pdf http://www.linguistics.ucla.edu/people/hayes/Teaching/2006_R... [2] http://www.linguistics.ucla.edu/people/hayes/Teaching/2006_Roediger_Karpicke_Review.pdf http://www.linguistics.ucla.edu/people/hayes/Teaching/2006_R... [3] http://news-info.wustl.edu/tips/page/normal/11091.html http://news-info.wustl.edu/tips/page/normal/11091.html [4] http://chronicle.com/article/Close-the-Book-Recall-Write/31819 http://chronicle.com/article/Close-the-Book-Recall-Write/318...
jeremydavid 16y ago
This is how I got through University. As I read my textbooks I wrote out every main point as a question. At the end of my reading I would have a huge list, and when I could answer them all I would finish studying. Never failed a test, or was ever forced to stay up late "cramming" for an exam.
csomar 16y ago
It may also be that the struggle involved in recalling something helps reinforce it in our brains. I think something happens here, it's like you send your mind a command: "You didn't memorize that thing, huh? Okay, be careful next time, this is needed for the test". Notice that when you read the next time, it's kind of your mind is prepared to put that information somewhere, in a manner that it can be retrieved later by the exam.
paufernandez 16y ago
That explains why I improve so much rewriting programs from scratch many times. Until it comes out naturally you have to do a lot of effort, which seems like a waste of time but...
detokaal 16y ago
You have to know what you don't know. In other words, you need a sense of the gaps that exist in your knowledge and then fill those gaps. This is a side effect of... http://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect http://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect ... and a key part of continuing education in any field. The best way for students of any age to grasp this is to take practice tests. Look at what you missed, and go fill in that gap. A similar method it to look at a completed project or paper and wait for the aha moment, "I didn't know you could do that." Then you go find the answers.

112 points by bootload 16y ago

Here's one example of this kind of phenomenon. If you really want to understand a question, write a paper about it. It's easy enough to read five or ten research papers in an area, and if you do that thoroughly, and write extensive notes as you do so, you'll start to feel that you understand the area and the issues involved. However, if the way you learn is much like mine, when you try to start writing you will realise that you haven't grasped it with anything like the fullness that you had thought. Start writing anyway: you have a bibliography, and hopefully you have a rough idea of the thesis you want to defend. Write the literature review sections. Sketch outlines of arguments, and then try to take them apart. Find counterexamples. If your position has any merit, you will eventually figure out some convincing arguments. Flesh them out: write a first draft, as rough as you like. Improve it by smoothing out poor phrasing and removing extravagant, unsupported claims (it may only be obvious once you've written them that this is what they are). After some time you will have a reasonably presentable paper. More than that, though, you will actually understand the issues: how the various ideas in the field fit together, why you really disagree with some of the papers you've read, and perhaps even what your own take on it is. It's hard work! But writing is what makes the difference. If you just read a lot you can come away with the misapprehension that you understand the field, when in reality you just know a lot about what people have written.

bartonfink 16y ago
I think you're spot on. You don't really understand something unless you can explain it to an interested yet uninformed outsider.
- mahmud 16y ago
  If you think writing prose solidifies your understanding of a subject, try formalizing its findings as knowledge .. in code. Most specifications can be turned into a database for some form of Prolog, play with it. At the very least, you might get yourself a type or model checker, or you might get lucky and actually get an expert system out of it.
- arghnoname 16y ago
  I think the strength comes from being forced to defend your positions. Any paper worth its salt anticipates and defuses most likely objections. It really does force away any areas where one might have hand-waved. I think (as someone else mentioned too) teaching is generally good because one never knows where and when a challenge may present itself, but I'd suspect a paper gives deeper results, whereas teaching gives broader results, and both give more than study with neither.
- ashearer 16y ago
  This is what I've found about programming as well. Computers are unforgiving of any gaps in your understanding, so explaining a process to a computer improves both you and the computer. Of course, not all ideas can be so neatly expressed in code. Constructing essays and explanations for people is similar, though it's easier to finesse gaps with rhetorical tricks.
ojbyrne 16y ago
Another way - if you really want to understand something, teach it. The preparation required to not embarrass yourself in front of students is huge.
arghnoname 16y ago
I never put two and two together, but I have long been amazed at how well I understand how the EU is put together, its history, etc, all just from having to write a 20 page paper on the topic once. This is true of other topics as well. More than once I've gone into it and came out the other side with the opposite opinion I started with. But despite the clear efficacy of this method, it never occurred to me to do it on my own, just to learn something. It's a lot of work and somewhat painful, but I suppose a more complete mastery always is.
cema 16y ago
And my experience is: I do not really understand it unless I can program it. To sum up this subthread: we all understand "it" better when we have experience of active involvement with it, not just passive studying.
csomar 16y ago
That works well, along with teaching; however, it takes extensively huge time and is very tiring (needs lot of efforts and patience).

kang 16y ago

Recalling information is not necessarily learning. That 'drawing-relation-diagrams' method contributes more towards understanding in my view. Also this is very limited in domain. As an example, here in India students learn English for an year or two before appearing in MBA examinations. One common observation is that many students do thousands of comprehensions and unseen passages but still stand nowhere in the 'reading between the lines' part, which is essentially, understanding. They spend all the time testing themselves, whereas 'testing' should follow after 'acquiring the knowledge'.

aik 16y ago
That's how it works in Japan too. Kids study English for 8 years and aren't able to speak a word or make basic linguistic connections. And it's all based on study and test taking.
- jarek 16y ago
  Sadly, that's most of second language 'learning' in public schools in most places around the world.
  - me2i81 16y ago
    Almost everyone I've met from the Netherlands is fluent in English, and has a better grasp of English grammar than many Americans. I wonder why their result is so much better than the Japanese. There could be many reasons for this, including that English and Dutch are much closer than English and Japanese, that the Dutch have more contact with English-speaking people and media, or that the Dutch schools have a better way of teaching language than the Japanese schools. I've also found this to be the case with younger Germans.
tygorius 16y ago
Recalling information is not necessarily learning. That 'drawing-relation-diagrams' method contributes more towards understanding in my view. As far as I know, all definitions of learning include the ability to recall. As the article points out, the students that drew concept maps "felt" they understood the material, but later were actually worse at demonstrating it with the follow-up test on understanding of the material (ie, both recall of the previous facts and the ability to draw inferences from them). It's true that rote memorization is not the same as learning, but that's not what was being tested in the study. They specifically tested to determine both recall and understanding.

jacques_chester 16y ago

I'm not sure this is a new finding. I believe it's called the rehearsal or retrieval effect. I recall reading about it in a textbook about learning theory a decade ago. It's the basis of the study/summarise/recall/revise method.

drivebyacct2 16y ago

Man, ionfish beat me to the punch. Things I learn best, are things that I put into practice or write about. Things that I have an opinion about usually form some sort of dialog that I consider to be analogous to writing. This has been the case for me in every subject imaginable. Politics, math, science: all things that are conceptual with ideas. I know that history encompasses ideas, but rout memorization of facts has never appealed to me. Writing about a time period or such requires not just that I know the facts, but that I recall them, restructure them and express them in my own words. Even if it's just filtering through a couple extras areas of my brain, it makes a huge difference in my ability to remember important facts later. I've never tried to pay attention to my knowledge before and after a test, but I've taken too many tests that I didn't feel accurately represented the material to be leery of offering as much praise as the NYT does.

splat 16y ago

This is certainly true of physics or mathematics. You can read all the textbooks you want, but you can't claim to understand those subjects unless you sit down and actually solve some problems or prove some theorems.

joubert 16y ago
Ditto for programming.

s-phi-nl 16y ago

"What I cannot create, I do not understand." -- Richard Feynman

ghshephard 16y ago

ARGH. The title completely ignores the fact that the experiment-subjects that retained the information the best DIDN'T just take a test, they also retrieved it from memory and wrote what they knew. THEN they were tested on it. Horrible, horrible summary. A better title might have been "What they say is true, Read, Write, [Recite] - Then you'll test well." That is: "Read the Source Material - let's say a chapter of 10-15 pages. Close your book. Meditate on it for a minute or five. Now, without looking at the source material, write a one or two page summary of the material. Review your summary. Now, without looking at either the book, or your summary, verbally summarize the chapter." It's the foundation of every study plan I've ever seen - and the NYT article goes off on this "Just take a Test" tangent that isn't even supported by the observations they make.

kenjackson 16y ago
Isn't the one or two page summary the test? The Science paper refers to the "test" as retrieval practice. I guess I'm a little unclear on how this is different than what you state or what the NYTimes states.
aik 16y ago
This entire article is so full of that kind of crap - full of baseless insinuations and twisted meanings - it completely infuriates me. With the influx of recent NYT articles on education I'm nearly convinced the NYT is purposely trying to sabotage education (or rather they have a clear agenda, which I wish I knew where came from, that they are pushing through by twisting other people's words). As you say, retrieval practice is absolutely fundamental. Every person does it! Studying doesn't comprise of one method - concept mapping or repeated study - it's a combination! What kind of ridiculous study was this? Also, the article makes it seem like all experts purport 'concept mapping' as the "holy grail" of learning, that's it, nothing more. The test obviously assisted by assisting in organizing and structuring the information for them, in addition to highlighting key points from the readings. Also, the article never mentions what type of test they were taking. Multiple-choice? Essay? Short answer? Verbal? None of the above? That information would be the single most important piece of this article. To me this article seems to simply highlight how poor students are at understanding how they themselves learn and think. What NYT seems to want you to think: Test taking is the best way to learn. Forget in depth studying techniques or figuring out how you yourself learn. More of the truth: Tests (depending on how they are set up) are useful in assisting with learning in various ways, and super useful in providing feedback and assessing where a student is and what to focus on in the future.

binaryfinery 16y ago

Basically how I got a First from Imperial. Have fun 9 months of the year. Last three months before exams, get all the problems, tests and exams available and answer them all.

ericHosick 16y ago

We retain (1): 10% of what we read 20% of what we hear 30% of what we see 50% of what we hear and see 70% of what we say 90% of what we teach I used to lecture at a college. After learning about this the answer for me was like duh... I made my students teach my classes. (1) Edgar Dale, Audio-Visual Methods in Teaching 3rd Edition. Holt, Rinehart, and Winston (1969)

joe_the_user 16y ago
Actively learning and interacting with material is useful. I'm more trouble by glib and simplistic statistics supposedly illustrating this.
barrkel 16y ago
When I was a kid, whenever I was learning about some aspect of computing, I would have a running monologue in my head about how I would explain this to someone else, to the point that I would write out on a piece of paper an explanation of e.g. an adder composed out of boolean gates, or of how 2's complement made negative numbers work, out of sheer enthusiasm for what I was learning.

jcro41 16y ago

Here are a few links that I've found helpful in learning about this strategy, number 4 is my favorite. [1] Will that be on the Test? [2] The Power of Testing Memory [3] Practicing Information Retrieval is key to memory attention, study finds [4] Close the Book. Recall. Write It Down [1] http://www.linguistics.ucla.edu/people/hayes/Teaching/2006_Roediger_Karpicke_Review.pdf http://www.linguistics.ucla.edu/people/hayes/Teaching/2006_R... [2] http://www.linguistics.ucla.edu/people/hayes/Teaching/2006_Roediger_Karpicke_Review.pdf http://www.linguistics.ucla.edu/people/hayes/Teaching/2006_R... [3] http://news-info.wustl.edu/tips/page/normal/11091.html http://news-info.wustl.edu/tips/page/normal/11091.html [4] http://chronicle.com/article/Close-the-Book-Recall-Write/31819 http://chronicle.com/article/Close-the-Book-Recall-Write/318...

jeremydavid 16y ago

This is how I got through University. As I read my textbooks I wrote out every main point as a question. At the end of my reading I would have a huge list, and when I could answer them all I would finish studying. Never failed a test, or was ever forced to stay up late "cramming" for an exam.

csomar 16y ago

It may also be that the struggle involved in recalling something helps reinforce it in our brains. I think something happens here, it's like you send your mind a command: "You didn't memorize that thing, huh? Okay, be careful next time, this is needed for the test". Notice that when you read the next time, it's kind of your mind is prepared to put that information somewhere, in a manner that it can be retrieved later by the exam.

paufernandez 16y ago

That explains why I improve so much rewriting programs from scratch many times. Until it comes out naturally you have to do a lot of effort, which seems like a waste of time but...

detokaal 16y ago

You have to know what you don't know. In other words, you need a sense of the gaps that exist in your knowledge and then fill those gaps. This is a side effect of... http://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect http://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect ... and a key part of continuing education in any field. The best way for students of any age to grasp this is to take practice tests. Look at what you missed, and go fill in that gap. A similar method it to look at a completed project or paper and wait for the aha moment, "I didn't know you could do that." Then you go find the answers.