5 ms·

I like the idea presented in TFA, but as a small counter-anecdote, the mileage I've gotten out of my reading has skyrocketed since I started doing spaced repeti

by kqr 3y ago

I like the idea presented in TFA, but as a small counter-anecdote, the mileage I've gotten out of my reading has skyrocketed since I started doing spaced repetition. I can now actually whip out enumerative combinatorics to solve problems, or do on the spot estimations of the standard error of log-odds differences from contingency tables, or compute the power or any number to any other with mental maths, without looking it up in my notes first. That's immensely powerful. But it also doesn't stop there. Since I now remember far more higher-level concepts, I can also make sense of more advanced reading in a way I was unable to when I had to dip back and forth between my notes and the text to try to piece things together. Essentially, remembering what I've read has allowed me to raise the level of abstraction of my thinking, which has been more helpful than I expected it to, across a number of different situations. ---- That said, I'm only a few months into my spaced repetition journey, so maybe there's a balance to strike and a judgment to make between forgetting and remembering, and I'm sure that's something I'll get better at finding as I get practise. Maybe the first few months of spaced repetition is the time before diminishing returns set in. I don't know but I'm excited to find out.

habitue 3y ago
Yeah, it really seems like this article is describing the default. It's good to accept and understand that writing stuff into notes doesn't ever imply you'll read it again. That doesn't mean that if you make the effort to read the notes again you won't get value out of them. It's just saying "You realize you aren't on track to read any of this ever again right?"
- emodendroket 3y ago
  I think they're different kinds of notes you're talking about. If you're doing applied study to learn something, sure. But most notes are not that directed or profound; it's more like "I liked this song on the radio; let me check out this artist whenever I have time." If you haven't bothered in a month you probably aren't ever going to. For directed study, though, I find making the notes if oten more helpful than referring to them again.
  - hxugufjfjf 3y ago
    I find this the case, but I've also found anomalies where I one day watch a movie that's been in my watchlist for years. One day something that seems stale just seems new again. Not sure why.
    - emodendroket 3y ago
      That's also true but if I haven't felt compelled to upgrade it from a notebook scribble it's somewhat less likely
drawkbox 3y ago
Spaced repetition is how we truly learn everything when dealing with it a little bit every day. A really great, fun game on this is from Nicki Case called "How To Remember Anything Forever-ish" [1] Additionally, the site has some other amazing games that really should be part of everyone's education and are Oregon Trail like in their impact. One of my favorites is the "Evolution of Trust" about game theory. [2] Probably the best game theory game out there. As an aside, you can program yourself and people with spaced repetition. Lots of propaganda uses it heavily but it can be used for good and bad. If you hear small facts everyday or small bits of promotion/propaganda everyday, it can over time evoke an pavlovian like response to the information, either good or bad. Using it can also bump up things you want to know about, and be aware when external information is being pumped. Basically spaced repetition mimics how you'd naturally come across something repetitively and the learning becomes automatic, it is iterative. Educational games like Nicky Case makes are awesome and really do help simplify and solidify certain topics so check them out. [3] [1] https://ncase.me/remember/ https://ncase.me/remember/ [2] https://ncase.me/trust/ https://ncase.me/trust/ [3] https://ncase.me/ https://ncase.me/
- kqr 3y ago
  I would argue most of the things you are mentioning are plain repetition, and not spaced repetition, which has an element of optimal spacing to minimise reviewing time while keeping retention high. Similarly, propaganda and the like builds on the recency/frequency heuristic in System 1 which makes things you commonly hear seem true, and it's triggered by repetition -- perhaps even more efficiently than by spaced repetition.
  - drawkbox 3y ago
    Yeah agreed. Spaced repetition is just a formal procedure of testing + time that taps into already prevalent repetition function to promote information to a higher place in your mind. The brain thinks if it hears about something frequently, it is more important. If those are spaced out you can stretch that but it is also naturally how we learn. Spaced repetition is based on iterative learning, small bits compounding to refine skills/memory. The point is to get it into a higher point in memory by regularly revisiting. Spaced repetition tunes into the natural repetition/learning cycle at a point that becomes muscle memory almost. Once it is in there, the spacing can grow and it can bring that all back up. In usability for instance three or less items feels simple, over seven feels complex. When you repeat something about seven times, whether that is immediate or over time, it moves to a higher importance in your mind/memory. When you stretch these out you can train your brain via spaced repetition but it all comes back to that it is a regular iterative practice and doesn't have to be the exact specs of spaced repetition. We are habitual creatures so if you can get a pattern going of regular learning of a topic, it becomes easier to process in small chunks over time. You can also use this to program or promote ideas and products as people when they hear things over a certain amount of times, as you mention, it moves up the importance in their thought process. When working on a side project for instance, if you work on it just a little bit everyday, or a medium amount every week, it will keep rolling. If you take large breaks you have to work to reload that information and it starts to stagnate. Small, repetitive and many times, spaced iterations, compound into shipping products if you set it up right. That can be used for almost anything from education to career to hobbies and more. At the core of spaced repetition is the spacing (time + testing) but really the small bits that are easy to repeat. Small bits of info regularly is almost more important than the spacing, "testing" size almost more important than the "time". The spaced repetition index cards techniques are all about this, small bits of info tested over time, stretching the time to make it last longer. This is why cramming doesn't work very long, you have to revisit regularly and over a certain amount of times with space in between and they have to be smaller chunks of information to remember, you can't really do that when the thing is due tomorrow. Our current work environments look more like cramming projects rather than open/closed spaced repetition and that is a problem.
AlexSW 3y ago
Where are you running into problems which require these mathematical techniques? I don't ever recall wishing I'd memorised any of these. For context, I did a Maths degree, so I'm familiar with them all. I just never find myself needing them.
- kqr 3y ago
  Mainly for back of the napkin calculations to evaluate feasibility of solutions/architectures/algorithms, and to evaluate whether to invest in an opportunity. Granted, they could all be replaced by computer evaluation, but when someone shows me some numbers and wants a quick opinion, I've found that if I ask if I can take a moment to type it into a device people are more likely to go "that sounds complicated, never mind" but if I only jot down some numbers on paper they are more likely to put weight on the conclusion.
padraigf 3y ago
Can you give a brief description of your process? I'm familiar with spaced-repetition, and have played around with Anki a bit. I'm just not sure how I would integrate it into my reading.
- kqr 3y ago
  I described it a couple of weeks ago here: https://two-wrongs.com/reading-slightly-more-incrementally.html https://two-wrongs.com/reading-slightly-more-incrementally.h... But I can try to summarise/expand. A large part of it is learning to write good flashcard prompts. That was like 80 % of the hurdle for me. It does not come naturally for me, so I needed to look some tips up and then I notice I'm getting better with practise. I started by reading https://andymatuschak.org/prompts/ https://andymatuschak.org/prompts/ and Wozniak's https://www.supermemo.com/en/blog/twenty-rules-of-formulating-knowledge https://www.supermemo.com/en/blog/twenty-rules-of-formulatin.... At least for me, another 15 % came from making it convenient to create flashcards. Actually designing the flashcard prompt takes a bit of mental effort (it's hard to write good prompts!) so if it then is technically challenging for me to create the flashcard from it, it's less likely to happen. I have a script that takes in essentially Markdown sections with bullet lists where each list item becomes its own flashcard with the title given by the Markdown heading. That works for fast entry for me. I write the Markdown on my phone, and then every now and then I use the script to batch import all I've written so far into my spaced repetition software. The last 5 % for me was switching over from writing my plain old notes into writing flashcards instead. That bit was fairly easy because I was already in the habit of making notes. I stop and write notes on: - Things that connect to something else I know, even if the authors don't point that connection out. - Things that seem insightful, even if I don't yet know what the insight is. - Things that surprise or delight me, or make me sad, or whatever. Any emotional reaction. - Fundamental facts that other results build on, unless I'm already very confident that I know that fact even if someone would quiz me unexpectedly. (So I wouldn't write a flashcard for Bayes' rule because I can recite it in my sleep – but I do have a flashcard for the log-odds version of it, where posterior = prior + sensitivity - false positive rate, because I haven't yet seen/used that version as much (though clearly I did remember it just now).) - Steps or details of a simple process that I'll likely want to follow later without having to reference the book. This is probably not an ideal way to select things to remember, but I have to go by something and these are usually fairly good points at which I take a short break from reading anyway, so I might as well create a flashcard. Then comes actually studying the flashcards, but that's not what you asked about so I stop here!
dassarin 3y ago
Can you suggest any books or resources on how to effectively apply spaced learning?
- kqr 3y ago
  Some ideas of where one might start. Please don't assume I endorse everything on all of these links – pick and choose the pieces that seem meaningful to you! https://gwern.net/spaced-repetition#using-it https://gwern.net/spaced-repetition#using-it http://super-memory.com/help/il.htm#What_is_incremental_learning.3F http://super-memory.com/help/il.htm#What_is_incremental_lear... https://www.supermemo.com/en/blog/twenty-rules-of-formulating-knowledge https://www.supermemo.com/en/blog/twenty-rules-of-formulatin...
unhammer 3y ago
Writing flash cards and doing the exercises takes time. How do you decide/prioritise what you need to remember (in the Age of Search Engines)?
- kqr 3y ago
  I avoid thinking about the opportunity cost and predicting what to learn for maximum efficiency – that could drive anyone crazy! I mainly follow my emotions and varying interests and try to convince myself that most of what I learn can be very useful, it just rests on me to find ways to use it. It helps that I'm a big fan of applying wisdom from different disciplines to find unexpected solutions to problems. I do however focus a lot on fundamental insights, the things that actually help me guide a web search in the first place. If I didn't have rooted deeply in my brain that the sample error is composed of bias, between-unit variance, and sample size, I wouldn't even know to start looking for ways to reduce between-unit variance as an economical alternative to increasing sample size.
- BeetleB 3y ago
  Michael Nielsen did an analysis on this based on his own usage. He found that the time he spent reviewing a card over a lifetime is about 10 minutes (i.e. he'd spend 10 minutes of his life reviewing the card over and over before it's memorized "for good"). So his calculus was: "In the future, of all the times I will need this information, will I spend more than 10 minutes looking it up?" If the answer is no, he won't bother creating flashcards. If the answer is yes, he will. Now that doesn't account for the time spent creating the card, but for most cards it will definitely take under a minute to create - so it doesn't change things much. Writing cards vs doing exercises is comparing very different concerns. One needs to eat as well, but it wouldn't make sense to ask "Write cards vs eating." Flashcards are for recall. Exercises are for understanding. It is true if you use a concept over and over in practice exercises, it will aid in recall as well. But for concepts you use less often (i.e. fewer exercises), you are more likely to forget them. Flashcards guard against that. Case in point: I took tons of engineering and physics courses. So the usual undergraduate calculus, along with trigonometric identities, are ingrained in my mind even though I haven't used them in over a decade. Now consider complex analysis (or complex calculus). It was one of my favorite math classes. But I rarely applied that knowledge in other courses. As a result, despite the fact that I basically reviewed the material twice after taking the course, I still remember almost none of it. I could force myself to do complex analysis exercises occasionally for the rest of my life. Or I could put them in flashcards. The former will definitely lead to a richer, deeper understanding, but is impractical due to the heavy time requirements. The latter is a good compromise.

by kqr 3y ago

habitue 3y ago

Yeah, it really seems like this article is describing the default. It's good to accept and understand that writing stuff into notes doesn't ever imply you'll read it again. That doesn't mean that if you make the effort to read the notes again you won't get value out of them. It's just saying "You realize you aren't on track to read any of this ever again right?"

emodendroket 3y ago
I think they're different kinds of notes you're talking about. If you're doing applied study to learn something, sure. But most notes are not that directed or profound; it's more like "I liked this song on the radio; let me check out this artist whenever I have time." If you haven't bothered in a month you probably aren't ever going to. For directed study, though, I find making the notes if oten more helpful than referring to them again.
- hxugufjfjf 3y ago
  I find this the case, but I've also found anomalies where I one day watch a movie that's been in my watchlist for years. One day something that seems stale just seems new again. Not sure why.
  - emodendroket 3y ago
    That's also true but if I haven't felt compelled to upgrade it from a notebook scribble it's somewhat less likely

drawkbox 3y ago

Spaced repetition is how we truly learn everything when dealing with it a little bit every day. A really great, fun game on this is from Nicki Case called "How To Remember Anything Forever-ish" [1] Additionally, the site has some other amazing games that really should be part of everyone's education and are Oregon Trail like in their impact. One of my favorites is the "Evolution of Trust" about game theory. [2] Probably the best game theory game out there. As an aside, you can program yourself and people with spaced repetition. Lots of propaganda uses it heavily but it can be used for good and bad. If you hear small facts everyday or small bits of promotion/propaganda everyday, it can over time evoke an pavlovian like response to the information, either good or bad. Using it can also bump up things you want to know about, and be aware when external information is being pumped. Basically spaced repetition mimics how you'd naturally come across something repetitively and the learning becomes automatic, it is iterative. Educational games like Nicky Case makes are awesome and really do help simplify and solidify certain topics so check them out. [3] [1] https://ncase.me/remember/ https://ncase.me/remember/ [2] https://ncase.me/trust/ https://ncase.me/trust/ [3] https://ncase.me/ https://ncase.me/

kqr 3y ago
I would argue most of the things you are mentioning are plain repetition, and not spaced repetition, which has an element of optimal spacing to minimise reviewing time while keeping retention high. Similarly, propaganda and the like builds on the recency/frequency heuristic in System 1 which makes things you commonly hear seem true, and it's triggered by repetition -- perhaps even more efficiently than by spaced repetition.
- drawkbox 3y ago
  Yeah agreed. Spaced repetition is just a formal procedure of testing + time that taps into already prevalent repetition function to promote information to a higher place in your mind. The brain thinks if it hears about something frequently, it is more important. If those are spaced out you can stretch that but it is also naturally how we learn. Spaced repetition is based on iterative learning, small bits compounding to refine skills/memory. The point is to get it into a higher point in memory by regularly revisiting. Spaced repetition tunes into the natural repetition/learning cycle at a point that becomes muscle memory almost. Once it is in there, the spacing can grow and it can bring that all back up. In usability for instance three or less items feels simple, over seven feels complex. When you repeat something about seven times, whether that is immediate or over time, it moves to a higher importance in your mind/memory. When you stretch these out you can train your brain via spaced repetition but it all comes back to that it is a regular iterative practice and doesn't have to be the exact specs of spaced repetition. We are habitual creatures so if you can get a pattern going of regular learning of a topic, it becomes easier to process in small chunks over time. You can also use this to program or promote ideas and products as people when they hear things over a certain amount of times, as you mention, it moves up the importance in their thought process. When working on a side project for instance, if you work on it just a little bit everyday, or a medium amount every week, it will keep rolling. If you take large breaks you have to work to reload that information and it starts to stagnate. Small, repetitive and many times, spaced iterations, compound into shipping products if you set it up right. That can be used for almost anything from education to career to hobbies and more. At the core of spaced repetition is the spacing (time + testing) but really the small bits that are easy to repeat. Small bits of info regularly is almost more important than the spacing, "testing" size almost more important than the "time". The spaced repetition index cards techniques are all about this, small bits of info tested over time, stretching the time to make it last longer. This is why cramming doesn't work very long, you have to revisit regularly and over a certain amount of times with space in between and they have to be smaller chunks of information to remember, you can't really do that when the thing is due tomorrow. Our current work environments look more like cramming projects rather than open/closed spaced repetition and that is a problem.

AlexSW 3y ago

Where are you running into problems which require these mathematical techniques? I don't ever recall wishing I'd memorised any of these. For context, I did a Maths degree, so I'm familiar with them all. I just never find myself needing them.

kqr 3y ago
Mainly for back of the napkin calculations to evaluate feasibility of solutions/architectures/algorithms, and to evaluate whether to invest in an opportunity. Granted, they could all be replaced by computer evaluation, but when someone shows me some numbers and wants a quick opinion, I've found that if I ask if I can take a moment to type it into a device people are more likely to go "that sounds complicated, never mind" but if I only jot down some numbers on paper they are more likely to put weight on the conclusion.

padraigf 3y ago

Can you give a brief description of your process? I'm familiar with spaced-repetition, and have played around with Anki a bit. I'm just not sure how I would integrate it into my reading.

kqr 3y ago
I described it a couple of weeks ago here: https://two-wrongs.com/reading-slightly-more-incrementally.html https://two-wrongs.com/reading-slightly-more-incrementally.h... But I can try to summarise/expand. A large part of it is learning to write good flashcard prompts. That was like 80 % of the hurdle for me. It does not come naturally for me, so I needed to look some tips up and then I notice I'm getting better with practise. I started by reading https://andymatuschak.org/prompts/ https://andymatuschak.org/prompts/ and Wozniak's https://www.supermemo.com/en/blog/twenty-rules-of-formulating-knowledge https://www.supermemo.com/en/blog/twenty-rules-of-formulatin.... At least for me, another 15 % came from making it convenient to create flashcards. Actually designing the flashcard prompt takes a bit of mental effort (it's hard to write good prompts!) so if it then is technically challenging for me to create the flashcard from it, it's less likely to happen. I have a script that takes in essentially Markdown sections with bullet lists where each list item becomes its own flashcard with the title given by the Markdown heading. That works for fast entry for me. I write the Markdown on my phone, and then every now and then I use the script to batch import all I've written so far into my spaced repetition software. The last 5 % for me was switching over from writing my plain old notes into writing flashcards instead. That bit was fairly easy because I was already in the habit of making notes. I stop and write notes on: - Things that connect to something else I know, even if the authors don't point that connection out. - Things that seem insightful, even if I don't yet know what the insight is. - Things that surprise or delight me, or make me sad, or whatever. Any emotional reaction. - Fundamental facts that other results build on, unless I'm already very confident that I know that fact even if someone would quiz me unexpectedly. (So I wouldn't write a flashcard for Bayes' rule because I can recite it in my sleep – but I do have a flashcard for the log-odds version of it, where posterior = prior + sensitivity - false positive rate, because I haven't yet seen/used that version as much (though clearly I did remember it just now).) - Steps or details of a simple process that I'll likely want to follow later without having to reference the book. This is probably not an ideal way to select things to remember, but I have to go by something and these are usually fairly good points at which I take a short break from reading anyway, so I might as well create a flashcard. Then comes actually studying the flashcards, but that's not what you asked about so I stop here!

dassarin 3y ago

Can you suggest any books or resources on how to effectively apply spaced learning?

kqr 3y ago
Some ideas of where one might start. Please don't assume I endorse everything on all of these links – pick and choose the pieces that seem meaningful to you! https://gwern.net/spaced-repetition#using-it https://gwern.net/spaced-repetition#using-it http://super-memory.com/help/il.htm#What_is_incremental_learning.3F http://super-memory.com/help/il.htm#What_is_incremental_lear... https://www.supermemo.com/en/blog/twenty-rules-of-formulating-knowledge https://www.supermemo.com/en/blog/twenty-rules-of-formulatin...

unhammer 3y ago

Writing flash cards and doing the exercises takes time. How do you decide/prioritise what you need to remember (in the Age of Search Engines)?

kqr 3y ago
I avoid thinking about the opportunity cost and predicting what to learn for maximum efficiency – that could drive anyone crazy! I mainly follow my emotions and varying interests and try to convince myself that most of what I learn can be very useful, it just rests on me to find ways to use it. It helps that I'm a big fan of applying wisdom from different disciplines to find unexpected solutions to problems. I do however focus a lot on fundamental insights, the things that actually help me guide a web search in the first place. If I didn't have rooted deeply in my brain that the sample error is composed of bias, between-unit variance, and sample size, I wouldn't even know to start looking for ways to reduce between-unit variance as an economical alternative to increasing sample size.
BeetleB 3y ago
Michael Nielsen did an analysis on this based on his own usage. He found that the time he spent reviewing a card over a lifetime is about 10 minutes (i.e. he'd spend 10 minutes of his life reviewing the card over and over before it's memorized "for good"). So his calculus was: "In the future, of all the times I will need this information, will I spend more than 10 minutes looking it up?" If the answer is no, he won't bother creating flashcards. If the answer is yes, he will. Now that doesn't account for the time spent creating the card, but for most cards it will definitely take under a minute to create - so it doesn't change things much. Writing cards vs doing exercises is comparing very different concerns. One needs to eat as well, but it wouldn't make sense to ask "Write cards vs eating." Flashcards are for recall. Exercises are for understanding. It is true if you use a concept over and over in practice exercises, it will aid in recall as well. But for concepts you use less often (i.e. fewer exercises), you are more likely to forget them. Flashcards guard against that. Case in point: I took tons of engineering and physics courses. So the usual undergraduate calculus, along with trigonometric identities, are ingrained in my mind even though I haven't used them in over a decade. Now consider complex analysis (or complex calculus). It was one of my favorite math classes. But I rarely applied that knowledge in other courses. As a result, despite the fact that I basically reviewed the material twice after taking the course, I still remember almost none of it. I could force myself to do complex analysis exercises occasionally for the rest of my life. Or I could put them in flashcards. The former will definitely lead to a richer, deeper understanding, but is impractical due to the heavy time requirements. The latter is a good compromise.