6 ms·

I'm surprised that no spaced repetition systems seem to exploit relationships between facts/cards. One might imagine that capturing a graph of similar ideas wou

by nmca 4y ago

I'm surprised that no spaced repetition systems seem to exploit relationships between facts/cards. One might imagine that capturing a graph of similar ideas would allow for better algorithms.

Scaevolus 4y ago
Anki supports linking cards when they're different facets of the same information (e.g., each language source/target for a given vocab word): https://faqs.ankiweb.net/linking-cards-together.html https://faqs.ankiweb.net/linking-cards-together.html More precision isn't that important-- at worst, you slightly over-review some information, but that's no different from using a flash card's information in normal life outside of a review period.
_Algernon_ 4y ago
The Anki FAQ goes into the reasoning a bit: https://faqs.ankiweb.net/linking-cards-together.html https://faqs.ankiweb.net/linking-cards-together.html >Some people want to extend this link between arbitrary cards. They want to be able to tell Anki "after showing me this card, show me that card", or "don’t show me that card until I know this card well enough". This might sound like a nice idea in theory, but in practice it is not practical. >For one, unlike the sibling card case above, you would have to define all the relations yourself. Entering new notes into Anki would become a complicated process, as you’d have to search through the rest of the deck and assign relationships between the old and new material. >Secondly, remember that Anki is using an algorithm to determine when the optimum time to show you material again is. Adding constraints to card display that cause cards to display earlier or later than they were supposed to will make the spaced repetition system less effective, leading to more work than necessary, or forgotten cards. >The most effective way to use Anki is to make each note you see independent from other notes. Instead of trying to join similar words together, you’ll be better off if you can determine the differences between them. Synonyms are rarely completely interchangeable - they tend to have nuances attached, and it’s not unusual for a sentence to become strange if one synonym is replaced with another.
- nmca 4y ago
  This seems to boil down to "it would be more complicated", mixed with "the trivial thing is a bad idea", both of which seem true. But I can imagine a more complicated scheme could be much better; done right.
  - _Algernon_ 4y ago
    I agree. It makes sense for cards which link previously learned facts to only show up once the base facts have been learned, for example. But I'm not sure how to design a good UI for it. I can easily see it as resulting in too much friction so nobody uses it.
    - s_m_t 4y ago
      Doesn't just putting the cards in order of concepts essentially accomplish this? When I was doing Anki I adjusted my new card rate so my retention rate stayed above 90%. If you do it this way then your chances of getting stuck in a certain areas of the dependency graph (if you were to model it) is pretty low. If it does end up happening then do some extra study on those concepts outside of SRS. This way you don't have to model the actual dependency graph. You really shouldn't be doing all your study in SRS anyways because so called "flashcard blindness" (i.e. only being able to recall the concepts in the context of your SRS program) is a real thing. Furthermore, in my experience, what really made Anki effective for me was to write my new cards just in time. Meaning no pre-made decks except for something like the primitives of what you are studying (i.e. symbol names, alphabets, extreme base level concepts, etc). When you write your cards just in time you can really connect the context of what you are learning to the concept. From this perspective it really seems strange to build an entire ontology based on dependency graphs of a concept you haven't even learned yet!
type-r 4y ago
I have found this to be a huge missing piece of any SRSs I've come across. Personally, I have a pipeline that looks at what knowledge I've already learned and only then reveals successive information. For example, I'm learning numbers in Spanish. First, I ask myself to go from doce => 12; then, 12 => doce. Then, once I have a few numbers in my memory, I can start to combine those into diez + dos = ? (answer doce). The idea is that this last example requires recalling three previously-learned Spanish numbers to arrive at the answer. This is a trivial example but this underlying structure of successive layers of knowledge shows up everywhere and is horribly under-exploited by our current learning systems. Programmatically this can be done. Manually, this stuff takes way too much work.
- DennisP 4y ago
  How would you do it programmatically? Seems like you'd always need someone to give it the relationships.
  - trane_project 4y ago
    Yes. That's required. I think they meant that the program uses those relationships to automatically advance the student to the next topic when it makes sense.
- trane_project 4y ago
  Check out my project: https://GitHub.com/trane-project/trane https://GitHub.com/trane-project/trane It does exactly this because I also thought this was a critical limitation of current systems. I have a feature coming out soon (already merged to master but I need to make a proper release) which should make it way easier to use with Markdown files acting as the front and back of flashcards that are automatically picked up when starting up the program.
  - aporetics 4y ago
    If you want to look at prior art, take a look at org-drill
kashunstva 4y ago
> I'm surprised that no spaced repetition systems seem to exploit relationships between facts/cards. In fact, the Anki community largely endorses as an article of faith the complete insular atomicity of cards. I’ve always treated that orthodoxy with skepticism because I’m pretty sure that’s not how my brain works. Nor anyone else’s, probably.
aliceryhl 4y ago
An interesting example is chessable, which is a spaced repetition system for chess. It keeps track of each move in a sequence of moves separately, but every time it asks you to re-learn a specific move, you are asked to play out the whole sequence, instead of just being shown the position and asking you to play the next move.
optionalsquid 4y ago
Wanikani [1], an SRS for learning Japanese kanji and vocabulary, kinda works like that, though on a simpler level than what I think you envision. Basically their system involves first learning to recognize common parts of kanji (radicals), then learning to recognize kanji made up of those radicals, and finally learning vocabulary that uses those kanji. Later items in the graph are unlocked by getting the previous items to a certain SRS stage. Additionally, the entire syllabus is divided into 60 discreet levels of (very) roughly the same number of items, the next level being unlocked by getting 90% of the current kanji to a certain SRS stage, which helps keep it all manageable. I tried using Anki before, but found the Wanikani "method" to work much better for me since I could constantly build on things I had learned previously. [1] https://wanikani.com/ https://wanikani.com/
trane_project 4y ago
Shameless plug to my own project: https://GitHub.com/trane-project/trane https://GitHub.com/trane-project/trane Dependency relationships between exercises are core to its design. I tried to use Anki but this limitation is pretty baked into Anki and made it unusable for cases where there's a clear order in which things must be learned (music in my case).
- nmca 4y ago
  This looks like a very cool project! The docs include this description of the core algorithm: > The space repetition algorithm in Trane is fairly simple and relies on computing a score for a given exercise based on previous trials rather than computing the optimal time at which the exercise needs to be presented again. This will most likely result in exercises being presented more often than they would in other spaced repetition software. Trane is not focused on memorization but on the repetition of individual skills until they are mastered, so I do not believe this to be a problem. Could you say more about it?
  - trane_project 4y ago
    Sure, here's the gist of how it works. - When Trane is asked for a batch of exercises, it performs a depth-first search of every branch in the dependency graph. It picks up exercises as it goes along, and stops exploring a branch when the dependencies of the last node are not met. - Each exercise is given a 1-5 score by the student and those scores are used to produce a combined score, rather than compute a date on which the exercise should be reviewed. Two reasons: - Trane is also meant to be used to learn skills, which do not fit the assumptions made by the SuperMemo algorithm. - I believe most of the gains come from regularly being reminded to perform an exercise, so even a simple scoring algorithm should get most of the gains. Eventually, I want to replace the existing scoring with something more complicated that also makes a distinction between memorization and skill exercises to get better results. But based on my testing, this works for the most part as is and the gains to be made from more complicated algorithms are marginal. - Once you get exercises from performing the search, you reduce that number into the final batch by grouping exercises based on difficulty and selecting a percentage from each bucket. This is done to make sure not too many very difficult or easy exercises are included in each batch. There's also some elements of randomness added to prevent the same exercises from appearing all the time. For example, the branches are explored in random order and the elements from each difficulty bucket are selected at random. But that is the gist of it.
majikaja 4y ago
In my personal system, I tag facts by sub-topic and the daily reps are ordered so that facts of the same sub-topic are shown consecutively to avoid context switching (eg. AAABBCCCCD not CABCCDBACA). In addition, there is logic so that after the mandatory daily reps are done, it will show cards that are close to the due-date (eg. within 0.25*LastInterval), again batched by sub-topic. This lets me efficiently get reviews out of the way on days when I have time to do so, without deviating too far from the basic SRS philosophy. As a result, for mature cards, I end up focusing on a small number of sub-topics on any particular day. I find that works a lot better. All of this is trivial to implement - I think it's best to create from scratch and specialize it for one's own domain instead of using a pre-existing solution. In my case the relationships are straightforward (sub-topics) but in others it might be more fuzzy. Programmatic fact-generation/deletion is also something I put a lot of effort into (again, domain-specific).
tester457 4y ago
Mochi does this. It uses markdown too and it's a pretty modern app. https://mochi.cards/ https://mochi.cards/ The dev is knubie on this forum
dandiep 4y ago
I have been working on a form of this for a language learning app [1]. The basic idea is that it gets you do free form output (discussion questions, word games, etc), gives you feedback on where you have errors and then sets up SR flashcards automatically (e.g. if you don’t know a word, conjugate it wrong, etc). It uses embeddings to make inferences across exercises and flashcards. This is handy for: - detecting that you already have a flashcard for that particular word/phrase/grammar issue - you know one sense of a word but not another - you correctly used a word that you have a flashcard for and that review can be put off It can look at your vocab and grammar skills of a particular area overall using Rasch scoring. Perhaps you answer almost all the questions about “cooking” correctly but none about “parts of the body” - well then the app can focus on the second area. There is also work to be done to build out better regressions given this data. If a word I am studying is way above my skill level, I will probably forget it faster. Or maybe I forgot a conjugation which is very common. It should be easier to remember because I’ll encounter it more. Whether or not my app succeeds, I think these types of approaches hold a ton of promise to increase learning efficiency. 1. https://squidgies.app https://squidgies.app
steve1977 4y ago
SuperMemo does (in a way): https://help.supermemo.org/wiki/Neural_creativity https://help.supermemo.org/wiki/Neural_creativity

by nmca 4y ago

I'm surprised that no spaced repetition systems seem to exploit relationships between facts/cards. One might imagine that capturing a graph of similar ideas would allow for better algorithms.

Scaevolus 4y ago

Anki supports linking cards when they're different facets of the same information (e.g., each language source/target for a given vocab word): https://faqs.ankiweb.net/linking-cards-together.html https://faqs.ankiweb.net/linking-cards-together.html More precision isn't that important-- at worst, you slightly over-review some information, but that's no different from using a flash card's information in normal life outside of a review period.

_Algernon_ 4y ago

The Anki FAQ goes into the reasoning a bit: https://faqs.ankiweb.net/linking-cards-together.html https://faqs.ankiweb.net/linking-cards-together.html >Some people want to extend this link between arbitrary cards. They want to be able to tell Anki "after showing me this card, show me that card", or "don’t show me that card until I know this card well enough". This might sound like a nice idea in theory, but in practice it is not practical. >For one, unlike the sibling card case above, you would have to define all the relations yourself. Entering new notes into Anki would become a complicated process, as you’d have to search through the rest of the deck and assign relationships between the old and new material. >Secondly, remember that Anki is using an algorithm to determine when the optimum time to show you material again is. Adding constraints to card display that cause cards to display earlier or later than they were supposed to will make the spaced repetition system less effective, leading to more work than necessary, or forgotten cards. >The most effective way to use Anki is to make each note you see independent from other notes. Instead of trying to join similar words together, you’ll be better off if you can determine the differences between them. Synonyms are rarely completely interchangeable - they tend to have nuances attached, and it’s not unusual for a sentence to become strange if one synonym is replaced with another.

nmca 4y ago
This seems to boil down to "it would be more complicated", mixed with "the trivial thing is a bad idea", both of which seem true. But I can imagine a more complicated scheme could be much better; done right.
- _Algernon_ 4y ago
  I agree. It makes sense for cards which link previously learned facts to only show up once the base facts have been learned, for example. But I'm not sure how to design a good UI for it. I can easily see it as resulting in too much friction so nobody uses it.
  - s_m_t 4y ago
    Doesn't just putting the cards in order of concepts essentially accomplish this? When I was doing Anki I adjusted my new card rate so my retention rate stayed above 90%. If you do it this way then your chances of getting stuck in a certain areas of the dependency graph (if you were to model it) is pretty low. If it does end up happening then do some extra study on those concepts outside of SRS. This way you don't have to model the actual dependency graph. You really shouldn't be doing all your study in SRS anyways because so called "flashcard blindness" (i.e. only being able to recall the concepts in the context of your SRS program) is a real thing. Furthermore, in my experience, what really made Anki effective for me was to write my new cards just in time. Meaning no pre-made decks except for something like the primitives of what you are studying (i.e. symbol names, alphabets, extreme base level concepts, etc). When you write your cards just in time you can really connect the context of what you are learning to the concept. From this perspective it really seems strange to build an entire ontology based on dependency graphs of a concept you haven't even learned yet!

type-r 4y ago

I have found this to be a huge missing piece of any SRSs I've come across. Personally, I have a pipeline that looks at what knowledge I've already learned and only then reveals successive information. For example, I'm learning numbers in Spanish. First, I ask myself to go from doce => 12; then, 12 => doce. Then, once I have a few numbers in my memory, I can start to combine those into diez + dos = ? (answer doce). The idea is that this last example requires recalling three previously-learned Spanish numbers to arrive at the answer. This is a trivial example but this underlying structure of successive layers of knowledge shows up everywhere and is horribly under-exploited by our current learning systems. Programmatically this can be done. Manually, this stuff takes way too much work.

DennisP 4y ago
How would you do it programmatically? Seems like you'd always need someone to give it the relationships.
- trane_project 4y ago
  Yes. That's required. I think they meant that the program uses those relationships to automatically advance the student to the next topic when it makes sense.
trane_project 4y ago
Check out my project: https://GitHub.com/trane-project/trane https://GitHub.com/trane-project/trane It does exactly this because I also thought this was a critical limitation of current systems. I have a feature coming out soon (already merged to master but I need to make a proper release) which should make it way easier to use with Markdown files acting as the front and back of flashcards that are automatically picked up when starting up the program.
- aporetics 4y ago
  If you want to look at prior art, take a look at org-drill

kashunstva 4y ago

> I'm surprised that no spaced repetition systems seem to exploit relationships between facts/cards. In fact, the Anki community largely endorses as an article of faith the complete insular atomicity of cards. I’ve always treated that orthodoxy with skepticism because I’m pretty sure that’s not how my brain works. Nor anyone else’s, probably.

aliceryhl 4y ago

An interesting example is chessable, which is a spaced repetition system for chess. It keeps track of each move in a sequence of moves separately, but every time it asks you to re-learn a specific move, you are asked to play out the whole sequence, instead of just being shown the position and asking you to play the next move.

optionalsquid 4y ago

Wanikani [1], an SRS for learning Japanese kanji and vocabulary, kinda works like that, though on a simpler level than what I think you envision. Basically their system involves first learning to recognize common parts of kanji (radicals), then learning to recognize kanji made up of those radicals, and finally learning vocabulary that uses those kanji. Later items in the graph are unlocked by getting the previous items to a certain SRS stage. Additionally, the entire syllabus is divided into 60 discreet levels of (very) roughly the same number of items, the next level being unlocked by getting 90% of the current kanji to a certain SRS stage, which helps keep it all manageable. I tried using Anki before, but found the Wanikani "method" to work much better for me since I could constantly build on things I had learned previously. [1] https://wanikani.com/ https://wanikani.com/

trane_project 4y ago

Shameless plug to my own project: https://GitHub.com/trane-project/trane https://GitHub.com/trane-project/trane Dependency relationships between exercises are core to its design. I tried to use Anki but this limitation is pretty baked into Anki and made it unusable for cases where there's a clear order in which things must be learned (music in my case).

nmca 4y ago
This looks like a very cool project! The docs include this description of the core algorithm: > The space repetition algorithm in Trane is fairly simple and relies on computing a score for a given exercise based on previous trials rather than computing the optimal time at which the exercise needs to be presented again. This will most likely result in exercises being presented more often than they would in other spaced repetition software. Trane is not focused on memorization but on the repetition of individual skills until they are mastered, so I do not believe this to be a problem. Could you say more about it?
- trane_project 4y ago
  Sure, here's the gist of how it works. - When Trane is asked for a batch of exercises, it performs a depth-first search of every branch in the dependency graph. It picks up exercises as it goes along, and stops exploring a branch when the dependencies of the last node are not met. - Each exercise is given a 1-5 score by the student and those scores are used to produce a combined score, rather than compute a date on which the exercise should be reviewed. Two reasons: - Trane is also meant to be used to learn skills, which do not fit the assumptions made by the SuperMemo algorithm. - I believe most of the gains come from regularly being reminded to perform an exercise, so even a simple scoring algorithm should get most of the gains. Eventually, I want to replace the existing scoring with something more complicated that also makes a distinction between memorization and skill exercises to get better results. But based on my testing, this works for the most part as is and the gains to be made from more complicated algorithms are marginal. - Once you get exercises from performing the search, you reduce that number into the final batch by grouping exercises based on difficulty and selecting a percentage from each bucket. This is done to make sure not too many very difficult or easy exercises are included in each batch. There's also some elements of randomness added to prevent the same exercises from appearing all the time. For example, the branches are explored in random order and the elements from each difficulty bucket are selected at random. But that is the gist of it.

majikaja 4y ago

In my personal system, I tag facts by sub-topic and the daily reps are ordered so that facts of the same sub-topic are shown consecutively to avoid context switching (eg. AAABBCCCCD not CABCCDBACA). In addition, there is logic so that after the mandatory daily reps are done, it will show cards that are close to the due-date (eg. within 0.25*LastInterval), again batched by sub-topic. This lets me efficiently get reviews out of the way on days when I have time to do so, without deviating too far from the basic SRS philosophy. As a result, for mature cards, I end up focusing on a small number of sub-topics on any particular day. I find that works a lot better. All of this is trivial to implement - I think it's best to create from scratch and specialize it for one's own domain instead of using a pre-existing solution. In my case the relationships are straightforward (sub-topics) but in others it might be more fuzzy. Programmatic fact-generation/deletion is also something I put a lot of effort into (again, domain-specific).

tester457 4y ago

Mochi does this. It uses markdown too and it's a pretty modern app. https://mochi.cards/ https://mochi.cards/ The dev is knubie on this forum

dandiep 4y ago

I have been working on a form of this for a language learning app [1]. The basic idea is that it gets you do free form output (discussion questions, word games, etc), gives you feedback on where you have errors and then sets up SR flashcards automatically (e.g. if you don’t know a word, conjugate it wrong, etc). It uses embeddings to make inferences across exercises and flashcards. This is handy for: - detecting that you already have a flashcard for that particular word/phrase/grammar issue - you know one sense of a word but not another - you correctly used a word that you have a flashcard for and that review can be put off It can look at your vocab and grammar skills of a particular area overall using Rasch scoring. Perhaps you answer almost all the questions about “cooking” correctly but none about “parts of the body” - well then the app can focus on the second area. There is also work to be done to build out better regressions given this data. If a word I am studying is way above my skill level, I will probably forget it faster. Or maybe I forgot a conjugation which is very common. It should be easier to remember because I’ll encounter it more. Whether or not my app succeeds, I think these types of approaches hold a ton of promise to increase learning efficiency. 1. https://squidgies.app https://squidgies.app

steve1977 4y ago

SuperMemo does (in a way): https://help.supermemo.org/wiki/Neural_creativity https://help.supermemo.org/wiki/Neural_creativity