I basically solved the same problem about a year ago. I was reading Gormenghast trilogy and for a considerable part of the book I stumbled upon rare English wor

I basically solved the same problem about a year ago. I was reading Gormenghast trilogy and for a considerable part of the book I stumbled upon rare English words of unknown or hazy meaning for me. I decided I need a tool to prepare Anki decks with those words, bu as I'm too lazy to manually log them, I thought I just use the same algorithm, take the top most infrequently occurring words and generate a CSV with definitions from GCIDE[0] to load into Anki. I used Rust and thought it's the textbook case for a trie, but what I found is that crates.io had quite a few tries but not all of them working or ergonomic enough. Only one of the libraries I tried (huh) gave me a slight advantage over a HashMap, although statistically insignificant, which is rather disappointing, considering the program outputted sorted frequency groups. I'm pretty pleased with the results. I considered adding stemming[1], silly me. Too hard of a problem to bother. 0. https://gcide.gnu.org.ua/ https://gcide.gnu.org.ua/ 1. https://en.wikipedia.org/wiki/Stemming https://en.wikipedia.org/wiki/Stemming