4 ms·

This looks really good. I wish I had had something like this many years ago when I was studying languages. Somebody has already suggested adding spaced repetit

This looks really good. I wish I had had something like this many years ago when I was studying languages. Somebody has already suggested adding spaced repetition and audio, which I agree with completely. One more suggestion: In addition to having the LLM give you the meaning and example for the context in which you originally saw the word, also ask it to provide the word’s other main meanings and examples of it being used in those senses. You might encounter a word first in a slang or technical sense; while it’s useful to learn that meaning, it’s also important to learn other, more common meanings. Below are some examples of words you might encounter first in technical contexts but would also be worth knowing in their more general meanings. (Examples suggested and defined by ChatGPT o1.) canonical Religious/General: Relating to a canon (e.g., church law) or a recognized body of works. Math/Computing: Conforming to a standard or simplest form (e.g., “canonical form” of an equation). resolution General: A firm decision or determination (often heard in “New Year’s resolution”). Tech/Imaging: The detail an image holds, typically measured in pixels, dots per inch (DPI), etc. protocol Diplomatic/General: The official procedure or set of rules governing state or ceremonial events. Computing: A set of conventions and rules for transmitting data between electronic devices. flux General: Continuous movement or change, often implying instability. Physics/Engineering: The amount of some quantity (e.g., heat, magnetism) passing through a given area over time.

PeterSmit 2y ago
I’ve had this same idea, and it doesn’t work. Or at least: it works quote well, but the problem is that you get hallucinations. And it can be incredibly discouraging to find out the flashcards you’ve been cramming are completlh wrong.
- tkgally 2y ago
  What models have you been using for that? While I haven’t tried automating the production of vocabulary lists through an API, within the last few weeks I have had the chat versions of ChatGPT 4o, Claude Sonnet 3.5, and one of the latest Gemini models produce annotated vocabulary lists based on literary texts in English, Russian, and Latin. I didn’t spot any hallucinations. I was asking only for the meanings of the words and phrases, though. I didn’t ask for things like pronunciations, grammatical categories, etc. In the past, when I’ve tried to get that kind of granular information from LLMs, there were indeed errors, presumably because of tokenization issues. A few days ago, I ran some similar tests with Japanese, asking for readings of kanji and jukugo in an extended text. All of the models I had tried before for such tasks had screwed up. This time, however, ChatGPT o1 scored 100%. It also was able to analyze sentence grammar accurately, unlike the other models I tried. I was impressed. At current API prices, though, o1 might be a bit too expensive for such a task.
  - arbayi 2y ago
    I wonder if there are any benchmarks specifically designed to evaluate LLMs' performance in language learning tasks
    - tkgally 2y ago
      I haven’t heard of any. It would great if there were....
- 3D30497420 2y ago
  I've had this same problem using ChatGPT and German. Even for basic German hallucinations can be unexpected and problematic. (I don't recall the model, but it was a recent one.) In one instance, I was having it correct akkusativ/dativ/nominativ sentences and it would say the sentence is in one case when I knew it was in another case. I'd ask ChatGPT if it was sure, and then it would change its answer. If pressed further, it would again change its answer. I was originally quite excited about using an LLM for my language practice, but now I'm pretty cautious with it. It is also why I'm very skeptical of AI-based language learning apps, especially if the creator is not a native speaker.
  - arbayi 2y ago
    Would agentic workflows come in handy in these cases? I mean having a controller agent after the sentence is created, where this agent would be able to search the web or have access to a database? or personal notes and ensure everything is correct.
    - 3D30497420 2y ago
      Maybe? I suppose it depends on the quality of the controller agent, which then comes back to the quality of the original LLM.
- learning-tr 2y ago
  I had this problem initially but found that if you use these then hallucinations mostly go away. 1. Role based "agents" with a router and logs (for auditing reasoning and decision making). 2. Cross validation and redundancy with the translation "agent" using a 2nd language (that is not English) that you are also native in to check if the translation carries the same "meaning" (sentiment) and cultural significance (Turkish is especially rich in symbolism and cultural memes). YMMV: I am a car salesman irl and have no formal training.
arbayi 2y ago
Thank you! I think that would be a good feature. I use the app, and it works fine when I add words as I encounter them, but when I later revisit the list, I sometimes wish I could see the sentences where the word were used (originally) to better understand it. Based on your suggestion, having a 'show examples' button/link below the meaning would be a cool feature to add.
- 5eyyah 2y ago
  `show examples` is good idea

PeterSmit 2y ago

I’ve had this same idea, and it doesn’t work. Or at least: it works quote well, but the problem is that you get hallucinations. And it can be incredibly discouraging to find out the flashcards you’ve been cramming are completlh wrong.

tkgally 2y ago
What models have you been using for that? While I haven’t tried automating the production of vocabulary lists through an API, within the last few weeks I have had the chat versions of ChatGPT 4o, Claude Sonnet 3.5, and one of the latest Gemini models produce annotated vocabulary lists based on literary texts in English, Russian, and Latin. I didn’t spot any hallucinations. I was asking only for the meanings of the words and phrases, though. I didn’t ask for things like pronunciations, grammatical categories, etc. In the past, when I’ve tried to get that kind of granular information from LLMs, there were indeed errors, presumably because of tokenization issues. A few days ago, I ran some similar tests with Japanese, asking for readings of kanji and jukugo in an extended text. All of the models I had tried before for such tasks had screwed up. This time, however, ChatGPT o1 scored 100%. It also was able to analyze sentence grammar accurately, unlike the other models I tried. I was impressed. At current API prices, though, o1 might be a bit too expensive for such a task.
- arbayi 2y ago
  I wonder if there are any benchmarks specifically designed to evaluate LLMs' performance in language learning tasks
  - tkgally 2y ago
    I haven’t heard of any. It would great if there were....
3D30497420 2y ago
I've had this same problem using ChatGPT and German. Even for basic German hallucinations can be unexpected and problematic. (I don't recall the model, but it was a recent one.) In one instance, I was having it correct akkusativ/dativ/nominativ sentences and it would say the sentence is in one case when I knew it was in another case. I'd ask ChatGPT if it was sure, and then it would change its answer. If pressed further, it would again change its answer. I was originally quite excited about using an LLM for my language practice, but now I'm pretty cautious with it. It is also why I'm very skeptical of AI-based language learning apps, especially if the creator is not a native speaker.
- arbayi 2y ago
  Would agentic workflows come in handy in these cases? I mean having a controller agent after the sentence is created, where this agent would be able to search the web or have access to a database? or personal notes and ensure everything is correct.
  - 3D30497420 2y ago
    Maybe? I suppose it depends on the quality of the controller agent, which then comes back to the quality of the original LLM.
learning-tr 2y ago
I had this problem initially but found that if you use these then hallucinations mostly go away. 1. Role based "agents" with a router and logs (for auditing reasoning and decision making). 2. Cross validation and redundancy with the translation "agent" using a 2nd language (that is not English) that you are also native in to check if the translation carries the same "meaning" (sentiment) and cultural significance (Turkish is especially rich in symbolism and cultural memes). YMMV: I am a car salesman irl and have no formal training.

arbayi 2y ago

Thank you! I think that would be a good feature. I use the app, and it works fine when I add words as I encounter them, but when I later revisit the list, I sometimes wish I could see the sentences where the word were used (originally) to better understand it. Based on your suggestion, having a 'show examples' button/link below the meaning would be a cool feature to add.

5eyyah 2y ago
`show examples` is good idea