I fed the two comments verbatim into Grok: > Curriculum learning: Training that begins with easy examples, gradually increasing difficulty. > RL (Reinforcemen

I fed the two comments verbatim into Grok: > Curriculum learning: Training that begins with easy examples, gradually increasing difficulty. > RL (Reinforcement Learning): Learning via trial-and-error with rewards, like training a robot or model to optimize actions. > ~7y ago: ~7 years ago (circa 2018). > OpenAI Gym days: Refers to using OpenAI Gym, a toolkit for RL, popular in robotics/AI research ~2016-2018. > LLMs and robotics: Large Language Models (LLMs) now leverage RL techniques from robotics for better performance. I think the last one is a semi-hallucinatory stretch. LLMs are large language models, ie. ChatGPT, Sonnet, Grok, R1. Robotics are ... robotics. Building robots. The actual answer to what the comment is saying is that until maybe a year back, we trained language models - still with RL, but with RL on token error, which isn't "real" RL because it executes tasks "by coincidence". That is, it happens to be that when you train a model to predict text, it also gains the ability to do tasks in the bargain, because the text contains agents that do tasks. A year or so ago, we started training models by having them do a task, judging if the task was successful or failed, and then performing RL on task outcome rather than token prediction. This is a return to "classic RL", but we had to pass through the "token RL regime" first so that the model could make progress on realistic tasks at all. It also means that LLMs can now increasingly be employed in robotics, where task RL training rules, as there is no massive preexisting robotics movements dataset like there is for text. (Also, NLP is Natural Language Processing, ie. what LLMs do.)