SGD is a method of learning from examples (mapping X -> y). RL is a method of search (maximizing reward signal in a potentially open-ended environment, based on

SGD is a method of learning from examples (mapping X -> y). RL is a method of search (maximizing reward signal in a potentially open-ended environment, based on feedback on the actions that a model takes). Go was solved by combining learning and search (e.g. narrowing the search space with learned, educated guesses). Pretrained dense GPTs don't search in any conventional way. However, when these GPTs are fine-tuned using RL, an aspect of search is reintroduced. One the most interesting directions labs are pursuing to improve on today's best LLMs is to fine-tune them with search (RL) after a learning-based (next-token prediction) pre-training.