The video hints at why achieving artificial general intelligence (AGI) is so difficult. As an example, it points out that AlphaGo was able to surpass human Go p

The video hints at why achieving artificial general intelligence (AGI) is so difficult. As an example, it points out that AlphaGo was able to surpass human Go players by playing against itself, which worked because winning Go games provides a clear reward function: winning the game. Similarly, the rumored Q* breakthrough involves solving simple math problems, another closed domain with a well-defined reward function. This has likely gotten people overly excited about the prospects for rapid advances in domains like open-ended text generation. Defining, much less evaluating, a reward function that maps to the "quality" of generated text in general for use during the inference phase is hard to imagine. So while grafting planning onto a language model works for constrained use cases like math, it is hard to see how it could work for generating high quality, coherent text in general.