> So the model is bounded by the context window, which is currently mostly at 512 tokens, and is thus, bounded in how much it can understand. This is begging t

> So the model is bounded by the context window, which is currently mostly at 512 tokens, and is thus, bounded in how much it can understand. This is begging the question as to whether the model "understands" anything at all. And once you adopt a definition of "understanding" that isn't equivalent to "got a high score on some pointless academic challenge" the answer is a resounding "no." The whole enterprise of AGI hype is based on this equivocation of words like "understanding" and "intelligence." We use a very restricted definition in proving that the tech is smart, and then switch out our restricted definition for the colloquial one when the audience isn't looking. > This will unlock better conversational abilities Shouldn't be hard given that as it stands there are none, except for creating a human-sounding slurry that is devoid of real content. > but also, better ways to understand how different pieces of textual information relate Is this a real need? What problem does this solve that forums + wikipedia + arxiv + google + a literate human hasn't already?