Yea makes me wonder what data could be used to train this theoretical symbolic model that precedes the language model? I’m thinking training on movies/tv has t

Yea makes me wonder what data could be used to train this theoretical symbolic model that precedes the language model? I’m thinking training on movies/tv has the physical element that pure text is missing.