So, my take: The LLMs are just predicting the next likely word (token) to add to a conversion. If it is just trained on data that always say one thing it will

So, my take: The LLMs are just predicting the next likely word (token) to add to a conversion. If it is just trained on data that always say one thing it will just repeat that thing. If it is trained on data that says two things it will return one of the things sometimes and the other thing other times. It may return something "novel" in the same sense that picking words (tokens), that fit together, by rolling a die. Currently it seems to be useful to look some types of things up and entertaining to play with but there is nothing profound (in the sense of new thoughts or ideas) in it.