4 ms·
My biggest concern with LLM’s though is that it’s actually the opposite of the library of Babel, it’s a remarkably closed system, tuned by humans and using a (r
by techpineapple 1y ago
My biggest concern with LLM’s though is that it’s actually the opposite of the library of Babel, it’s a remarkably closed system, tuned by humans and using a (relative to the library of babel) small search space.
There’s meaning in what LLM’s say because actually the way it remixes its source material is much less sophisticated than we think, and thus its meaning is a direct representation of the human meaning it’s trained on.
Would you argue against the notion that there is rich meaning captured by the libraries of the world the LLM’s are trained on?
- tocs3 1y agoSo, my take: The LLMs are just predicting the next likely word (token) to add to a conversion. If it is just trained on data that always say one thing it will just repeat that thing. If it is trained on data that says two things it will return one of the things sometimes and the other thing other times. It may return something "novel" in the same sense that picking words (tokens), that fit together, by rolling a die. Currently it seems to be useful to look some types of things up and entertaining to play with but there is nothing profound (in the sense of new thoughts or ideas) in it.