3 ms·
LLMs develop a model of the world, experienced through human-generated text. The same as I've never been to China, but I have an internal model of China due to
by gaganyaan 3y ago
LLMs develop a model of the world, experienced through human-generated text. The same as I've never been to China, but I have an internal model of China due to experiencing it through human-generated text.
- danmaz74 3y agoThere is a huge difference between your model and an LLM model in your example. Your model "knows" that China has rivers (of which you have direct experience), coastlines (same), trees, mountains etc. etc. An LLM has no tangible experience of any of the concepts included in the text.
- gaganyaan 3y agoBoth my internal model and an LLM are built from sensory data. My model knows what a river is from having experienced it through sensors that an LLM doesn't currently have, but don't expect that limitation to last. Also, given how many words humans have written describing every part of the experience, LLMs can generate a pretty good understanding of it as-is.