3 ms·
> but go ahead and try to understand what the real-world entities I renamed were. Obviously not. Insufficient data. Try that substitution exercise again with t
by xcv123 3y ago
> but go ahead and try to understand what the real-world entities I renamed were.
Obviously not. Insufficient data. Try that substitution exercise again with terabytes of data (Common Crawl dataset) then feed it into a neural network and see if it can find any patterns.
- deleted 3y ago[deleted]
- YeGoblynQueenne 3y agoThis is not a matter of data quantity, but data quality. Text data doesn't have the information needed to learn a world model from it and so it doesn't matter how big a model you train, it will never learn the right representation (i.e. the right patterns).
- xcv123 3y ago> Text data doesn't have the information needed to learn a world model from it As I mentioned in another reply you have a philosophical disagreement and are missing the point. It provides "a" model of the world. Not necessarily the "one true model" that is your own personal life experience of the world since birth. There is enough information to build "a" world model in all the text ever written by humans about the world. That is obvious. So your argument is completely subjective and philosophical, not technical. GPT now includes imagery in its training set BTW.
- YeGoblynQueenne 3y agoI don't do philosophical arguments. But it's obvious this conversation not productive so there's no reason for me to insist.
- xcv123 3y agoBut you are essentially making a philosophical argument about what defines a "world model". It is a disagreement about a subjective interpretation of the term.
- YeGoblynQueenne 3y agoBe careful about jumping to conclusions just to convince yourself you're right and I'm wrong. I do not assume what you think I assume. But it's obvious you're not looking for a curious conversation and reaching any kind of common understanding.
- xcv123 3y agoWe are talking about the definition of "world model". If you disagree with it then your argument is not technical, it is philosophical in nature. This is like arguing with a religious fundamentalist.
- xcv123 3y agoHere's a definition I found somewhere. Do you agree with it? A "world model" refers to the internal representation of the external environment that an agent (e.g., a human, animal, or machine) maintains and uses to navigate and make decisions within that environment. In other words, it's a mental or computational model of the world as perceived and understood by the agent. Obviously LLMs have a world model. It's not your favourite model or preferred model or the best model in town, but it is a model of the world whether you like it or not, whether you agree with it or not.