4 ms·
Your ascribing way too much intelligence to a very simplistic text prediction algorithm. Neither GPT-3 nor GPT-4 (assuming it will just be a bigger GPT-3) have
by tsimionescu 6y ago
Your ascribing way too much intelligence to a very simplistic text prediction algorithm. Neither GPT-3 nor GPT-4 (assuming it will just be a bigger GPT-3) have any concept of identity that could influence their predictions in the way you seem to assume.
If GPT-4 would be trained on text produced by GPT-3 and on text that analyzes that, what GPT-4 would learn is how to sound like GPT-3 and like detractors or admirers of GPT-3.
While impressive achievements, GPT-2/3/4/X are just models that predict what the most likely next word is based on a huge corpus of text. There is no model of the world in there, no objects, no identities. Just probabilities of glyphs in certain orders, without any meaning attached.
- haolez 6y agoI mostly agree with you, but I wonder what's the distance between what you described and our actual intelligence.
- tsimionescu 6y agoI don't think there is any kind of similarity between a text prediction model and human (or animal) intelligence. At most we maybe have a subsystem that could work somewhat similarly. It's very clear though that human intelligence has a model of the world being made up of agents which interact with objects. Words and sentences refer to entities in this model in our head, and our use of language essentially consists of describing instances of this model, with various purposes. Text prediction works exclusively on text, no model of the world, and it has a single purpose: to produce the text that would be most likely to follow the prompt based on its training. This isn't just a theoretical difference. GPT-3 couldn't be used to, for example, be presented with a situation and asked to respond to it in such a way that it achieves some goal.