3 ms·
It’s not that interesting. He argued against the nonexistent straw man that no system that can predict the next word could be doing reasoning, which is self-evi
by wrs 3y ago
It’s not that interesting. He argued against the nonexistent straw man that no system that can predict the next word could be doing reasoning, which is self-evidently false, as you are perfectly able to predict words yourself.
The actual question is whether focusing on improving word prediction is the way to achieve reasoning.
If a system could reliably predict the last word of a detective novel, it would arguably be doing reasoning, sure. If it could reliably predict the last word of “the simple one word solution to global warming is”, it would be really amazing. That isn’t an argument that such a system can exist or that you know how to get there.
- zarzavat 3y agoGPT4 isn’t just doing word prediction though. It has image input in GPT4V, and image output via Dalle. OpenAI are well on the way down the path of multimodality. It would be fair enough to say that GPT3 doesn’t know what a dog is, it only knows about the word “dog”, but GPT4 has a conception of a dog that is both visual and literary, and as they add more modes the depth of understanding will increase.
- astrange 3y agoDalle is a whole separate model that isn't multimodal; GPT4 just write prompts. If you play with it you can actually see dalle usually doesn't listen to most of what it writes. GPT4V seems to be truly multimodal but is also a different model from GPT4; the writing style is different and it's maybe less intelligent.
- zeusk 3y agohttps://www.adept.ai/blog/fuyu-8b https://www.adept.ai/blog/fuyu-8b Doesn't seem like multimodal inference is as complex as you're making it out to be. It all hinges on what you claim "understanding" to be; are trained weights understanding?
- deleted 3y ago[deleted]
- belter 3y agoI think you are replying to an argument he was not making. :-)
- wrs 3y agoReally? “The better a neural network can predict the next word in text, the better it understands it. […] People say that deep learning won’t lead to reasoning, but in order to predict that next word, […] who was the murderer, that requires some reasoning, a fair amount of reasoning […]” Predicting the next word in that situation might require understanding. It doesn’t follow that trying to make it predict the next word better will result in understanding.