4 ms·
I get what you're saying. I wonder if you are fully following through the implications. If GPT were generally intelligent, we shouldn't need to devote a specia
by civilized 4y ago
I get what you're saying. I wonder if you are fully following through the implications.
If GPT were generally intelligent, we shouldn't need to devote a special research project to teaching it math. We could just throw a math textbook at it, explanations and worked examples, and it would figure it out from there. Almost certainly its training data contains a great deal of such material already. That this doesn't work suggests its mental architecture is insufficient to grasp what it's been told. (Note that people are not advantaged with any specialized symbolic representation of numbers, like the integer data type a computer has. We manipulate numerical symbols as text, same as the AI.)
It's all well and good that it can improve when a special effort is made, but it sounds like even with that special effort, it still doesn't show the level of competence one would expect from a human-level intelligence with access to virtually infinite, untiring silicon computational resources.
GPT has access to abundant materials to learn the laws of arithmetic from, and it can tell you what they are (because it memorizes everything) but it isn't really understanding what it's learned. That points to a shortcoming of the architecture that won't be solved by merely throwing more data at it.
- famouswaffles 4y agoAre humans not generally intelligent ?. Since when has been the answer to "not good at math" been chuck a textbook at it ? You would have limited success doing this with people. Do people not explain things they don't fully understand? Understanding is not binary. This is kind of problem I keep seeing. Expectations and post shifting have grown so much that a significant chunk of the human population wouldn't even pass so called General Intelligence requirements.
- civilized 4y agoI learned arithmetic laws from a textbook and practice problems. I didn't need a teacher gesticulating or any fancy multimodal stuff. It's symbolic manipulation. What is the machine missing that I was given? There's no post shifting. The research community has been setting itself realistically attainable benchmarks. Now that the research community has made a lot of progress against its benchmarks, we have hype, claims of general intelligence. Which attracts people like me, who compare the hype to actual performance. And as I said elsewhere, the performance of GPT on the questions I posed is only comparable to a human with a severe traumatic brain injury.
- famouswaffles 4y agoI don't care what you did. All due respect, you are one person. I care about what can observed by people in general. Dunno what to tell you other than textbook and practice problems is far from the solution you think it is for a big chunk of the population.
- civilized 4y agoWe have a machine that is dedicated to symbolic manipulation (prediction of new symbols based on past ones). It seems reasonable to compare it to a person that is good at symbolic manipulation. Why would we compare the machine to average people who mostly don't think that way? Why would training a neural network architecture that lives and breathes symbols somehow yield an entity with intelligence like that of an average human? Most humans learn primarily from completely different sources.