3 ms·
What does it mean to be “more than statistics”? Is there anything that cannot be described as a statistic?
by canjobear 3y ago
What does it mean to be “more than statistics”? Is there anything that cannot be described as a statistic?
- Karellen 3y agoI think the part of the article that the parent comment is disagreeing with is: > “It is certainly much more than a stochastic parrot, and it certainly builds some representation of the world—although I do not think that it is quite like how humans build an internal world model,” says Yoshua Bengio, an AI researcher at the University of Montreal. > At a conference at New York University in March, philosopher Raphaël Millière of Columbia University [...] went a step further and showed that GPT can execute code, too, however. The philosopher typed in a program to calculate the 83rd number in the Fibonacci sequence. “It’s multistep reasoning of a very high degree,” he says. And the bot nailed it. When Millière asked directly for the 83rd Fibonacci number, however, GPT got it wrong: this suggests the system wasn’t just parroting the Internet. Rather it was performing its own calculations to reach the correct answer. > This impromptu ability demonstrates that LLMs develop an internal complexity that goes well beyond a shallow statistical analysis. Researchers are finding that these systems seem to achieve genuine understanding of what they have learned.
- circuit10 3y agoBut that isn’t claiming that it isn’t based on statistics, it’s just saying that the analysis being done isn’t shallow, which is clearly true
- sharemywin 3y agoI think to some extent it's using the attention to pay attention to which "statistics" to pay attention to.