4 ms·
Older AI models show signs of cognitive decline, study shows
- snypher 2y agoSo did the previous generation models test higher when released, and now don't test as high? I may have misread the article but it seemed like the older models had a worse score all along, which can't really be called 'decline'?
- reportgunner 2y agoThey are getting worn out from old age just like humans do.
- xg15 2y ago...by which they mean "earlier-generation models have lower scores in a test measuring cognitive function than newer models". The entire study was an exercise in academic clickbait, something that even other scientists complained about: > Other scientists have been left unconvinced about the study and its findings, going so far as to critisize the methods and the framing — in which the study's authors are accused of anthropomorphizing AI by projecting human conditions onto it. There is also criticism of the use of MoCA. This was a test examined purely for use in humans, it is suggested, and would not render meaningful results if applied to other forms of intelligence. The study authors defend with the classic "It's just a joke, bro" card: > Responding to the discussion, lead author of the study Roy Dayan, a doctor of medicine at the Hadassah Medica Center in Jerusalem, commented that many of the responses to the study have taken the framing too literally. Because the study was published in the Christmas edition of the BMJ, they used humor to present the findings of the study — including the pun "Age Against the Machine" — but intended the study to be considered seriously.
- casey2 2y agoThe joke is that people tend to anthropomorphize LLM based chatbots so saying older models get age related diseases like humans is funny. The findings are the inverse, newer models do better on dementia tests than older models. Whether a test was "intended" for humans or not is irrelevant in bench-marking AI. An AGI should reasonably be expected to answer such question.
- jcz_nz 2y agoThis “study” is… total BS. I’d be super suspicious of anything else Roy Dayan ever puts his name on. Clickbait is the most generous interpretation.