3 ms·

Perhaps they should start with "undergrad level abilities." Pretty sure even undergrads can write a paper without made up references.

• OutOfHere 2y ago
  They claim that it jumped from toddler to high-school, so jumping to PhD does not break the pattern. One can argue that undergrad degrees are just a marketing gimmick and were not really meant to exist in a true academic sense.
  • tensor 2y ago
    Unless you look at the actual objective functions for training: primarily, produce human like text, and second, produce answers that "feel good" to a human via reinforcement learning. Neither function optimizes for an understanding of what truth means, generating confidences, reasoning, let alone higher level objectives like "doing PhD level research". Given that none of these things are changing I wouldn't expect any major difference in its ability to provide truthful reasoned responses. It make continue to pick up some of these abilities as a coincidental byproduct of learning to generate text, but it will still be a glorified text generator without these changes. That said maybe they will have some new training objective function, we'll see. Either way these systems are a far cry from human-level when it comes to having actual goals beyond being a toy text generator.
  • dragonwriter 2y ago
    > It jumped from toddler to high-school, No, it didn't. The idea that it did is based on manifestly false assumption that correlations between specific tests and general mental ability that apply on-average to humans apply to LLMs. We can easily observe this is false, but flim-flam salesmen expect us to ignore it in favor of exaggerated hype that obscures rather than assists any meaningful discussion of the utility and capacity of LLMs.