3 ms·
Imagine what the alternative would imply. AI would be solved, and thus, intelligence itself. Predicting tokens is not actually true intelligence, and that’s not
by hans1729 4y ago
Imagine what the alternative would imply. AI would be solved, and thus, intelligence itself. Predicting tokens is not actually true intelligence, and that’s not really the point of these models. This is a step on the letter, not the rooftop. It looks a lot like we’ll get there though, if you compare the state of the art to ANYTHING labeled AI five years ago. Thats the exciting part.
[edit] to emphasize: predicting tokens is a very interesting mechanic, but in a design of intelligent software, it would be no more than that: the mechanic of one or more of its components/modules/subsystems. The real deal is to figure out what those components are. Once you have that part done, you can implement it in a language of your choice, be it token prediction, asm or powerpoint :-)
- CRG 4y agoIt's also smaller than GPT-2 (1.2B vs 1.6B) and trained with a lot less language data (6% of the training mix).