Gary Marcus has been incredibly vocal against LLM for quite a while now. The problem is that the guy does not understand that modern LLM are no longer just clas

Gary Marcus has been incredibly vocal against LLM for quite a while now. The problem is that the guy does not understand that modern LLM are no longer just classifiers, as the article seems to imply. The underlying transformer architecture has completly transformed the field. For instance, training an AI to recognize cats has nothing to do with training a LLM, texts are fed to the machine without any labels. Attention is where these models take their strenght. It is a way to learn how to predict the next token by learning which elements are connected between the input and the output so far. These architectures model a sequence, which is very different than simply finding the next token with a basic probabilities as ancient approaches did.