3 ms·
LLMs work by taking an input text and repeatedly predicting the next word. In order to predict the next word in a sentence, you basically have to think like a h
by olalonde 3y ago
LLMs work by taking an input text and repeatedly predicting the next word. In order to predict the next word in a sentence, you basically have to think like a human would, which includes doing things like logical deduction. LLMs are expected to learn those sort of functions.
As a side note, GPT-4 (not sure about other models) is capable of doing logical deductions when prompted.
- Jensson 3y ago> As a side note, GPT-4 (not sure about other models) is capable of doing logical deductions when prompted. But not capable of doing logical deduction of the data it is trained on, just the data you give it. It is very stupid about all the data in its training corpus, since it is encoded like a grammar and not a knowledge base. This wouldn't be a problem if not for there being a million times more data in the trained set than what you can give it. But this mean that we can't really train the AI to be smart about a wide range of things, since the training corpus is stupid and the live data is very limited. (And it isn't even that smart about the live data, given how expensive it is to compute for that little amount of knowledge)