3 ms·
You’re talking specifically about perceptrons and feed forward neural networks. LLMs didn’t exist in then. Attention only came out in 2017…
by dartos 2y ago
You’re talking specifically about perceptrons and feed forward neural networks.
LLMs didn’t exist in then. Attention only came out in 2017…
- tossandthrow 2y agoYes? Are you saying that attention is less expressive?
- dartos 2y agoI’m saying that LLMs (models trained on language specifically) are not automatically capable of the same generic function solving. The network itself can be trained to solve most functions (or all, I forget precisely if NNs can solve all functions) But the language model is not necessarily capable of solving all functions, because it was already trained on language.