2 ms·

I mean, the QKV part of transformers is like an "ASIC" ... well, for an (approximate) lookup table. (also important to note that NNs/LLMs operate on... abstrac

I mean, the QKV part of transformers is like an "ASIC" ... well, for an (approximate) lookup table. (also important to note that NNs/LLMs operate on... abstract vectors, not "language" -- not relevant as a response to your post though).

exe34 2y ago
actually I think you are on to something - abstract vectors are the tokens of thought - mentalese if you've read any Dennett.