3 ms·
It isn’t clear though. Neural networks were inspired by the brain, but transformers? It is totally plausible but do we really think just in words?
by seanmcdirmid 11mo ago
It isn’t clear though. Neural networks were inspired by the brain, but transformers? It is totally plausible but do we really think just in words?
- SAI_Peregrinus 11mo ago> It is totally plausible but do we really think just in words? I find that proposition totally implausible. Some people certainly report only thinking in words & having a continuous inner monologue, but I'm not one of them. I think, then I describe my thoughts in words if I'm speaking or writing or thinking about speaking or writing.
- coldtea 11mo ago>Neural networks were inspired by the brain, but transformers? It is totally plausible but do we really think just in words? LLMs might be trained via words, but as a backend transformers are not just for words. They're for high dimensional structured sequences. To make an analogy, transformers are not working on: Vector<Word> but Vector<ContextualizedEmbedding> where words just happens to be a handy training set we use. And, we too, might not think in words, but I bet that we do think using multi-dimensional sequences/vectors.