3 ms·
Most definitely, what I meant though is how do we know mental attention is not essentially a big softmax (over some context, e.g. short or long term memory)? T
by smallnamespace 6y ago
Most definitely, what I meant though is how do we know mental attention is not essentially a big softmax (over some context, e.g. short or long term memory)?
The eye physically focusing on one thing at time seems like a special case (and this fact isn't even true of all animals, e.g. many prey species), and not a part of the brain's attention mechanism.
- Straw 6y agoOh, I don't know, it very well might! However, the brain's physical structure doesn't look conducive to 4096x4096 matmuls!