That may be true on a mathematical level, but that's also the answer to just about any neural net question... it's all just a a weighted sum. My understanding o

That may be true on a mathematical level, but that's also the answer to just about any neural net question... it's all just a a weighted sum. My understanding of "attention" on a higher level is the ability to concentrate more neurons on "important" areas of an image than less important ones. An imperfect analogy is how the human visual system has better resolution at your eye-line's center than at it's edges. In this analogy, your brain should not waste effort processing image details in your peripheral vision.