3 ms·
Attention (see Google papers) seems like a pretty promising way of dealing with countable states. In quantum mechanics, we change the question we are asking. In
by jayajay 10y ago
Attention (see Google papers) seems like a pretty promising way of dealing with countable states. In quantum mechanics, we change the question we are asking. Instead of "which countable eigenstate are we in?", we ask "how much of our current state is in the Nth eigenstate?". We just expand our current state over the basis of interest. The coefficient of the state of interest is a continuous variable. This is what attention does for neural networks.