The intuition is that the relative frequency at which past tokens get attention from future tokens is a good proxy for their relative importance. The model the

The intuition is that the relative frequency at which past tokens get attention from future tokens is a good proxy for their relative importance. The model the authors use, in fact, maps attention scores to features in the frequency domain.