Is the behavior that the attention + FF displacements tend point in the same direction known? I am kind of surprised they are even in the same latent space acr

Is the behavior that the attention + FF displacements tend point in the same direction known? I am kind of surprised they are even in the same latent space across layers. The FF network could be doing arbitrary rotations, right? I suspect I misunderstand what is going on.