Good work! This is very interesting. Here's a related work that construct low-rank approximation for attention: https://arxiv.org/abs/2505.12942 https://arxiv.o

Good work! This is very interesting. Here's a related work that construct low-rank approximation for attention: https://arxiv.org/abs/2505.12942 https://arxiv.org/abs/2505.12942. Maybe the idea of Query calibration matrix Rxx is of interest to the author!