I argue that the attention mechanism is just Hopf algebra convolution. The QK mechanism selects possible candidates and the OV mechanism picks one of these poss

I argue that the attention mechanism is just Hopf algebra convolution. The QK mechanism selects possible candidates and the OV mechanism picks one of these possible candidates. I have recently written a paper on this https://arxiv.org/abs/2302.01834 https://arxiv.org/abs/2302.01834 I also have a discord channel https://discord.cofunctional.ai https://discord.cofunctional.ai. Ironically, it's the same mechanism as what renormalization in QFT does. I'm getting increasingly convinced that it's also how the brain works.