More accessible references: https://mcbal.github.io/post/an-energy-based-perspective-on-attention-mechanisms-in-transformers/ https://mcbal.github.io/post/an-e

More accessible references: https://mcbal.github.io/post/an-energy-based-perspective-on-attention-mechanisms-in-transformers/ https://mcbal.github.io/post/an-energy-based-perspective-on-... (Modern continuous Hopfield networks section) https://arxiv.org/abs/2008.02217 https://arxiv.org/abs/2008.02217 Note that the connection to Hebbian learning hinges on the softmax function, in particular its exponential!