It’s debatable to what degree ”attention” in LLMs relates to ”attention” in psychology. See Cosma Shalizi’s note on this http://bactra.org/notebooks/nn-attenti

It’s debatable to what degree ”attention” in LLMs relates to ”attention” in psychology. See Cosma Shalizi’s note on this http://bactra.org/notebooks/nn-attention-and-transformers.html http://bactra.org/notebooks/nn-attention-and-transformers.ht...