Don’t let the mathematical terms fool you, these are just fan theories. For a real investigation you need debug access. For a good example: https://clementneo.

Don’t let the mathematical terms fool you, these are just fan theories. For a real investigation you need debug access. For a good example: https://clementneo.com/posts/2023/02/11/we-found-an-neuron https://clementneo.com/posts/2023/02/11/we-found-an-neuron For image recognition, machine learning researchers eventually figured out the neural networks are paying attention mostly to textures. Hopefully we will have a better understanding of what language models are doing someday.