Yes, there's no physical reason it couldn't be fixed. However, I think it's less a matter of computing power, and more one of architecture and training. The in

Yes, there's no physical reason it couldn't be fixed. However, I think it's less a matter of computing power, and more one of architecture and training. The inference-time code could be changed to keep a log of which neurons were activated the most when it generated a token. But then, how would we train it to use that log and report it accurately? Training on Internet documents isn't enough. Chain-of-thought reasoning helps as a workaround. However, beware that even then, the justifications aren't a reliable explanation. There's an interesting paper on this: Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting https://arxiv.org/abs/2305.04388 https://arxiv.org/abs/2305.04388