See also the new Deepseek paper on engram transformers for some progress in this area: https://arxiv.org/pdf/2601.07372v1 https://arxiv.org/pdf/2601.07372v1 Th

See also the new Deepseek paper on engram transformers for some progress in this area: https://arxiv.org/pdf/2601.07372v1 https://arxiv.org/pdf/2601.07372v1 They observe significant gains in factual knowledge retrieval capabilities, but reasoning barely moves the needle.