He also mentions sparsely activating only the neurons that matter, which they explore in https://arxiv.org/pdf/1701.06538.pdf https://arxiv.org/pdf/1701.06538.p

He also mentions sparsely activating only the neurons that matter, which they explore in https://arxiv.org/pdf/1701.06538.pdf https://arxiv.org/pdf/1701.06538.pdf Personally I didn't find it very satisfying, I imagine something more fundamental and self referential.