It depends how you parse it. It is clearly true that they 'can' explain neurons, in the sense that at least some of the neurons are quite well explained. On the

It depends how you parse it. It is clearly true that they 'can' explain neurons, in the sense that at least some of the neurons are quite well explained. On the other hand, it's also the case that the vast majority of neurons are not well explained at all by this method (or likely any method). I̵t̵'̵s̵ ̵o̵n̵l̵y̵ ̵b̵e̵c̵a̵u̵s̵e̵ ̵o̵f̵ ̵a̵ ̵q̵u̵i̵r̵k̵ ̵o̵f̵ ̵A̵d̵a̵m̵W̵ ̵t̵h̵a̵t̵ ̵t̵h̵i̵s̵ ̵i̵s̵ ̵p̵o̵s̵s̵i̵b̵l̵e̵ ̵a̵t̵ ̵a̵l̵l̵,̵ ̵i̵f̵ ̵G̵P̵T̵-̵2̵ ̵w̵a̵s̵ ̵t̵r̵a̵i̵n̵e̵d̵ ̵w̵i̵t̵h̵ ̵S̵G̵D̵ ̵a̵l̵m̵o̵s̵t̵ ̵n̵o̵ ̵n̵e̵u̵r̵o̵n̵s̵ ̵w̵o̵u̵l̵d̵ ̵b̵e̵ ̵i̵n̵t̵e̵r̵p̵r̵e̵t̵a̵b̵l̵e̵.̵ EDIT: This last part isn't true. I think they are only looking at the intermediate layer of the FFN which does have a privileged basis.