3 ms·
I think this is a generous usage of "can." As the article admits, these explanations are 'imperfect' and I think that is definitely true.
by whimsicalism 4y ago
I think this is a generous usage of "can." As the article admits, these explanations are 'imperfect' and I think that is definitely true.
- sebzim4500 4y agoIt depends how you parse it. It is clearly true that they 'can' explain neurons, in the sense that at least some of the neurons are quite well explained. On the other hand, it's also the case that the vast majority of neurons are not well explained at all by this method (or likely any method). I̵t̵'̵s̵ ̵o̵n̵l̵y̵ ̵b̵e̵c̵a̵u̵s̵e̵ ̵o̵f̵ ̵a̵ ̵q̵u̵i̵r̵k̵ ̵o̵f̵ ̵A̵d̵a̵m̵W̵ ̵t̵h̵a̵t̵ ̵t̵h̵i̵s̵ ̵i̵s̵ ̵p̵o̵s̵s̵i̵b̵l̵e̵ ̵a̵t̵ ̵a̵l̵l̵,̵ ̵i̵f̵ ̵G̵P̵T̵-̵2̵ ̵w̵a̵s̵ ̵t̵r̵a̵i̵n̵e̵d̵ ̵w̵i̵t̵h̵ ̵S̵G̵D̵ ̵a̵l̵m̵o̵s̵t̵ ̵n̵o̵ ̵n̵e̵u̵r̵o̵n̵s̵ ̵w̵o̵u̵l̵d̵ ̵b̵e̵ ̵i̵n̵t̵e̵r̵p̵r̵e̵t̵a̵b̵l̵e̵.̵ EDIT: This last part isn't true. I think they are only looking at the intermediate layer of the FFN which does have a privileged basis.
- whimsicalism 4y ago> EDIT: This last part isn't true. I think they are only looking at the intermediate layer of the FFN which does have a privileged basis. it does?
- sebzim4500 4y agoYeah, that's where they apply the activation function and that happens per neuron so you can't rotate everything and expect the same result.