4 ms·
How would you characterize the degree of explainability of large language models (e.g. gpt3)? Anything surprising or non-intuitive there?
by p1esk 4y ago
How would you characterize the degree of explainability of large language models (e.g. gpt3)? Anything surprising or non-intuitive there?
- behnamoh 4y agoi think a more general question is how would you measure explainability when you see it? is there some sort of metric for that?
- visarga 4y agoOne weak point I see - this tool only measures how much an individual input token would be changed to decrease the loss. The reason a token might have large gradients might be related to how many times it appears in the training set and how consistent is the training set with the evaluation set, not just how much the prediction disagrees with the target label. So it jointly measures data coverage, consistency and data to target fit. Just my intuition, might be wrong.