4 ms·
> exploiting the fact that factual knowledge in an LLMs has generally been shown to be localized to particular transformer layers This is surprising
by totetsu 2y ago
> exploiting the fact that factual knowledge in an LLMs has generally been shown to be localized to particular transformer layers
This is surprising
- dinobones 2y agoWhy is this surprising? It makes sense that “facts” exist in earlier layers, then these become more abstract as you become deeper. This reminds me of residual connections from CNNs and vision.
- snthpy 2y agoInteresting. I hadn't considered that before but makes sense.
- deleted 2y ago[deleted]