3 ms·
As the author stated in this post, it's not actually one vector, but a list of one vector per layer. If I understand it correctly, these vectors can have differ
by semi-extrinsic 3y ago
As the author stated in this post, it's not actually one vector, but a list of one vector per layer. If I understand it correctly, these vectors can have different total magnitude across the layers. If the PCA (or other technique) identifies that layers 17, 36 and 41 are important for "concept X", the vectors for those layers will be the strongest when repeng'ing for that concept.
- danm1618 3y agoA note worth mentioning is that the PCA is not trained across the layers, but independently on each layer, across the provided examples. Nevertheless, it's conceivable that specific layers could possess a significant control vector, but not solely because of directly leveraging the first principal component.