3 ms·
Is what your saying imply that there is a rotation matrix you can apply to each activation output to make it less entangled?
by ipunchghosts 1y ago
Is what your saying imply that there is a rotation matrix you can apply to each activation output to make it less entangled?
- goldemerald 1y agoNot quite. For an underlying semantic concept (e.g., smiling face), you can go from a basis vector [0,1,0,...,0] to the original latent space via a single rotation. You could then induce said concept by manipulating the original latent point by traversing along that linear direction.
- ipunchghosts 1y agoI think we are saying the same thing. Please correct me though where I am wrong. You could look at the maps in some way but instead of the basis being one hot dimensions (the standard basis), it could be rotated.
- akarshkumar0101 1y agoWe mention this issue exactly in the fourth paragraph in Section 4 and in Appendix F!