3 ms·
The linear representation hypothesis is rather quite intreguing, I am curious what was the intuition behind it.
by 3abiton 1y ago
The linear representation hypothesis is rather quite intreguing, I am curious what was the intuition behind it.
- colah3 1y agoSee https://transformer-circuits.pub/2022/toy_model/index.html#motivation https://transformer-circuits.pub/2022/toy_model/index.html#m... If you're new to this, I'd mostly just look at all the empirical examples. The slightly harder thing is to consider the fact that neural networks are made of linear functions with non-linearities between them, and to try to think about when linear directions will be computationally natural as a result.