3 ms·
The downside is that it's a supervised technique, so you need to already know what you're looking for. It would be nice to have an unsupervised tool that could
by Legend2440 3y ago
The downside is that it's a supervised technique, so you need to already know what you're looking for. It would be nice to have an unsupervised tool that could list out all the things the network has learned.
- JoshuaDavid 3y agoAnthropic has published some cool stuff in that direction: https://transformer-circuits.pub/2023/monosemantic-features https://transformer-circuits.pub/2023/monosemantic-features
- firejake308 3y agoWhoa, this is super cool! I can imagine if we had something like this for ChatGPT, we could use it to do some serious prompt engineering. Imagine seeing what specific neurons you were activating with your prompt, and being able to identify which word in your prompt was triggering an undesired behavior. Super cool stuff, excited to see if it scales