3 ms·
Looks very neat. How do you use it? Looks like it's just to inspect a personal model and can't be applied to external models, is that right?
by segmondy 3y ago
Looks very neat. How do you use it? Looks like it's just to inspect a personal model and can't be applied to external models, is that right?
- robkop 3y agoThere's a task on my list to write a full tutorial using it to replicate some recent interpretability research (finding induction heads is up first). But even without a full tutorial, I've been surprised how quickly people have been able to pick up and understand it just by selecting a model and playing around. If you are interested there is this brilliant tutorial [1] by Callum McDougall for the Transformer Lens library. Going through its steps but completing them in Transpector would be a great way to learn it and build out intuition about transformers/ where research is today. On the model side, I've added a supported model list [2] and a gif of how to switch between models [3], I appreciate the feedback on what information is the most useful for the readme. Furthermore just being aware your question may have been in regards to API access only models (GPT4, Bard...), unfortunately Transpector requires access to the model weights and activations so currently it's not possible to use with those. [1]: https://colab.research.google.com/drive/1LpDxWwL2Fx0xq3lLgDQvHKM5tnqRFeRM?usp=share_link https://colab.research.google.com/drive/1LpDxWwL2Fx0xq3lLgDQ... [2]: https://github.com/R0bk/Transpector/blob/main/docs/supportedModels.md https://github.com/R0bk/Transpector/blob/main/docs/supported... [3]: https://github.com/R0bk/Transpector/blob/main/README.md https://github.com/R0bk/Transpector/blob/main/README.md