4 ms·
You'd have to make refusal a high rank subspace, and that seems like it could be quite difficult. One alternative approach I've seen is to make refusal behavior
by pizza 2y ago
You'd have to make refusal a high rank subspace, and that seems like it could be quite difficult. One alternative approach I've seen is to make refusal behavior more likely to just output the end-of-speech token.
- eigenvalue 2y agoThere are ways to do that from linear algebra, like orthogonalization processes (e.g., Gram-Schmidt process) followed by basis expansion. Or random projections can also be used in a similar way. And I’m sure there are much fancier techniques that draw upon higher math (like Grassmannians or Teichmuller mappings).
- ziofill 2y agoWould you be able to share some links to these techniques? They sound related to something I’m working on but in an entirely different field.
- mistercow 2y agoI wonder if you could do this with multiple alignment training passes, where you extract the refusal direction each time, and suppress it in future training passes.