4 ms·
Interesting. Is there a paper that explains this in a bit more detail, like [0] for abliteration (underlying the Heretic software, afaik)? [0] https://arxiv.or
by c7b 13d ago
Interesting. Is there a paper that explains this in a bit more detail, like [0] for abliteration (underlying the Heretic software, afaik)?
[0] https://arxiv.org/abs/2406.11717 https://arxiv.org/abs/2406.11717
- wren6991 13d agoThat paper you linked has all of the information you need. The linear algebra they do on the weights there to null out one direction reduces to a single (dot-product + broadcast-multiply + add) on the activations.