3 ms·
So... distribute a LoRA (or equivalent) that modifies the base weights with the abliteration vectors. That makes sense as it would be possible to try different
by rhdunn 2mo ago
So... distribute a LoRA (or equivalent) that modifies the base weights with the abliteration vectors. That makes sense as it would be possible to try different abliterations and keep the storage space down.
- wren6991 2mo agoYeah, orthogonalisation is a special case of a rank-1 LoRA.