3 ms·

So... distribute a LoRA (or equivalent) that modifies the base weights with the abliteration vectors. That makes sense as it would be possible to try different

by rhdunn 12d ago

So... distribute a LoRA (or equivalent) that modifies the base weights with the abliteration vectors. That makes sense as it would be possible to try different abliterations and keep the storage space down.

wren6991 11d ago
Yeah, orthogonalisation is a special case of a rank-1 LoRA.
- ziofill 11d ago
  I don’t think this is true.
  - wren6991 11d ago
    Could you please explain why? It's an additive update to the weights, adding an outer product of a vector with its transpose, which must have rank 1. What am I missing?