2 ms·Yeah, orthogonalisation is a special case of a rank-1 LoRA.by wren6991 6d agoYeah, orthogonalisation is a special case of a rank-1 LoRA.ziofill 6d agoI don’t think this is true.wren6991 6d agoCould you please explain why? It's an additive update to the weights, adding an outer product of a vector with its transpose, which must have rank 1. What am I missing?