4 ms·

Can rank decomposition be used to reduce the original weight matrices as well? Or are they assumed to be compressed already?

• metanonsense 4y ago
  Those fully trained networks are usually considered full-rank. At least that is what they say in the LoRA paper.