VeRA makes LoRA ~10x more parameter efficient while retaining the same performance. It's somewhat like a recursive LoRA scheme, where the LoRA A and B matrices

VeRA makes LoRA ~10x more parameter efficient while retaining the same performance. It's somewhat like a recursive LoRA scheme, where the LoRA A and B matrices are also decomposed using two small trainable vector parameters.