3 ms·
Tell us more about LoRA
by EGreg 4y ago
Tell us more about LoRA
- deleted 4y ago[deleted]
- visarga 4y agoInstead of a matrix NN they use two matrices, Nd and d*N, and assuming d << N it would be like a lightweight addition. Keeping the original matrix unchanged, this new addition is simply added in parallel and initialised to low values.