3 ms·
Just for quick reference: LLM - the model weights - the data of the model itself LoRA - a small addition to the model for more efficient fine-tuning and dist
by visarga 4y ago
Just for quick reference:
LLM - the model
weights - the data of the model itself
LoRA - a small addition to the model for more efficient fine-tuning and distribution
- EGreg 4y agoTell us more about LoRA
- deleted 4y ago[deleted]
- visarga 4y agoInstead of a matrix NN they use two matrices, Nd and d*N, and assuming d << N it would be like a lightweight addition. Keeping the original matrix unchanged, this new addition is simply added in parallel and initialised to low values.