3 ms·
LoRA stands for low rank approximation - it’s a very clever way to fine tune large models with a fraction of the compute, the idea being that you need much less
by jerpint 3y ago
LoRA stands for low rank approximation - it’s a very clever way to fine tune large models with a fraction of the compute, the idea being that you need much less capacity to estimate the finetuning
- taneq 3y agoThanks, yeah I know what LoRA is and I didn't think it was so effective on... are we OK with calling it the 'cognitive' side yet? Everything I've read from the LocalLLaMA finetuning community is that you can fine tune presentation but it's not really an effective way to teach models new knowledge.