3 ms·
Sorry this is moving too fast for me. So if I understand correctly, LoRa kind of does what Alpaca does but using different data. So what is Alpaca-Lora? I know
by zapdrive 4y ago
Sorry this is moving too fast for me. So if I understand correctly, LoRa kind of does what Alpaca does but using different data.
So what is Alpaca-Lora? I know you get Alpaca by retraining Llama using Stanford Alpaca 52k instruction-following data? So if I am guessing right, you get Aplaca-Lora by retraining Alpaca using Lora's data?
- return_to_monke 4y agoI think your first statement is incorrect. Lora seems to be a method to fine-tune and optimize the weights of models like Alpaca. It is not a different dataset. This reduces model sizes and therefore also compute costs. See the abstract of https://arxiv.org/pdf/2106.09685.pdf https://arxiv.org/pdf/2106.09685.pdf