4 ms·
No, this is an open reproduction of Alpaca on the leaked LLaMA 7B weight set.
by Shank 4y ago
No, this is an open reproduction of Alpaca on the leaked LLaMA 7B weight set.
- thewataccount 4y agoThis isn't actually a direct reproduction. It's a bit missleading because it only says it at the bottom. This actually uses LORAs which train way faster and are much more flexible, but they perform a little worse. There is someone that is doing a direct reproduction of Alpaca in the alpaca repo issues though. > The weights are based on the published fine-tunes from alpaca-lora
- GaggiX 4y ago>they perform a little worse. Be aware that LoRA performs on-par or better than fine-tuning in model quality if trained correctly as the paper shows: https://arxiv.org/abs/2106.09685 https://arxiv.org/abs/2106.09685
- thewataccount 4y agoAh thank you for the correction this is good to know! I'll admit I'm not super experienced with them directly. I think the current LoRAs for Alpaca are reportedly performing a little bit worse - I'm guessing from a lack of optimization so I'm sure it'll improve.