4 ms·
This isn't actually a direct reproduction. It's a bit missleading because it only says it at the bottom. This actually uses LORAs which train way faster and are
by thewataccount 4y ago
This isn't actually a direct reproduction. It's a bit missleading because it only says it at the bottom. This actually uses LORAs which train way faster and are much more flexible, but they perform a little worse.
There is someone that is doing a direct reproduction of Alpaca in the alpaca repo issues though.
> The weights are based on the published fine-tunes from alpaca-lora
- GaggiX 4y ago>they perform a little worse. Be aware that LoRA performs on-par or better than fine-tuning in model quality if trained correctly as the paper shows: https://arxiv.org/abs/2106.09685 https://arxiv.org/abs/2106.09685
- thewataccount 4y agoAh thank you for the correction this is good to know! I'll admit I'm not super experienced with them directly. I think the current LoRAs for Alpaca are reportedly performing a little bit worse - I'm guessing from a lack of optimization so I'm sure it'll improve.