3 ms·

I am hoping the actual Alpaca weights leak at some point. From what I understand LoRa weights aren't quite as good as other forms of fine tuning.

antimatter15 4y ago
From my experimentation it seems like there's some significant loss in accuracy running the tuned LoRa models through llama.cpp (due to bugs/differences in inference or tokenization), even aside from losses due to quantization.
loxias 4y ago
Look at the changeset :) It contains a link for "ggml-alpaca-7b-14.bin". Because there's no substantive change to the code, I assume this fork exists (and this HN post exists) purely as a method to distribute the weights.
- Tiberium 4y ago
  You're misinterpreting the comment a bit, the poster above is asking for the original Alpaca weights. The code in the repo uses a reproduction of Alpaca with different training code and method of fine-tune (LoRA).
GaggiX 4y ago
There is no reason to leak the Alpaca weights, finetuning the model is not that expensive, eventually someone will finetune one and release it. (Also LoRA should performs on-par or better than fine-tuning in model quality)
- ComplexSystems 4y ago
  There are different fine tuning methods. Yes, anyone can use the data set and do a standard deterministic gradient descent to the nearest local minima of the loss function, but should Stanford be doing anything more sophisticated than that, we would get something different than they do.
  - GaggiX 4y ago
    It is enough to use a different seed to get a different result; I do not think it is important to recreate a copy of the Stanford model weights, as long as the performance is equivalent.
  - danielbln 4y ago
    Stanford released the exact training data as well as the training script with all parameters. Boot up a p4.2xlarge (8 A100 GPUs) which costs about $40/hour and let it run for a 2-3 hours and voila. See the Readme in their repo where it mentions the fine-tuning script[0] [0] https://github.com/tatsu-lab/stanford_alpaca https://github.com/tatsu-lab/stanford_alpaca

antimatter15 4y ago

From my experimentation it seems like there's some significant loss in accuracy running the tuned LoRa models through llama.cpp (due to bugs/differences in inference or tokenization), even aside from losses due to quantization.

loxias 4y ago

Look at the changeset :) It contains a link for "ggml-alpaca-7b-14.bin". Because there's no substantive change to the code, I assume this fork exists (and this HN post exists) purely as a method to distribute the weights.

Tiberium 4y ago
You're misinterpreting the comment a bit, the poster above is asking for the original Alpaca weights. The code in the repo uses a reproduction of Alpaca with different training code and method of fine-tune (LoRA).

GaggiX 4y ago

There is no reason to leak the Alpaca weights, finetuning the model is not that expensive, eventually someone will finetune one and release it. (Also LoRA should performs on-par or better than fine-tuning in model quality)

ComplexSystems 4y ago
There are different fine tuning methods. Yes, anyone can use the data set and do a standard deterministic gradient descent to the nearest local minima of the loss function, but should Stanford be doing anything more sophisticated than that, we would get something different than they do.
- GaggiX 4y ago
  It is enough to use a different seed to get a different result; I do not think it is important to recreate a copy of the Stanford model weights, as long as the performance is equivalent.
- danielbln 4y ago
  Stanford released the exact training data as well as the training script with all parameters. Boot up a p4.2xlarge (8 A100 GPUs) which costs about $40/hour and let it run for a 2-3 hours and voila. See the Readme in their repo where it mentions the fine-tuning script[0] [0] https://github.com/tatsu-lab/stanford_alpaca https://github.com/tatsu-lab/stanford_alpaca