4 ms·
>I'm interested in seeing the results of an Alpaca method You're talking apples to oranges. The "Alpaca method" is a dataset generation method. Nothing about A
by MacsHeadroom 4y ago
>I'm interested in seeing the results of an Alpaca method
You're talking apples to oranges. The "Alpaca method" is a dataset generation method. Nothing about Alpaca's training method is novel, interesting, or efficient. Alpaca used the same standard training method everyone else uses, A100 clusters.
If you mean LoRA/PEFT training which people used to replicate Alpaca then that is also apples to oranges because LoRA/PEFT is a finetuning method not a pre-training method.
- deleted 4y ago[deleted]
- UncleEntity 4y agoOne could take the alpaca dataset and fine tune using the LoRA/PEFT method and compare to the Stanford alpaca fine tuned llama model. Presumably…