4 ms·
Is it accurate to say they were trained for less than $600? Wouldn't that just be the finetuning that was done to the already existing LLaMA parameters which li
by UncleOxidant 4y ago
Is it accurate to say they were trained for less than $600? Wouldn't that just be the finetuning that was done to the already existing LLaMA parameters which likely cost way more than $600 to train?
- simonw 4y agoYeah, exactly. LLaMA 7B itself cost $80,000+ to train (82,432 GPU hours). Stanford spent $100 on fine-tuning compute and $500 on OpenAI credits to generate their 52,000 sample instruction training set.