3 ms·
Cost to train this wouldn't be very high. A regular CPU can generate or train about 2 words per second. 52,000 training examples of an average of 300 words w
by londons_explore 4y ago
Cost to train this wouldn't be very high.
A regular CPU can generate or train about 2 words per second. 52,000 training examples of an average of 300 words would take 3 CPU months. And it is fully parallelizable.
And you can probably make it go far faster on a GPU.
- simonw 4y agoThey say they trained it for $600.
- IanCal 4y agoUnder $100. > Fine-tuning a 7B LLaMA model took 3 hours on 8 80GB A100s, which costs less than $100 on most cloud compute providers. They also spent $500 creating the dataset using openais APIs.
- danielbln 4y agoThose $500 would be $50 if they use gpt3.5-turbo, which is a magnitude cheaper than davinci-3.