4 ms·
How much would they cost aprox per training? What would the quality of the model be, compared to what Karpathy uses on his video “let’s build gpt from scratch”
by nico 3y ago
How much would they cost aprox per training?
What would the quality of the model be, compared to what Karpathy uses on his video “let’s build gpt from scratch”?
In that video, he builds a decoder-only transformer model, that learns Shakespeare from 1MB of data and trains in 15min
- jiggawatts 3y agoThese just fine-tune existing models, so the cost is single digit dollars. Whatever it costs in electricity to run a desktop GPU for a day or a few days.