3 ms·
Anybody have an idea of what kind of hardware (and cost) you would need to train the model and to execute it ? Obviously storage is not a major factor here.
by christkv 4y ago
Anybody have an idea of what kind of hardware (and cost) you would need to train the model and to execute it ?
Obviously storage is not a major factor here.
- HarHarVeryFunny 4y agoI seem to recall that the training cost for ChatGPT was in the tens of millions of dollars. Execution cost is on the order of ~$1 per interaction.
- shagie 4y agoThe cost per day isn't something that there are any reliable sources for. The closest to an authoritative source on it is https://twitter.com/sama/status/1599671496636780546 https://twitter.com/sama/status/1599671496636780546 > average is probably single-digits cents per chat; trying to figure out more precisely and also how we can optimize it An attempt to work through it from related resources is https://twitter.com/tomgoldsteincs/status/1600196981955100694 https://twitter.com/tomgoldsteincs/status/160019698195510069... In particular https://twitter.com/tomgoldsteincs/status/1600196990905614336 https://twitter.com/tomgoldsteincs/status/160019699090561433... > So what would this cost to host? On Azure cloud, each A100 card costs about $3 an hour. That's $0.0003 per word generated. > But it generates a lot of words! The model usually responds to my queries with ~30 words, which adds up to about 1 cent per query. --- It is much less than $1/interaction.