3 ms·
Shouldn't inference be 99.999% of the compute cost over time? Especially for MS. Look how many Copilots they are cramming into their products
by pwarner 3y ago
Shouldn't inference be 99.999% of the compute cost over time?
Especially for MS.
Look how many Copilots they are cramming into their products
- visarga 3y agoUpfront, training costs 1000x more than inference - about 0.01/token vs 0.01/1000 tokens. But considering the user base size and the size of the training set - 15T tokens for GPT-4, I estimate the total inference cost becomes equal to training at around 10K tokens/user/month and 100M users.