3 ms·
Training costs are fixed. Inference costs are variable. The difference matters.
by simonw 9mo ago
Training costs are fixed. Inference costs are variable. The difference matters.
- vrighter 9mo agoNo they are not. They are exponentially increasing. Due to the exponential scaling needed for linear gain. Otherwise they'd fall behind their competition.
- simonw 9mo agoFixed cost here means that the training costs stay the same no matter how many customers you have - unlike serving costs which have to increase to serve more people.
- 3form 9mo agoYet somewhere above you said: >Training costs are fixed at whatever billions of dollars per year. Which I think is the part people disagree with.
- simonw 9mo agoI used the word "fixed" there to indicate that the cost of training is unaffected by how many users you have, unlike the cost of serving the model which increases as your usage increases.