2 ms·
> The fixed cause due to R&D is what makes it unprofitable but not each request. If the amortisation period is too short (what is it now? 8 months? 6 months?)
by lelanthran 1y ago
> The fixed cause due to R&D is what makes it unprofitable but not each request.
If the amortisation period is too short (what is it now? 8 months? 6 months?) that "profit" from each inference token has to cover the training costs before the amortisation schedule ends.
In short, if you're making a profit of $1 on each unit sold, but require a capex of $10 in order to provide the unit sold, you need to sell at least 10 of those units to break even.
The training is the capex, the inference profit is the profit/unit sold. When a SOTA model lasts only 8 months, the inference has to make all that back in 8 months in order to be considered profitable.