3 ms·
> inference APIs aren’t subsidised How do you get to that conclusion? There is no inference without training, so each sale of a single inference token has a co
by lelanthran 1y ago
> inference APIs aren’t subsidised
How do you get to that conclusion? There is no inference without training, so each sale of a single inference token has a cost that includes both the inference as well as the amortised cost of training.
- simianwords 1y agoThat's what the post means. OpenAI doesn't lose money on each request but rather gains it. To recuperate the fixed costs in R&D.
- lelanthran 1y ago> OpenAI doesn't lose money on each request but rather gains it. To recuperate the fixed costs in R&D. Right, but that is why I used the word "amortise"; there is only a limited time to recuperate that cost. If you spend $120 in training, and it takes 6 months for the next SOTA to drop from a competitor, you have to make clear $10/m after inference costs.