4 ms·
> Without training cost you can infer only the marginal cost of serving this kind of models. Which is by far the most interesting number of the two. > Moreov
by amelius 3mo ago
> Without training cost you can infer only the marginal cost of serving this kind of models.
Which is by far the most interesting number of the two.
> Moreover, you don't know the actual size of closed models (what if Fable is a 10T model? What if it's 1T?)
If you get close in output quality, then does that matter?
- embedding-shape 3mo ago> If you get close in output quality, then does that matter? When you're trying to estimate/infer the costs of serving the tokens and even include the cost of training the weights in order to output tokens then yeah, why wouldn't that matter?
- jvuygbbkuurx 3mo agoThat only matters if you are an investor not if you are a consumer.
- embedding-shape 3mo agoWell, or if you're participating in a discussion on HN where the sub-topic happens to be "if labs are subsidising tokens on API pricing" and literally the cost of serving the tokens is relevant to the sub-topic people are trying to discuss...
- solenoid0937 3mo agoConsumers want better models too, of course it matters
- vb-8448 3mo ago> Which is by far the most interesting number of the two. Only if you don't have to continuously train new models, and you are not at a runway risk.