3 ms·
Given how little most of us can know about the true cost of inference for these providers (and thus the financial sustainability of their services), this is an
by throwaway132448 8mo ago
Given how little most of us can know about the true cost of inference for these providers (and thus the financial sustainability of their services), this is an interesting signal. Not sure how to interpret it, but it doesn’t feel like it bodes well.
- not_math 8mo agoGiven that providers of open source models can offer Kimi K2.5 at input $0.60 and output $2.50 per million tokens, I think the cost of inference must be around that. We would still need to compare the tokens per second.
- Computer0 8mo agoFair but we technically do not know the parameter count