4 ms·
Number of params isn’t really the relevant metric imo. Top models don’t support local inference. More relevant is tokens per dollar or per second.
by janalsncm 8mo ago
Number of params isn’t really the relevant metric imo. Top models don’t support local inference. More relevant is tokens per dollar or per second.
- dakolli 8mo agoIts an open source model, why wouldn't it be relevant for people who want to self host.....
- lm28469 8mo agoIt does since you can run this model locally on a < $3k machine
- qeternity 8mo agoNumber of parameters is at least a proxy for model capability. You can achieve incredible tok/dollar or tok/sec with Qwen3 0.6b. It just won't be very good for most use cases.
- janalsncm 8mo agoModel capability is the other axis on their chart. So they could have put Qwen 0.6b there, it would be in the bottom right corner. I know what they are trying to do. They are attempting show a kind of pareto frontier but it’s a little awkward.