2 ms·

Again, put those under test with your private benchmarks, then compare the results with hosted models. I'm not saying it's completely useless, or that I don't

Again, put those under test with your private benchmarks, then compare the results with hosted models. I'm not saying it's completely useless, or that I don't think it won't be better in the future. What I am saying is that even the top "weights available" models today really don't come close to today's SOTA. This is very clear when you have benchmarks to get hard concrete numbers that aren't influenced by public benchmarking data.

seanmcdirmid 11mo ago
> even the top "weights available" models today really don't come close to today's SOTA. This is the statement thatI'm disagreeing with. They do come close, even if they are somehow less, it is a fixed distance away where the hosted models aren't more than a magnitude better. Hosted models are still better, just not incredibly so.