4 ms·
Dozens of providers of open-weight models. I have one session going with synthetic.new and another going with novita.ai right now.
by joshheitzman 1mo ago
Dozens of providers of open-weight models. I have one session going with synthetic.new and another going with novita.ai right now.
- gymbeaux 1mo agoBut they’re slower (and benchmark worse) than Opus, GPT, et al. Why?
- joshheitzman 1mo agoPersonally I don't care about common benchmarks as I don't find actual coding agent performance correlates strongly with them. One reason to use open-weight models is that they don't hide the reasoning, so you can do very aggressive context management in your harness to use significantly less tokens. Smaller prompts are faster since KV is N^2 plus it can be dramatically cheaper (if you balance your aggressive context management with maintaining the prefix cache as much as possible). Even paying for API prices directly and using agents as much as I want I spend less per month than the $200 I spent on a Claude MAX sub when I had it.