3 ms·

no FIM though :(, imo most slept on usecase for local models

• harkh 6mo ago
  I agree, what would you say are currently the best local options?
  • zargon 6mo ago
    For FIM, there's Qwen3 Coder Next. Although Mistral's model card seems to indicate that Devstral 2 doesn't support FIM, it seems very odd that it wouldn't. I have been meaning to test it.
    • thot_experiment 6mo ago
      Qwen Coder 30B A3B is far better than Qwen Coder Next imo. I may have inference issues or it's just a problem with running Coder Next at IQ4 XS, vs Q8 for the earlier/smaller model but I don't find the 80B to be much better at coding, even in instruct mode, and the insane speed and low latency of the smaller model is way more useful. Good one-line completions often happen in 300ms.
• deleted 6mo ago
  [deleted]