4 ms·

I believe they can currently be used productively for non-coding tasks (classification, light summary)... but they definitely are not even close to SOTA when it

• _hugerobots_ 5d ago
  Defining productivity is a use-case scenario, and a wildly generalized assumption for most people in this argument. Local infrastructure doesn't need to be sota for absolutely every single need for a dev lab, but it absolutely can be delivered with non-api frontier class models.
  • slowin 5d ago
    Just to be clear, I'm specifically talking about coding. I think local models can help with productivity today, just not coding. I'm also a huge fan of local models and think it's absolutely imperative that they continue to advance so we can move off of the Anthropic/OpenAI hosted models. It's important to accurately asses where we are in that journey though.
    • srcreigh 5d ago
      I think the issue is generalization, if you were more specific about which local models aren’t good enough for which tasks compared to which frontier models in your experience, it’d be a lot more informative
      • slowin 5d ago
        I can't just go into any codebase and ask a local model to "Implement this feature: xxx" and get acceptable output. I hope to someday soon though!
    • _hugerobots_ 5d ago
      Like the other commenter, I'm confused about the 'just not coding' conclusion. I'm using Qwen 27B on a 5090 at > 100tk/s with 150k context (which isn't enough admittedly), and DeepSeek v4 Flash with 1million context on a gb10/spark. Both of which are performing surface level, and deep needle precision infrastructure architecture. They code 24-7, stupendously.