23 ms·

Stupid question, but are there models worth using that specialize in a particular programming language? For instance, I'd love to be able to run a local model o

Stupid question, but are there models worth using that specialize in a particular programming language? For instance, I'd love to be able to run a local model on my GPU that is specific to C/C++ or Python. If such a thing exists, is it worth it vs one of the cloud-based frontier models? I'm guessing that a model which only covers a single language might be more compact and efficient vs a model trained across many languages and non-programming data.

cpburns2009 7mo ago
I'd be interested in this too. I think that's what post-training can achieve but I've never looked into it.
girvo 7mo ago
I'm currently experimenting with (trying to) fine tune Qwen3.5 to make it better at a given language (Nim in this case); but I am quite bad at this, and honestly am unsure if it's even really fully feasible at the scale I have access to. Certainly been fun so far though, and I have a little Asus GX10 box on the way to experiment some more!
- embedding-shape 7mo ago
  Been playing around with fine-tuning models for specific languages as well (Clojure and Rust mostly), but the persistent problem is high quality data sets, mostly I've been generating my own based on my own repositories and chat sessions, what approach are you taking for gathering the data?
  - girvo 7mo ago
    Yeah I have a body of work of 15 years, plus I’ve been building labelled sets from open source (the source code isn’t quite enough on its own) Now I’m using that to generate synthetic sets and clean it up, but man I’m struggling hah. Fun though.
  - numberwan9 7mo ago
    Have you looked on HF? Here's one that is fine tuned on Rust https://huggingface.co/Fortytwo-Network/Strand-Rust-Coder-14B-v1 https://huggingface.co/Fortytwo-Network/Strand-Rust-Coder-14...
epolanski 7mo ago
My own experience trying many different models is that general intelligence of the model is more important. If you want it to stick to better practices you have to write skills, provide references (example code it can read), and provide it with harnessing tools (linters, debuggers, etc) so the agent can iterate on its own output.
Fulgidus 7mo ago
Months ago I tested a concept revolving this issue and made a weird MCP-LSP-LocalLLM hybrid thing that attempts to enhance unlucky, fast changing, or unpopular languages (mine attempts with Zig) Give it a look, maybe it could inspire you: https://github.com/fulgidus/zignet https://github.com/fulgidus/zignet Bottom-line: fine-tuning looks like the best option atm

cpburns2009 7mo ago

I'd be interested in this too. I think that's what post-training can achieve but I've never looked into it.

girvo 7mo ago

I'm currently experimenting with (trying to) fine tune Qwen3.5 to make it better at a given language (Nim in this case); but I am quite bad at this, and honestly am unsure if it's even really fully feasible at the scale I have access to. Certainly been fun so far though, and I have a little Asus GX10 box on the way to experiment some more!

embedding-shape 7mo ago
Been playing around with fine-tuning models for specific languages as well (Clojure and Rust mostly), but the persistent problem is high quality data sets, mostly I've been generating my own based on my own repositories and chat sessions, what approach are you taking for gathering the data?
- girvo 7mo ago
  Yeah I have a body of work of 15 years, plus I’ve been building labelled sets from open source (the source code isn’t quite enough on its own) Now I’m using that to generate synthetic sets and clean it up, but man I’m struggling hah. Fun though.
- numberwan9 7mo ago
  Have you looked on HF? Here's one that is fine tuned on Rust https://huggingface.co/Fortytwo-Network/Strand-Rust-Coder-14B-v1 https://huggingface.co/Fortytwo-Network/Strand-Rust-Coder-14...

epolanski 7mo ago

My own experience trying many different models is that general intelligence of the model is more important. If you want it to stick to better practices you have to write skills, provide references (example code it can read), and provide it with harnessing tools (linters, debuggers, etc) so the agent can iterate on its own output.

Fulgidus 7mo ago

Months ago I tested a concept revolving this issue and made a weird MCP-LSP-LocalLLM hybrid thing that attempts to enhance unlucky, fast changing, or unpopular languages (mine attempts with Zig) Give it a look, maybe it could inspire you: https://github.com/fulgidus/zignet https://github.com/fulgidus/zignet Bottom-line: fine-tuning looks like the best option atm