3 ms·
I know it isn’t your question exactly, and you probably know this, but the models for coding assist tools are generally fine tunes of models for coding specific
by CardenB 1y ago
I know it isn’t your question exactly, and you probably know this, but the models for coding assist tools are generally fine tunes of models for coding specific purposes. Example: in OpenAI codex they use GPT-5-codex
- neutronicus 1y agoI think the question is, can I throw a couple thousand bucks of GPU time at fine-tuning a model to have knowledge of our couple million lines of C++ baked into the weights instead of needing to fuck around with "Context Engineering". Like, how feasible is it for a mid-size corporation to use a technique like LoRA, mentioned by GP, to "teach" (say, for example) Kimi K2 about a large C++ codebase so that individual engineers don't need to learn the black art of "context engineering" and can just ask it questions.
- pu_pe 1y agoI'm curious about it too. I think there are two bottlenecks, one is that training a relatively large LLM can be resource-intensive (so people go for RAGs and other shortcuts), and making it finetuned to your use cases might make it dumber overall.
- koakuma-chan 1y ago> making it finetuned to your use cases might make it dumber overall. LoRa doesn't overwrite weights.
- pu_pe 1y agoDo you need to overwrite weights to produce the effect I mentioned above?
- koakuma-chan 1y agoGood point
- koakuma-chan 1y agoI think they fine tune them for tool calling, not knowledge