3 ms·
Slightly related question: what's a good coding LLM to run on a 4070 12GB card? Also, do coding LLMs use treesitter to "understand" code?
by bloopernova 3y ago
Slightly related question: what's a good coding LLM to run on a 4070 12GB card?
Also, do coding LLMs use treesitter to "understand" code?
- sob727 3y agoI'm also interested in the answer to that.
- explorigin 3y agoDepends on what you want to use it for. I use deepseek-coder v1 (1.5 is too verbose). I use it like a customized web search to quickly build one-off scripts in python. If you're wanting something to be your hands so you don't have to type, open-source LLMs and IDE integration is not reliably there yet. Follow the AIDER discord to stay up on the latest in this area.
- ianschmitz 3y agoI’m pretty new to running these locally, but here’s my understanding: Best models currently: codellama or deepseek coder. 6.7B or 1B depending on how much latency you can tolerate Treesittee: from looking at the logs of the chat completions requests for Continue or Twinny extensions for VS Code, they both appear to just send a chunk of the document as well as a special placeholder to indicate where the cursor currently is.
- wiradikusuma 3y agoAnother one is https://github.com/WisdomShell/codeshell/blob/main/README_EN.md https://github.com/WisdomShell/codeshell/blob/main/README_EN... and it has its own IntelliJ plugin.
- Havoc 3y agoTry deepseek 6.7B
- fancy_pantser 3y ago> do coding LLMs use treesitter...? It's up to the app to put that into the context. Generally, coding LLMs do well if you provide them the source tree, graph, search results, notable files, etc in the context. The is how Sourcegraph's Cody product works, for example.