3 ms·
Enhancing Code Completion for Rust in Cody
- shahahmed 2y agoarguably you can reduce even more latency by keeping the model on-device as well, but that would mean revealing the weights of the fine-tuned model. If the user preferred reduced latency and had the RAM, is that an option?
- s1mplicissimus 2y agothe model is probably most of the "secret sauce" of cody, so if they gave that away people could just copy it around like mp3s. my guess
- morgante 2y agoCompletely incorrect, as Sourcegraph has not historically trained models and Cody swaps between many open source and 3rd party models.
- rdedev 2y agoLooking at their GitHub page or seems like they are using existing LLM services. It should be possible to modify cody to make it work with a local llm
- ado__dev 2y agoYou don’t have to modify anything. We support local LLMs for chat and completions with Ollama. https://sourcegraph.com/blog/local-code-completion-with-ollama-and-cody https://sourcegraph.com/blog/local-code-completion-with-olla...
- daemonologist 2y agoThis is true, but only if you have a GPU (/accelerator) comparable in performance to the one backing the service, or at least comparable after accounting for the local benefit. This is an expensive proposition because it will be sitting idle between completions and when you're not coding.
- sqs 2y agoNot for this fine-tuned model yet, but Cody supports local models: https://sourcegraph.com/docs/cody/clients/install-vscode#supported-local-ollama-models-with-cody https://sourcegraph.com/docs/cody/clients/install-vscode#sup.... I just used Cody with Ollama for local inference on a flight where the wifi was broken, and it never fails to blow my mind: https://x.com/sqs/status/1803269013310759236 https://x.com/sqs/status/1803269013310759236.
- sqs 2y agoAwesome to see our Rust fine-tune here on HN. We (Sourcegraph team) are here if anyone has questions or ideas for us! BTW, Cody is open source (Apache 2): https://github.com/sourcegraph/cody https://github.com/sourcegraph/cody.
- IshKebab 2y agoYour Cody page doesn't answer a very obvious question: does the LLM run locally or is this going to send all my code to Sourcegraph? I assume that is a deliberate omission and the answer is the latter.
- jdorfman 2y agoThe default LLM is Claude 3 Sonnet hosted by Anthropic. However you can run local models via Ollama: https://sourcegraph.com/docs/cody/clients/install-vscode#supported-local-ollama-models-with-cody https://sourcegraph.com/docs/cody/clients/install-vscode#sup...
- tamimio 2y agoThank you, cody.dev kept giving me a white blank page.
- mutematrix 2y agoSorry about that! I just looked into this (also on the Sourcegraph team). If you get a chance, could you check if it's working now?
- riquito 2y agoJust wanted to thank you, is a great product. I find it to have a better context and awareness of my code compared to copilot
- ekidd 2y agoFrom the article: > Although most LLMs are trained on corpora that include several programming languages, we often observe differential performance across languages, especially languages like Rust that are not well represented in popular training datasets. Interesting! My experience with Rust and Copilot is that Rust is actually a relatively strong language for Copilot, in many ways. I suspect that this might be a mix of different things: - Is the training set code for Rust relatively high-quality? - Do strongly-typed languages have a better chance of catching incorrect completions quickly?
- cageface 2y agoMy very limited experience with GPT and Rust is that the code it generates rarely satisfies the borrow checker out of the box. I’ve had much better luck with garbage collected languages.
- ekidd 2y agoInteresting. I find that Copilot is quite good with the borrow checker, at least if it can "see" the signature of the function that it's calling. Sometimes it inserts an extra "&" where it really shouldn't. That's a no-op, clippy catches it, and I take it out. But I tend to write my code in a style that easily passes the borrow checker 99% of the time. I keep mutation very local, and rely on a more "functional" architecture. So Copilot is naturally going to write new code in the same borrow-checker-friendly style, which may keep it out of trouble. When using ChatGPT, I've actually had more luck asking it to write Python, and then to translate the Python into Rust. ChatGPT seems to be better at algorithms in Python, and it's surprisingly good (in my personal experience) at inserting the fine details of Rust types and references when translating.
- cageface 2y agoDoes your more functional style involve making a lot of copies?
- MarkMarine 2y agoI’ve been using Cody and liking it, but one thing I noticed is there is no loop that even takes the data from the IDE and incorporates the IDE’s feedback into the code it generates. It’ll happily generate code the IDE draws red wiggle lines under and just continue on with its day. Would be nice to see a tighter integration
- sqs 2y agoYeah, that is needed to make everything more accurate. We are working on 3 related things to make Cody better understand the structure of your code and take feedback from the compiler (and other tools). All are experimental and not enabled by default, but you can peek at them if you're interested. 1. (experimental) LSP-based context — https://github.com/sourcegraph/cody/pull/3493 https://github.com/sourcegraph/cody/pull/3493 2. (experimental) TypeScript tsc context — https://github.com/sourcegraph/cody/pull/3843 https://github.com/sourcegraph/cody/pull/3843 3. (experimental) OpenCtx, which can pull in diagnostics from tools outside your editor — https://openctx.org/ https://openctx.org/ And then of course the agent to actually iterate on the generated code based on this feedback. We're really excited about this stuff, and it'll arrive in Cody when we can get it to a high quality bar and prove it helps.
- MarkMarine 2y agoAwesome! I’ll take a look
- esafak 2y agoCody is decent. It's my copilot of choice at the moment. It indexes your code base so it can give contextual responses.
- sqs 2y agoWhat would make it better than decent for you?
- esafak 2y ago1. Write the commit messages too. 2. Allow the user to refactor across selected portions of the entire project, not just one file.
- sqs 2y agoFor 1, set `"cody.experimental.commitMessage": true` in VS Code and then click the Cody icon above the commit message box in VS Code's source control sidebar. Let me know how that works for you (here or at https://community.sourcegraph.com https://community.sourcegraph.com). For 2, we're working on it! And eventually we'll support cross-repository refactors in Cody using our batch changes (https://sourcegraph.com/docs/batch-changes https://sourcegraph.com/docs/batch-changes). What kind of refactors do you want to make?
- esafak 2y ago1. I use Jetbrains. I don't suppose experimental flags are supported there too? 2. A reasonable near-term goal could be to make changes in one file propagate across the code base, so it remains consistent. This requires analyzing the call graph, which is something Sourcegraph already knows how to do. The next step could be to execute multi-file refactors without specifying which files to modify; e.g., "add support for WebAuthn using our auth provider's SDK".
- sqs 2y ago1. Ah, this commit message feature isn't available in JetBrains yet, unfortunately. Most things carry over since they both use the same underlying code, but this is more editor-specific. 2. Your "reasonable near-term goal" is right in line with what we are thinking. Basically, "auto-edits" is f(edit that meets deterministic trigger criteria) --> multi-file diff. Here are some examples we have in mind; any others you had in mind? update a type signature --> ripple that change across to all call sites/value literals (kinda like symbolic rename does today, but for more than just the name) rename something --> update the docs change code with a comment nearby --> update the comment if it's now invalid update a func impl --> propose a new test case