8 ms·
The lifecycle of a code AI completion
- mayank 3y agoVery interesting! I wonder to what extent this assumption is true in tying completions to traditional code autocomplete. > One of the biggest constraints on the retrieval implementation is latency If I’m getting a multi line block of code written automagically for me based on comments and the like, I’d personally value quality over latency and be more than happy to wait on a spinner. And I’d also be happy to map separate shortcuts for when I’m prepared to do so (avoiding the need to detect my intent).
- ado__dev 3y agoThis is great feedback and something we are looking at in regards to Cody. We value developer choice and at the moment for Chat developers can choose between various LLM models (Claude 3 Opus, GPT 4-Turbo, Mixtral 8x7b) that offer different benefits. For autocomplete, at the moment we only support Starcoder because it has given us the best return on latency + quality, but we'd def love to support (and give users the choice to set an LLM of their choice, so if they prefer waiting longer for higher quality results, they should be able to) You can do that with our local Ollama support, but that's still experimental and YMMV. Here's how to set it up: https://sourcegraph.com/blog/local-code-completion-with-ollama-and-cody https://sourcegraph.com/blog/local-code-completion-with-olla...
- dsissitka 3y ago> We value developer choice and at the moment for Chat developers can choose between various LLM models (Claude 3 Opus, GPT 4-Turbo, Mixtral 8x7b) that offer different benefits. I wish y'all would put a little more effort into user experience. When you go to subscribe it says: > Claude Instant 1.2, Claude 2, ChatGPT 3.5 Turbo, ChatGPT 4 Turbo Preview Trying to figure out what's supported was tedious enough[0] that I just ended up renewing my Copilot subscription instead. [0] Your contact page for "information about products and purchasing" talks about scheduling a meeting. One of your welcome emails points us to your Discord but then your Discord points us to your forum.
- ado__dev 3y agoThank you for the feedback. I totally agree there and we'll address this.
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- ypeterholmes 3y agoFantastic article and impressive work by this company. They're basically wrapping LLM's with a working memory, and tying it to user input. And thus we step a little closer to AGI/ASI.
- 0x008 3y agoIt’s probably Retrieval Augmented Generation? Just wanted to put this here in case somebody is not familiar with this pattern.
- ForHackernews 3y ago> Congratulations, you just wrote a code completion AI! > In fact, this is pretty much how we started out with Cody autocomplete back in March! Am I wrong in thinking that there's only like 3(?) actual AI companies and everything else is just some frontend to ChatGPT/LLama/Claude? Is this sustainable? I guess the car industry is full of rebadged models with the same engines and chassis. It's just wild that we keep hearing about the AI boom as though there's a vibrant competitive ecosystem and not just Nvidia, a couple of software partners and then a sea of whiteboxers.
- airstrike 3y agoyou've got to differentiate between training, inference and hardware. they all benefit from the "AI boom" but at different levels and have varying levels of substitutability (Google tells me that's a real word)
- lozenge 3y agoI mean... the article goes on to explain all their value add... which of course can be replicated, but it's not as if you can just grab an API key and do the same.
- tayo42 3y agoWhich part do you think you couldn't do on your own with your own api key?
- ec109685 3y agoThey’ve been thinking about the problem continuously for the past year, can run experiments across their user base and thus have a lot more context and insight than someone whipping up an implementation.
- tayo42 3y agowell they wrote a whole blog about how they do it... and im sure this isn't the only one on how to approach this. they also only have a 30% acceptance rate, which is only 10% over their initial attempt with just a prompt.
- 090rf 3y ago[dead]
- Lucasoato 3y agoOne thing I really miss is a standard way to block any copilot/ai code completion tool from reaching specific files. That’s particularly important for .env files, containing sensitive info. We don’t want leaking secrets outside our machine, imagine the risks if they become part of the next training dataset. That’d be really easy to standardize, it’s just another .gitignore-like file.
- eddd-ddde 3y agoWhy not just use .gitignore itself? It's also unlikely they want to train on build files and such.
- myko 3y agoThat makes sense but not everyone uses git. A new file that you can point to your gitignore in would probably be a good idea.
- orhmeh09 3y agoIf you do not use git, you might not be GitHub's target audience.
- fragmede 3y agoThat's clever, but GitHub is recognized as the way to do things, so I've seen a couple repos where the only commits are of binaries to download, which technically uses git, but I wouldn't really say that's using git.
- Atotalnoob 3y agoYou can purchase GitHub copilot and use it (even for corporations) and not use GitHub. My company buys copilot for all devs, but doesn’t use GH or GHE
- telotortium 3y ago
- Exuma 3y agohow does this compare to copilot? i would be willing to switch (or try it at least) if it can give better experience inside neovim
- phillipcarter 3y agoCody is very good. I recommend giving it a try. FWIW I believe Cody is much more actively developed than Copilot is these days, and so it has a more comprehensive feature set.
- ado__dev 3y agoI wrote a blog post comparing Cody to Copilot a little while ago. Some of the stuff might be outdated now, but I think it still captures the essence of the differences between the two. Obviously I'm a little biased as I work for Sourcegraph, but I tried to be as fair as one could be. Happy to dive deeper into any details. https://sourcegraph.com/blog/copilot-vs-cody-why-context-matters-for-code-ai https://sourcegraph.com/blog/copilot-vs-cody-why-context-mat... Our biggest differentiators are context, choice, and scale. We've been helping developers find and understand code for the last 10 years and are now applying a lot of that to Cody in regards to fetching the right context. When it comes choice, we support multiple LLMs and are always on the lookout in supporting the right LLM for the job. We recently rolled out Claude 3 Opus as well as Ollama support for offline/local inference. Cody also has a free tier where you can give it a try and compare for yourself, which is what I always recommend people do :) On Neovim, Cody actually does have experimental support for neovim: https://sourcegraph.com/docs/cody/clients/install-neovim https://sourcegraph.com/docs/cody/clients/install-neovim. Not all features are supported as in VS Code though.
- nicklecompte 3y ago> Because of the language-specific nature of this heuristic, we generally do not support multi-line completions for all languages. However, we’re always happy to extend our list of supported languages and, since Cody is open-source, you can also contribute and improve the list. (link to https://github.com/sourcegraph/cody/blob/main/vscode/src/completions/language.ts https://github.com/sourcegraph/cody/blob/main/vscode/src/com...) That link to the list of supported languages is broken. I couldn't find a similar file elsewhere in the repo: maybe the list got folded up into a function in another file? Also a bit annoying that I couldn't find the info on the company's website (though I gave up pretty quickly).
- friendlynokill 3y agoThis should be the correct link: https://github.com/sourcegraph/cody/blob/main/vscode/src/graph/lsp/languages.ts https://github.com/sourcegraph/cody/blob/main/vscode/src/gra...
- nicklecompte 3y agoI don't think it is. There is a test file which includes C#, Kotlin, etc among supported languages, and those aren't included in the file you linked: https://github.com/sourcegraph/cody/blob/main/vscode/src/completions/get-inline-completions-tests/languages.test.ts https://github.com/sourcegraph/cody/blob/main/vscode/src/com... But this test didn't seem to include TypeScript so it's obviously not comprehensive. I'm not convinced this information is actually in one place.
- sqs 3y agoThe list of supported languages is at https://sourcegraph.com/docs/cody/faq#what-programming-languages-does-cody-support https://sourcegraph.com/docs/cody/faq#what-programming-langu.... It works for all programming languages, but it works better on some than others, depending on the LLM and the language-specific work we've done on context-fetching, syntax-related heuristics, etc. On the LLM point, Cody supports multiple LLMs for both chat and autocomplete (Claude 3, GPT-4 Turbo, Mixtral, StarCoder, etc.), which is great for users but makes it tough to give any formal definition of "supported language".
- yaohui-wyh 3y agoThis article has been around for some time, and it still shines. It focuses on building a product rather than just prototyping an LLM wrapper and waiting for the dark magic of GenAI. Chapters like "The Strive for Faster Latencies" and "Post-Processing" are truly inspiring. Creating production-level DevTools demands much more effort than merely wrapping around a ChatCompletion endpoint and mindlessly stuffing a context window with everything accessible inside the IDE (so-called "prompt engineering").
- metadat 3y ago> This article has been around for some time, .. Interesting, what gave you this impression? This article was first published only 4-6 months ago around Oct 2023. Don't get me wrong, I'm a fan of sourgraph and their founder, Quinn, is quite charismatic. I almost went for a job offer with them 5+ years ago. But let's be real, startups are not a winning game for a non-founder, thank goodness I stuck it out with BigCorp. In any case, this isn't that old of a post, cheers.
- swores 3y agoThey seem to have got the correct impression that it's been "around for some time" as "some time" does not mean "a very long time" and you just confirmed that it indeed isn't a just-published article but one which, coming out last year, has been around "for some time". I'm guessing you're just reading into the phrase an implication that isn't there about it being particularly old.
- tom_ 3y ago"Some time" does have pretty much that implication. The exact amount of time is unspecified, but I'm unsure that 6 months is enough to count. Maybe this is an American vs British English thing? You dan't have to trust me: https://dictionary.cambridge.org/dictionary/english/for-some-time https://dictionary.cambridge.org/dictionary/english/for-some...
- 3y ago
- jerrygoyal 3y agohas anyone tried Cody and Github Copilot and compared? I'm using GitHub Copilot and wouldn't mind switching to a better alternative.
- ado__dev 3y ago(I left this comment earlier but I'll c+p here as well) I wrote a blog post comparing Cody to Copilot a little while ago. Some of the stuff might be outdated now, but I think it still captures the essence of the differences between the two. Obviously I'm a little biased as I work for Sourcegraph, but I tried to be as fair as one could be. Happy to dive deeper into any details. https://sourcegraph.com/blog/copilot-vs-cody-why-context-mat https://sourcegraph.com/blog/copilot-vs-cody-why-context-mat... Our biggest differentiators are context, choice, and scale. We've been helping developers find and understand code for the last 10 years and are now applying a lot of that to Cody in regards to fetching the right context. When it comes choice, we support multiple LLMs and are always on the lookout in supporting the right LLM for the job. We recently rolled out Claude 3 Opus as well as Ollama support for offline/local inference. Cody also has a free tier where you can give it a try and compare for yourself, which is what I always recommend people do :)
- hobo_in_library 3y agothat url got cut off and 404s (prob copy/pasta from another comment) Correct url: https://sourcegraph.com/blog/copilot-vs-cody-why-context-matters-for-code-ai https://sourcegraph.com/blog/copilot-vs-cody-why-context-mat...
- ado__dev 3y agoThank you for that. I copied an earlier comment and I guess the URL got cut off.
- veselin 3y agoI used them both. I ended up disabling copilot. The reason is that the completions do not always integrate with the rest of the code, in particular with non-matching brackets. Often it just repeats some other part of the code. I had much fewer cases of this with Cody. But, arguably, the difference is not huge. But then add on top of this choice of models.