4 ms·

The list of supported languages is at https://sourcegraph.com/docs/cody/faq#what-programming-languages-does-cody-support https://sourcegraph.com/docs/cody/faq#w

by sqs 3y ago

The list of supported languages is at https://sourcegraph.com/docs/cody/faq#what-programming-languages-does-cody-support https://sourcegraph.com/docs/cody/faq#what-programming-langu.... It works for all programming languages, but it works better on some than others, depending on the LLM and the language-specific work we've done on context-fetching, syntax-related heuristics, etc. On the LLM point, Cody supports multiple LLMs for both chat and autocomplete (Claude 3, GPT-4 Turbo, Mixtral, StarCoder, etc.), which is great for users but makes it tough to give any formal definition of "supported language".

nicklecompte 3y ago
Even that link doesn't claim to be comprehensive, including an extremely vague "shell scripting languages (like Bash, PowerShell)," as if I am supposed to cross my fingers and hope this means a nonstandard shell like fish works, when it seems like that could introduce weird or inscrutable bugs coming from incompatible bash syntax. (One under-discussed downside of LLM code generation is that it further penalizes innovation in programming languages.) What I find frustrating is that Cody is doing deterministic language-specific tricks that don't depend on the underlying LLM, so why not just give a list of all the languages you did deterministic tricks for and call that the supported languages? Why be vague? Why deflect to the underlying LLM to encourage people to try your product with languages you won't do any work to support? Claiming it works on "all programming languages" is lazy and dishonest, and clearly false. There's nothing magical about LLMs that relieves software developers of the duty to specify the limitations of their software.
- sqs 3y ago
  The code is Apache 2 and we linked it from the blog post. You can see all of the tricks! Nothing here is dishonest. We’re open about the limitations of the LLM being the main determining factor, and the numerous languages we’ve prioritized making it work well on. I see your point about not mentioning “etc.” in the shell scripting parenthetical. Will update.

by sqs 3y ago

nicklecompte 3y ago

Even that link doesn't claim to be comprehensive, including an extremely vague "shell scripting languages (like Bash, PowerShell)," as if I am supposed to cross my fingers and hope this means a nonstandard shell like fish works, when it seems like that could introduce weird or inscrutable bugs coming from incompatible bash syntax. (One under-discussed downside of LLM code generation is that it further penalizes innovation in programming languages.) What I find frustrating is that Cody is doing deterministic language-specific tricks that don't depend on the underlying LLM, so why not just give a list of all the languages you did deterministic tricks for and call that the supported languages? Why be vague? Why deflect to the underlying LLM to encourage people to try your product with languages you won't do any work to support? Claiming it works on "all programming languages" is lazy and dishonest, and clearly false. There's nothing magical about LLMs that relieves software developers of the duty to specify the limitations of their software.

sqs 3y ago
The code is Apache 2 and we linked it from the blog post. You can see all of the tricks! Nothing here is dishonest. We’re open about the limitations of the LLM being the main determining factor, and the numerous languages we’ve prioritized making it work well on. I see your point about not mentioning “etc.” in the shell scripting parenthetical. Will update.