6 ms·

Show HN: Codeium – a free, fast AI codegen extension

86 points by varunkmohan 4y ago

I'm Varun, CEO of Exafunction, and we just released Codeium to open up access of generative AI to all developers for free. In the spirit of Show HN, we created a playground version for anyone to try this tech in the browser (click Try in Browser)! We have built scalable, low-latency ML infra for many top AI companies in the past, and we are excited to leverage that tech into a product that we, as developers, would love. We hope that you do too, and we would appreciate any feedback that this community has for us!

khou22 4y ago
Been using this for the past couple of weeks and can attest it's been game changer for my productivity. Was skeptical at first, but I find myself less on Google search and more in flow. Thanks for giving it away for free
- culi 4y ago
  Have you tried Copilot or CodeWhisperer previously?
  - khou22 4y ago
    Yeah I used Copilot for a few weeks before since it was better known. Ended up cancelling my Copilot because I didn't notice a huge difference in quality/features and Codeium is free (primary motivation tbh)
Micoloth 4y ago
I'll go ahead and ask the obvious question.. What's the difference with Copilot? In my exactly 3 minutes test, it did worse, even if only slightly. Right now it is free, but that won't last, right? Do i have to expect any shady data collection practices? More interestingly, how is this trained? Have you guys done your own finetuning of gpt3 or it's something completely different?
- varunkmohan 4y ago
  Right now, we are providing a free alternative to Copilot and expanding the set of features we support. We are actively working on improving the quality of suggestions and would appreciate feedback. We guarantee that all users that join now will be able to use the product free forever. We explicitly allow users to opt out of sending data to us post inference. We believe we're in the early innings of code assistants and would like to democratize the technology so we don't have plans to monetize this tier of usage. The model is hosted internally and trained on public code. For more details on the specifics, please take a look at our FAQ page on the website.
  - Micoloth 4y ago
    Cool.. Thanks for your answer I have to say, in my quick tests completions do arrive slightly but noticeably faster than Copilot's.. Which is enough to make this interesting! Unless you are using a much smaller model, in which case it's too easy ahah. Thanks and good luck for your further plans, I'll keep trying this
    - varunkmohan 4y ago
      Thanks! We've made a lot of optimizations internally for latency. We've gotten a lot of feedback that it is faster than Copilot but obviously they operate at much larger scale.
  - a1371 4y ago
    Say this on your landing page because it's a good differentiator, but it's not clear
    - varunkmohan 4y ago
      Thanks for the feedback!
- culi 4y ago
  > In my exactly 3 minutes test, it did worse, even if only slightly. What exactly does this mean?
gigatexal 4y ago
I am dubious to use this at work since I can’t be sure it won’t send code from the codebase I am working on to the cloud. When something is free you’re usually the product.
- varunkmohan 4y ago
  We understand the concern! We describe our security policies in detail on our website, but we offer users the option to opt out from code snippet telemetry. This means after the inference, we do not store any code-based information on our end. We also know that you might not take our word for it, which is why we are working on getting SOC2 attestation for these policies. Also, to answer your pricing point, we are actively working on additional features beyond code completion. We want to try to make the code completion itself to be free, and definitely free forever for early users as a thank you for using Codeium from early on.
  - gigatexal 4y ago
    Well that’s good. Let me pay for it, heck let me host it so I am sure. But getting audited is a good first step.
    - varunkmohan 4y ago
      We're actively thinking of ways where we host the service within a customer's tenant - more coming soon!
      - gigatexal 4y ago
        Amazing! Looking forward to it!
maille 4y ago
Alternatives to Copilot are more than welcomed. Do you plan to offer a plugin for MSVC ?
- varunkmohan 4y ago
  We have gotten requests for more IDEs, and are actively working on creating support for them so that as many developers as possible can leverage this technology!
culi 4y ago
Cool. Well I guess until there's some community-driven open-source alternative I'll be rooting for y'all against Copilot and CodeWhisperer. Maybe you can pick up some of the Kite talent to join the effort
- varunkmohan 4y ago
  Thanks for the support! We are actively looking to hire experts in program synthesis and ML infrastructure to help deliver this technology at scale
swyx 4y ago
> scalable, low-latency ML infra I'm really really curious as to the technical details of getting the inferences cheap and fast enough to enable a copilot like experience when you're not GitHub with inifinity resources. Sam Altman has said that each chat in chatgpt is single digit cents per chat. GPT3 davinci is 0.02 per 1k tokens, not to mention the high latency. it'd be impossible to build "copilot for law" based on gpt3 because the UX wouldnt work with that latency or cost (you'd be running 100s of inferences if you count every debounced keystroke). What cost per user/per inference/per day (however you think about it) do you need to get this down to? and what order of magnitude speedups did you need to accomplish to get this done? (congrats btw, happy user here thanks to Prem)
- pqn 4y ago
  Hey swyx :) Great question, we've got a blog post coming soon with some of these technical and other details... we've employed a lot of tricks here, debouncing included.
  - swyx 4y ago
    inject this straight into my veins
- mhuffman 4y ago
  I have a ML infra that does between 1 million and 2 million inferences per day (granted it is not a chat bot or code generator, but it is legit ML inference in a similar mode) and it only cost $120 per month for the server (also runs multiple containers and is a web server), which mostly sits idle.
  - swyx 4y ago
    right, but what size of model? i assume these large language models are... well, large
fcoury 4y ago
I can't get it to work. Here's what I am getting: 2022/12/06 18:55:36 api_server_client.go:120: Completion error: rpc error: code = Unavailable desc = HTTP Logical service in fail-fast 2022-12-06 18:55:36.849 [INFO]: Completion request errored (236.63ms) 2022-12-06 18:55:36.849 [ERROR]: 14 UNAVAILABLE: HTTP Logical service in fail-fast
- varunkmohan 4y ago
  Please join our Discord (https://discord.gg/3XFf78nAx5 https://discord.gg/3XFf78nAx5) and we are happy to help debug this for you!
  - pmayrgundter 4y ago
    Same.. web interface works but neither the plugin or discord link (here or on your site) are working for me. EDIT: clicking on the "return to vs code" link from your site was needed. now auth'd
zanussbaum 4y ago
Has been a huge boost over using Copilot. I accidentally was using Copilot instead of Codeium and was confused why the generations took so long until I realized! Great product
monkeydust 4y ago
Any plans to allow fine-tuning on custom codebase?
- varunkmohan 4y ago
  We are currently working on making suggestions more tailored to your codebase without having to persist / upload the codebase. We will have updates on this shortly!
waldenyan20 4y ago
Amazingly fast! Would love to see the suggestions be a little better but it seems like a great team working on this.
Cilvic 4y ago
I briefly tried it and it was indeed very fast, but I couldn't get simple things for python. It would just keep adding import statements or try to continue writing my comment (even though I wanted the code that matches the comment). Also I couldn't find any way to toggle alternative suggestions.
- fortenforge 4y ago
  If you start the function you're trying to write (just by writing `def` on a new line), you should usually be able to get it to start implementing it. You can see alternate suggestions on VSCode by hitting Option + [. We don't currently support seeing alternate suggestions on JetBrains IDEs, but we're working on it!
solarkraft 4y ago
The name is very close to [VS]Codium, the open source version of VSCode.
vanilla_whey 4y ago
The VSCode Plugin works like a charm! Any plans to support vim/neovim in the future?
- varunkmohan 4y ago
  Yes, that's on the list of plugins to create. We're keeping folks updated on the timelines on our Discord.

86 points by varunkmohan 4y ago

khou22 4y ago

Been using this for the past couple of weeks and can attest it's been game changer for my productivity. Was skeptical at first, but I find myself less on Google search and more in flow. Thanks for giving it away for free

culi 4y ago
Have you tried Copilot or CodeWhisperer previously?
- khou22 4y ago
  Yeah I used Copilot for a few weeks before since it was better known. Ended up cancelling my Copilot because I didn't notice a huge difference in quality/features and Codeium is free (primary motivation tbh)

Micoloth 4y ago

I'll go ahead and ask the obvious question.. What's the difference with Copilot? In my exactly 3 minutes test, it did worse, even if only slightly. Right now it is free, but that won't last, right? Do i have to expect any shady data collection practices? More interestingly, how is this trained? Have you guys done your own finetuning of gpt3 or it's something completely different?

varunkmohan 4y ago
Right now, we are providing a free alternative to Copilot and expanding the set of features we support. We are actively working on improving the quality of suggestions and would appreciate feedback. We guarantee that all users that join now will be able to use the product free forever. We explicitly allow users to opt out of sending data to us post inference. We believe we're in the early innings of code assistants and would like to democratize the technology so we don't have plans to monetize this tier of usage. The model is hosted internally and trained on public code. For more details on the specifics, please take a look at our FAQ page on the website.
- Micoloth 4y ago
  Cool.. Thanks for your answer I have to say, in my quick tests completions do arrive slightly but noticeably faster than Copilot's.. Which is enough to make this interesting! Unless you are using a much smaller model, in which case it's too easy ahah. Thanks and good luck for your further plans, I'll keep trying this
  - varunkmohan 4y ago
    Thanks! We've made a lot of optimizations internally for latency. We've gotten a lot of feedback that it is faster than Copilot but obviously they operate at much larger scale.
- a1371 4y ago
  Say this on your landing page because it's a good differentiator, but it's not clear
  - varunkmohan 4y ago
    Thanks for the feedback!
culi 4y ago
> In my exactly 3 minutes test, it did worse, even if only slightly. What exactly does this mean?

gigatexal 4y ago

I am dubious to use this at work since I can’t be sure it won’t send code from the codebase I am working on to the cloud. When something is free you’re usually the product.

varunkmohan 4y ago
We understand the concern! We describe our security policies in detail on our website, but we offer users the option to opt out from code snippet telemetry. This means after the inference, we do not store any code-based information on our end. We also know that you might not take our word for it, which is why we are working on getting SOC2 attestation for these policies. Also, to answer your pricing point, we are actively working on additional features beyond code completion. We want to try to make the code completion itself to be free, and definitely free forever for early users as a thank you for using Codeium from early on.
- gigatexal 4y ago
  Well that’s good. Let me pay for it, heck let me host it so I am sure. But getting audited is a good first step.
  - varunkmohan 4y ago
    We're actively thinking of ways where we host the service within a customer's tenant - more coming soon!
    - gigatexal 4y ago
      Amazing! Looking forward to it!

maille 4y ago

Alternatives to Copilot are more than welcomed. Do you plan to offer a plugin for MSVC ?

varunkmohan 4y ago
We have gotten requests for more IDEs, and are actively working on creating support for them so that as many developers as possible can leverage this technology!

culi 4y ago

Cool. Well I guess until there's some community-driven open-source alternative I'll be rooting for y'all against Copilot and CodeWhisperer. Maybe you can pick up some of the Kite talent to join the effort

varunkmohan 4y ago
Thanks for the support! We are actively looking to hire experts in program synthesis and ML infrastructure to help deliver this technology at scale

swyx 4y ago

> scalable, low-latency ML infra I'm really really curious as to the technical details of getting the inferences cheap and fast enough to enable a copilot like experience when you're not GitHub with inifinity resources. Sam Altman has said that each chat in chatgpt is single digit cents per chat. GPT3 davinci is 0.02 per 1k tokens, not to mention the high latency. it'd be impossible to build "copilot for law" based on gpt3 because the UX wouldnt work with that latency or cost (you'd be running 100s of inferences if you count every debounced keystroke). What cost per user/per inference/per day (however you think about it) do you need to get this down to? and what order of magnitude speedups did you need to accomplish to get this done? (congrats btw, happy user here thanks to Prem)

pqn 4y ago
Hey swyx :) Great question, we've got a blog post coming soon with some of these technical and other details... we've employed a lot of tricks here, debouncing included.
- swyx 4y ago
  inject this straight into my veins
mhuffman 4y ago
I have a ML infra that does between 1 million and 2 million inferences per day (granted it is not a chat bot or code generator, but it is legit ML inference in a similar mode) and it only cost $120 per month for the server (also runs multiple containers and is a web server), which mostly sits idle.
- swyx 4y ago
  right, but what size of model? i assume these large language models are... well, large

fcoury 4y ago

I can't get it to work. Here's what I am getting: 2022/12/06 18:55:36 api_server_client.go:120: Completion error: rpc error: code = Unavailable desc = HTTP Logical service in fail-fast 2022-12-06 18:55:36.849 [INFO]: Completion request errored (236.63ms) 2022-12-06 18:55:36.849 [ERROR]: 14 UNAVAILABLE: HTTP Logical service in fail-fast

varunkmohan 4y ago
Please join our Discord (https://discord.gg/3XFf78nAx5 https://discord.gg/3XFf78nAx5) and we are happy to help debug this for you!
- pmayrgundter 4y ago
  Same.. web interface works but neither the plugin or discord link (here or on your site) are working for me. EDIT: clicking on the "return to vs code" link from your site was needed. now auth'd

zanussbaum 4y ago

Has been a huge boost over using Copilot. I accidentally was using Copilot instead of Codeium and was confused why the generations took so long until I realized! Great product

monkeydust 4y ago

Any plans to allow fine-tuning on custom codebase?

varunkmohan 4y ago
We are currently working on making suggestions more tailored to your codebase without having to persist / upload the codebase. We will have updates on this shortly!

waldenyan20 4y ago

Amazingly fast! Would love to see the suggestions be a little better but it seems like a great team working on this.

Cilvic 4y ago

I briefly tried it and it was indeed very fast, but I couldn't get simple things for python. It would just keep adding import statements or try to continue writing my comment (even though I wanted the code that matches the comment). Also I couldn't find any way to toggle alternative suggestions.

fortenforge 4y ago
If you start the function you're trying to write (just by writing `def` on a new line), you should usually be able to get it to start implementing it. You can see alternate suggestions on VSCode by hitting Option + [. We don't currently support seeing alternate suggestions on JetBrains IDEs, but we're working on it!

solarkraft 4y ago

The name is very close to [VS]Codium, the open source version of VSCode.

vanilla_whey 4y ago

The VSCode Plugin works like a charm! Any plans to support vim/neovim in the future?

varunkmohan 4y ago
Yes, that's on the list of plugins to create. We're keeping folks updated on the timelines on our Discord.