4 ms·
Google CodeGemma: Open Code Models Based on Gemma [pdf]
- tosh 3y agoIt's really sad that Cursor does not support local models yet (afaiu they fetch the URL you provide from their server). Is there a VS Code plugin or other editor that does? With models like CodeGemma and Command-R+ it makes more and more sense to run them locally.
- sp332 3y agoAccording to https://forum.cursor.sh/t/support-local-llms/1099/7 https://forum.cursor.sh/t/support-local-llms/1099/7 the Cursor servers do a lot of work in between your local computer and the model. So porting all that to work on users' laptops is going to take a while.
- ericskiff 3y agoI've been playing with Continue: https://github.com/continuedev/continue https://github.com/continuedev/continue
- tosh 3y agoty for the pointer!
- tosh 3y agoAlready on ollama: https://ollama.com/library/codegemma https://ollama.com/library/codegemma
- ado__dev 3y agoCody supports local inference with Ollama for both Chat and Autocomplete. Here's how to set it up: https://sourcegraph.com/blog/local-chat-with-ollama-and-cody https://sourcegraph.com/blog/local-chat-with-ollama-and-cody :)
- tosh 3y agoty for the pointer!
- knoopx 3y agohttps://github.com/huggingface/llm-vscode https://github.com/huggingface/llm-vscode "llm.backend": "ollama", "llm.url": "http://localhost:11434/api/generate", "llm.modelId": "codegemma:2b-code-q8_0", "llm.configTemplate": "Custom", "llm.fillInTheMiddle.enabled": true, "llm.fillInTheMiddle.prefix": "<|fim_prefix|>", "llm.fillInTheMiddle.middle": "<|fim_middle|>", "llm.fillInTheMiddle.suffix": "<|fim_suffix|>", "llm.tokensToClear": ["<|fim_prefix|>", "<|fim_middle|>", "<|fim_suffix|>", "<|file_separator|>"],
- sheepscreek 3y agoDownload the model weights here (PyTorch, GGUF): https://huggingface.co/collections/google/codegemma-release-66152ac7b683e2667abdee11 https://huggingface.co/collections/google/codegemma-release-... I am really liking the Gemma line of models. Thoroughly impressed with the 2B and 7B non-code optimized variants. The 2B especially packs a lot of punch. I reckon its quality must be at par with some older 7B models, and it runs blazing fast on Apple Silicon - even at 8 bit quantization.
- tosh 3y agoGemma 2b instruct worked well for me for categorization. I would also say it felt 7b-ish. Very impressed. The initial release left me a bit underwhelmed but 1.1 is better and punches above its weight. Also looking fwd to use 2b models on iOS and Android (even if they will be heavy on the battery).
- kolbe 3y agoMy issue so far with the various code assistants isn't the quality necessarily, but the ability of them to draw in context from the rest of the code base without breaking the bank or proving so much info that the middle gets ignored. Are there any systems doing that well these days?
- grey8 3y agoIf I'm not mistaken, this is not on the models itself, but rather on the implementation of the addon. I haven't found an open source VSCode or WebStorm addon yet that allows me to use a local model and implements code completion and commands as good as GitHub Copilot. They either miss a chat feature and/or inline action / code completion and/or fill-in-the-middle models. And if they do, they don't provide the context as intelligently (? an assumption!) as GH's Copilot does. One alternative I liked was Supermaven: It's really really fast and has a huge context window, so it knows almost your whole project. That was nice! But - one thing I ultimately didn't continue using it for: It doesn't support chat and/or inline commands (CTRL+I on VSCode's GH Copilot). I feel like a really good Copilot alternative is definitely a still missing. But: Regarding your question, I think GitHub Copilot's VSCode extension is the best - as of now. The WebStorm extension is sadly not as good, it misses the "inline command" function which IMHO is a must.
- skybrian 3y agoCould you use one tool for code completion and another for chat?
- evilduck 3y agoContinue.dev allows for this. You can even mix hosted Chat options like GPT-4 (via API) with local completion. I typically use a smaller model for faster text completion and a larger model (with a bigger context) for chat. https://github.com/continuedev/continue https://github.com/continuedev/continue
- mediaman 3y agoI think most of them allow for that. Works in vscode and vscode-derived (e.g., cursor) editors.
- typpo 3y agoIf anyone wants to eval this locally versus codellama, it's pretty easy with Ollama[0] and Promptfoo[1]: prompts: - "Solve in Python: {{ask}}" providers: - ollama:chat:codellama:7b - ollama:chat:codegemma:instruct tests: - vars: ask: function to return the nth number in fibonacci sequence - vars: ask: convert roman numeral to number # ... YMMV based on your coding tasks, but I notice gemma is much less verbose by default. [0] https://github.com/ollama/ollama https://github.com/ollama/ollama [1] https://github.com/promptfoo/promptfoo https://github.com/promptfoo/promptfoo
- ramzanhanif4353 3y ago[flagged]
- danielhanchen 3y agoMade Code Gemma 7b 2.4x faster and use 68% less VRAM with Unsloth if anyone wants to finetune it! :) Have a Tesla T4 Colab notebook with ChatML: https://colab.research.google.com/drive/19lwcRk_ZQ_ZtX-qzFP3qZBBHZNcMD1hh?usp=sharing https://colab.research.google.com/drive/19lwcRk_ZQ_ZtX-qzFP3...
- trisfromgoogle 3y agoLove to see your work, Daniel -- thank you, as always! Playing with the Colab now =). Go Unsloth, and thanks from the Gemma team!
- danielhanchen 3y agoThanks! :) Appreciate it a lot! :)
- zexodus 3y agoIt performed surprisingly well at categorization tasks where json results where involved compared to gemma:2b and gemma:7b