5 ms·

I guess similar to ollama (recently discussed: https://news.ycombinator.com/item?id=36802582 https://news.ycombinator.com/item?id=36802582) which also has suppo

by imrehg 3y ago

I guess similar to ollama (recently discussed: https://news.ycombinator.com/item?id=36802582 https://news.ycombinator.com/item?id=36802582) which also has support for code-focused models (see: https://ollama.ai/library https://ollama.ai/library). I tried pretty much all of them with Continue in VSCode, and it's a bit hit and miss, but the main difference is the way the workflows work (Copilot is mostly line completion, Continue is mostly chat or patches). So the main value add here for me would be a more Copilot-like workflow (which seems to align better with the day-to-day experience I has so far).

fzzzy 3y ago
This is interesting because I started using continue specifically because it doesn't just do tab completion. I have normal Github Copilot turned on again as well now after having it off for a while and the tab completion does seem to be better than it was a few months ago. Still, explicitly asking continue to write a diff has a higher 'this is correct' rate for me because I can describe what I want in detail. Tab Completion seems fine for cases where I add one new line and the next two lines are just incredibly obvious. I am going to experiment with writing a little comment first to see if it primes the tab completion to do something non-obvious.
- imrehg 3y ago
  Maybe it depends on the work people are doing. I am doing a lot of data engineering where I can be often tabbing through while adjusting bits that are off, or doing Python portion by portion, and depending on the complexity it does go from "finish this string I'm writing" to few lines worth of code. I'm often surprised that how quickly the model discovers reasonable patterns (even across files as well, that's often necessary to be correct). With the diffs I find that by the time I describe things in a way it has a chance of working, I might have just written the whole thing myself. Especially as the diffs are often need correctly. With tabs that correction is part of a fast feedback look, with diffs it's so far a slower loop and just more awkward. Of course, with changes to workflows one or the other can shine and if there's an interface that's faster than typing the instructions out that might just supercharge things for the diff/chat type too.
yustee 3y ago
You should check-out https://github.com/smallcloudai/refact https://github.com/smallcloudai/refact. It has both autocomplete and chat. It's in active development, with lots of new features coming soon (context search, fine-tuning for larger models, etc)
- imrehg 3y ago
  It looks like that needs an NVidia GPU which I don't have, or a cloud service, which is out of scope for me. The autocomplete + chat only seem to work on their own model, all the other models currently one or the other. Nonetheles it does look interesting, cheers for the suggestion!
fintechie 3y ago
Have you tried Continue with Phind-CodeLlama-34B-v2? I've been waiting to explore local alternatives to my current go-to tools (Cursor + GPT4) but it seems that local LLMs quality isn't there just yet... ?
- imrehg 3y ago
  I haven't tried that particular one yet, I might not have enough RAM to run it, though, but gonna give it a try. It seems like one of the larger code-focused local models. Cheers for the suggestion!
gardaani 3y ago
Be careful with Continue. It has opt-out telemetry and by default it seems to send all prompts to the developers. Have a look at ~/.continue/telemetry.log to see what is being sent. Don't enter any passwords, credentials or other sensitive data in prompts unless telemetry is disabled.
- behnamoh 3y ago
  I guess I should "dis-continue" using Continue.
  - mycall 3y ago
    or you could change how you manage your secrets and magic strings.
- Nezteb 3y ago
  Oof I'm not a fan of their method of disabling telemetry either: https://github.com/continuedev/continue/issues/567 https://github.com/continuedev/continue/issues/567

by imrehg 3y ago

fzzzy 3y ago

This is interesting because I started using continue specifically because it doesn't just do tab completion. I have normal Github Copilot turned on again as well now after having it off for a while and the tab completion does seem to be better than it was a few months ago. Still, explicitly asking continue to write a diff has a higher 'this is correct' rate for me because I can describe what I want in detail. Tab Completion seems fine for cases where I add one new line and the next two lines are just incredibly obvious. I am going to experiment with writing a little comment first to see if it primes the tab completion to do something non-obvious.

imrehg 3y ago
Maybe it depends on the work people are doing. I am doing a lot of data engineering where I can be often tabbing through while adjusting bits that are off, or doing Python portion by portion, and depending on the complexity it does go from "finish this string I'm writing" to few lines worth of code. I'm often surprised that how quickly the model discovers reasonable patterns (even across files as well, that's often necessary to be correct). With the diffs I find that by the time I describe things in a way it has a chance of working, I might have just written the whole thing myself. Especially as the diffs are often need correctly. With tabs that correction is part of a fast feedback look, with diffs it's so far a slower loop and just more awkward. Of course, with changes to workflows one or the other can shine and if there's an interface that's faster than typing the instructions out that might just supercharge things for the diff/chat type too.

yustee 3y ago

You should check-out https://github.com/smallcloudai/refact https://github.com/smallcloudai/refact. It has both autocomplete and chat. It's in active development, with lots of new features coming soon (context search, fine-tuning for larger models, etc)

imrehg 3y ago
It looks like that needs an NVidia GPU which I don't have, or a cloud service, which is out of scope for me. The autocomplete + chat only seem to work on their own model, all the other models currently one or the other. Nonetheles it does look interesting, cheers for the suggestion!

fintechie 3y ago

Have you tried Continue with Phind-CodeLlama-34B-v2? I've been waiting to explore local alternatives to my current go-to tools (Cursor + GPT4) but it seems that local LLMs quality isn't there just yet... ?

imrehg 3y ago
I haven't tried that particular one yet, I might not have enough RAM to run it, though, but gonna give it a try. It seems like one of the larger code-focused local models. Cheers for the suggestion!

gardaani 3y ago

Be careful with Continue. It has opt-out telemetry and by default it seems to send all prompts to the developers. Have a look at ~/.continue/telemetry.log to see what is being sent. Don't enter any passwords, credentials or other sensitive data in prompts unless telemetry is disabled.

behnamoh 3y ago
I guess I should "dis-continue" using Continue.
- mycall 3y ago
  or you could change how you manage your secrets and magic strings.
Nezteb 3y ago
Oof I'm not a fan of their method of disabling telemetry either: https://github.com/continuedev/continue/issues/567 https://github.com/continuedev/continue/issues/567