3 ms·

I kinda just want something that can keep up with the original version of Copilot. It was so much better than the crap they’re pumping out now (keeps messing up

I kinda just want something that can keep up with the original version of Copilot. It was so much better than the crap they’re pumping out now (keeps messing up syntax and only completing a few characters at a time).

heeton 2y ago
Have you tried supermaven? It replaced copilot for me a couple of months ago.
- karolist 2y ago
  I tried it, uses GPT-4o, the $10 sign up credit dissapeared in a few hours of intense coding, I'm not paying $500/mo for a fancy autocomple. Manual instruct style chat about code with Claude-Sonnet-3.5 is the best price/perf I've tried so far, through poe.com I use around 30k credits per day of coding of the 1M monthly allotment, I think it was $200/y. It's not available directly in my country. I've tried a bunch of local models too but Claude is just next level and inference is very cheap.
  - PufPufPuf 2y ago
    On the pricing page it says $10/month, what you describe sounds like pay-as-you-go, are you sure it was this service you tried?
    - karolist 2y ago
      Yes that the same, in the "Your Account" section of the service you have "Chat Credits" field, it was seeded with $10 initially and I got it down to $0.11 in one evening. You are right about the pricing page, it does say it's $10/month, but I believe it's $10/month up to some amount of queries in actuality, because they obviously can't offer infinite inference for $10. Maybe I was "holding it wrong", sending too much complete files as part of prompt and the token use and context window skyrocketed, but I absolutely used my initial $10 in one evening, unlike with Claude...
    - hobofan 2y ago
      From what I can tell inside the product, the public pricing seems deceptive. It appears that for the $10/month you just get access to additional features (e.g. bigger context) + a budget of $5/month of credits. The credits possibly translate 1:1 to the usage costs of the underlying models you chose. I asked it to add some missing documentation with the Claude 3.5 Sonnet model to a medium-sized Python file (2k lines) as a first test and that used up 13 cents of the credits. If it were working in a really productive way (where I'd also include more files as context), I'd probably burn through $20-50/day. I'm wondering why they are not publicly showing some more transparent pricing. Yeah, this method will bump their signup numbers for investors, but it also absolutely wrecks their churn numbers, when people immediately cancel in the first hour.
    - jessicant 2y ago
      Supermaven has a $10/month subscription which covers unlimited use of the in editor code autocomplete stuff. It also has a pay as you go chat which is integrated into the IDE w/ hotkeys, however the above subscription gives you $5 free credits for it a month. This doesn't call their own model but stuff like gpt-4o which is presumably why they don't offer unlimited free usage.
  - M4v3R 2y ago
    Supermaven does not use GPT-4o for their autocompletions, they have their own model for that and for your paid subscription you get unlimited use of autocomplete. There is a separate, optional feature called Supermaven Chat which is basically a wrapper around GPT-4o / Claude 3.5 where you can chat about fragments of your code, refactor them etc. This costs money, but you can also supply your own API key for OpenAI/Anthropic models. Or just use the web interface of any LLM if you just want to chat about your codebase.
terhechte 2y ago
Have you tried supermaven? (https://supermaven.com https://supermaven.com). I find it much better than copilot. Using it daily.
razodactyl 2y ago
Supposedly they were training on feedback provided by the plugin itself but that approach doesn't make sense to me because: - I don't remember the shortcuts most of the time. - When I run completions I double take and realise they're wrong. - I am not a good source of data. All this information is being fed back into the model as positive feedback. So perhaps reason for it to have gone downhill. I recall it being amazing at coding back in the day, now I can't trust it. Of course, it's anecdotal which is also problematic in itself but I have definitely noticed the issue where it will fail and stop autocompleting or provide completely irrelevant code.
- theptip 2y ago
  It could also be that back in the day they were training with a bit more code than they should have been (eg private repos) and now the lawyers are more involved the training set is smaller/more sanitized. Pure speculation of course.

heeton 2y ago

Have you tried supermaven? It replaced copilot for me a couple of months ago.

karolist 2y ago
I tried it, uses GPT-4o, the $10 sign up credit dissapeared in a few hours of intense coding, I'm not paying $500/mo for a fancy autocomple. Manual instruct style chat about code with Claude-Sonnet-3.5 is the best price/perf I've tried so far, through poe.com I use around 30k credits per day of coding of the 1M monthly allotment, I think it was $200/y. It's not available directly in my country. I've tried a bunch of local models too but Claude is just next level and inference is very cheap.
- PufPufPuf 2y ago
  On the pricing page it says $10/month, what you describe sounds like pay-as-you-go, are you sure it was this service you tried?
  - karolist 2y ago
    Yes that the same, in the "Your Account" section of the service you have "Chat Credits" field, it was seeded with $10 initially and I got it down to $0.11 in one evening. You are right about the pricing page, it does say it's $10/month, but I believe it's $10/month up to some amount of queries in actuality, because they obviously can't offer infinite inference for $10. Maybe I was "holding it wrong", sending too much complete files as part of prompt and the token use and context window skyrocketed, but I absolutely used my initial $10 in one evening, unlike with Claude...
  - hobofan 2y ago
    From what I can tell inside the product, the public pricing seems deceptive. It appears that for the $10/month you just get access to additional features (e.g. bigger context) + a budget of $5/month of credits. The credits possibly translate 1:1 to the usage costs of the underlying models you chose. I asked it to add some missing documentation with the Claude 3.5 Sonnet model to a medium-sized Python file (2k lines) as a first test and that used up 13 cents of the credits. If it were working in a really productive way (where I'd also include more files as context), I'd probably burn through $20-50/day. I'm wondering why they are not publicly showing some more transparent pricing. Yeah, this method will bump their signup numbers for investors, but it also absolutely wrecks their churn numbers, when people immediately cancel in the first hour.
  - jessicant 2y ago
    Supermaven has a $10/month subscription which covers unlimited use of the in editor code autocomplete stuff. It also has a pay as you go chat which is integrated into the IDE w/ hotkeys, however the above subscription gives you $5 free credits for it a month. This doesn't call their own model but stuff like gpt-4o which is presumably why they don't offer unlimited free usage.
- M4v3R 2y ago
  Supermaven does not use GPT-4o for their autocompletions, they have their own model for that and for your paid subscription you get unlimited use of autocomplete. There is a separate, optional feature called Supermaven Chat which is basically a wrapper around GPT-4o / Claude 3.5 where you can chat about fragments of your code, refactor them etc. This costs money, but you can also supply your own API key for OpenAI/Anthropic models. Or just use the web interface of any LLM if you just want to chat about your codebase.

terhechte 2y ago

Have you tried supermaven? (https://supermaven.com https://supermaven.com). I find it much better than copilot. Using it daily.

razodactyl 2y ago

Supposedly they were training on feedback provided by the plugin itself but that approach doesn't make sense to me because: - I don't remember the shortcuts most of the time. - When I run completions I double take and realise they're wrong. - I am not a good source of data. All this information is being fed back into the model as positive feedback. So perhaps reason for it to have gone downhill. I recall it being amazing at coding back in the day, now I can't trust it. Of course, it's anecdotal which is also problematic in itself but I have definitely noticed the issue where it will fail and stop autocompleting or provide completely irrelevant code.

theptip 2y ago
It could also be that back in the day they were training with a bit more code than they should have been (eg private repos) and now the lawyers are more involved the training set is smaller/more sanitized. Pure speculation of course.