7 ms·

I am always excited to see these Open Weight models released, I think its very good for the ecosystem and definitely has its place in many situations. However

I am always excited to see these Open Weight models released, I think its very good for the ecosystem and definitely has its place in many situations. However since I use LLMs as a coding assistant (mostly via "rubber duck" debugging and new library exploration) I really don't want to use anything other than the absolutely best in class available now. That continues to be GPT4-turbo (or maybe Claude 3). Does anyone know if there is any model out there that can be run locally and compete with GPT4-turbo? Or am I asking for something that is impossible?

fnordlord 3y ago
Do you mind my asking, if you're working on private codebases, how you go about using GPT/Claude as a code assistant? I'm just removing IP and pasting into their website's chat interface. I feel like there's got to be something better out there but I don't really know anyone else that's using AI code assistance at all.
- BoorishBears 3y ago
  Use the API (or a product that uses the API) If you can trust Azure or AWS or GCP with your IP, you can get Claude 3 and GPT-4 Turbo through at least one of them If your IP is so secret you can't do that, then I wouldn't imagine you'd be using the chat interface
- wincy 3y ago
  I haven’t used it but I’ve heard https://cursor.sh/ https://cursor.sh/ might work?
- Zambyte 3y ago
  Personally I don't paste anything. I ask for code examples that demonstrate what I want, and then I adapt it to my needs. It's definitely less powerful than directly sharing code, but it is what it is. I also run a personal language model server, but that is far less capable than the models available as services. It can still be better than nothing for code O can't share with APIs. I also use gpt.el a but for editor integration, but I honestly haven't workeded that into my workflow very much yet.
- bpiche 3y ago
  Vscode with GitHub copilot is great, been using it for about a year and a half, no complaints. The business tier allegedly doesn’t save/train on your data
- free_bip 3y ago
  Unless you have the privilege of being an enterprise customer with an SLA guaranteeing privacy, there's not much you can do other than using local models. I believe OpenAI says they don't train based on API requests but that's more of a "trust me bro" than any kind of guarantee.
  - zamadatix 3y ago
    Team and Enterprise come with the non-training guarantee, free and premium do not. Pretty much anyone can sign up for Team (I have, and I'm not a company) but you need to buy at least 2 seats for a total of $50/m. The rate limits are much better with that as well though.
- danenania 3y ago
  I'm building Plandex (https://github.com/plandex-ai/plandex https://github.com/plandex-ai/plandex), a terminal-based AI coding tool which currently uses the OpenAI api--I'm working on support for Anthropic and OSS models right now and hoping I can ship it later today. You can self-host it so that data is only going to the model provider (i.e. OpenAI) and nowhere else, and it gives you fine-grained control of context, so you can pick and choose exactly which files you want to load in. It's not going to pull in anything in the background that you don't want uploaded. There's a contributor working on integration with local models and making some progress, so that will likely be an option in the future as well, but for now it should at least be a pretty big improvement for you compared to the copy-paste heavy ChatGPT workflow.
  - fnordlord 3y ago
    Very cool! I'll take a look.
- paradite 3y ago
  I built a desktop tool to help reduce the amount of copy-pasting and improve the output quality for coding using ChatGPT or Claude: https://prompt.16x.engineer/ https://prompt.16x.engineer/
moffkalast 3y ago
You're asking for something that doesn't exist yet. Command R+, WizardLM-2-8x22B probably come closest.
butterchaos 3y ago
Nope, I don't even see what the excitement is for. We seem to be in denial of the scaling problems we face in that we can't even beat out the 1 year model. I subscribed and unsubscribed to Claude 3 in about an hour. It is just not better than chatGPT4. It is incredible to me that with all the motivation and resources of Meta, the best they can do is to produce a language model that isn't worth the time to even bother trying if a chatGPT4 subscriber.

fnordlord 3y ago

Do you mind my asking, if you're working on private codebases, how you go about using GPT/Claude as a code assistant? I'm just removing IP and pasting into their website's chat interface. I feel like there's got to be something better out there but I don't really know anyone else that's using AI code assistance at all.

BoorishBears 3y ago
Use the API (or a product that uses the API) If you can trust Azure or AWS or GCP with your IP, you can get Claude 3 and GPT-4 Turbo through at least one of them If your IP is so secret you can't do that, then I wouldn't imagine you'd be using the chat interface
wincy 3y ago
I haven’t used it but I’ve heard https://cursor.sh/ https://cursor.sh/ might work?
Zambyte 3y ago
Personally I don't paste anything. I ask for code examples that demonstrate what I want, and then I adapt it to my needs. It's definitely less powerful than directly sharing code, but it is what it is. I also run a personal language model server, but that is far less capable than the models available as services. It can still be better than nothing for code O can't share with APIs. I also use gpt.el a but for editor integration, but I honestly haven't workeded that into my workflow very much yet.
bpiche 3y ago
Vscode with GitHub copilot is great, been using it for about a year and a half, no complaints. The business tier allegedly doesn’t save/train on your data
free_bip 3y ago
Unless you have the privilege of being an enterprise customer with an SLA guaranteeing privacy, there's not much you can do other than using local models. I believe OpenAI says they don't train based on API requests but that's more of a "trust me bro" than any kind of guarantee.
- zamadatix 3y ago
  Team and Enterprise come with the non-training guarantee, free and premium do not. Pretty much anyone can sign up for Team (I have, and I'm not a company) but you need to buy at least 2 seats for a total of $50/m. The rate limits are much better with that as well though.
danenania 3y ago
I'm building Plandex (https://github.com/plandex-ai/plandex https://github.com/plandex-ai/plandex), a terminal-based AI coding tool which currently uses the OpenAI api--I'm working on support for Anthropic and OSS models right now and hoping I can ship it later today. You can self-host it so that data is only going to the model provider (i.e. OpenAI) and nowhere else, and it gives you fine-grained control of context, so you can pick and choose exactly which files you want to load in. It's not going to pull in anything in the background that you don't want uploaded. There's a contributor working on integration with local models and making some progress, so that will likely be an option in the future as well, but for now it should at least be a pretty big improvement for you compared to the copy-paste heavy ChatGPT workflow.
- fnordlord 3y ago
  Very cool! I'll take a look.
paradite 3y ago
I built a desktop tool to help reduce the amount of copy-pasting and improve the output quality for coding using ChatGPT or Claude: https://prompt.16x.engineer/ https://prompt.16x.engineer/

moffkalast 3y ago

You're asking for something that doesn't exist yet. Command R+, WizardLM-2-8x22B probably come closest.

butterchaos 3y ago

Nope, I don't even see what the excitement is for. We seem to be in denial of the scaling problems we face in that we can't even beat out the 1 year model. I subscribed and unsubscribed to Claude 3 in about an hour. It is just not better than chatGPT4. It is incredible to me that with all the motivation and resources of Meta, the best they can do is to produce a language model that isn't worth the time to even bother trying if a chatGPT4 subscriber.