3 ms·
That's a pretty odd stance. I've finetuned llama/mistral models that greatly outperform GPT4 with just a prompt. You have to know when to RAG, finetune, or RAG
by redox99 3y ago
That's a pretty odd stance. I've finetuned llama/mistral models that greatly outperform GPT4 with just a prompt.
You have to know when to RAG, finetune, or RAG+finetune.
- kossTKR 3y agoHow narrow is the dataset to be outperforming greatly? Just curious about what the usecase is for a 7b model in a business context - ie. what does it do?
- redox99 3y agoCode assistant for a niche programming language that GPT4 knows very little about and barely gets a hello world right.
- singularity2001 3y agogreatly outperform GPT4 *for* just a prompt your overfitting to training data convinces no-one that you created a "better GPT4"
- redox99 3y agoDo you always assume other people are incompetent? That's not very nice of you. I mostly work on AI, so I know if I'm overfitting or not. It performs provably better in it's domain (a niche programming language). GPT4 can barely write a hello world for it. I'm not creating a "better GPT4" general chatbot. I'm finetuning for a specific task.
- lostmsu 3y agoYou are making an extraordinary claim, and they require extraordinary evidence. Unless presented it is a good idea to assume they are bogus.
- simonw 3y ago"I've finetuned llama/mistral models that greatly outperform GPT4 with just a prompt" If you write about your experiments with that in detail I guarantee you'll get a lot of interest. The community is crying out for good, well documented, replicable examples of this kind of thing.
- redox99 3y agoI'm so behind in this area. I had finetuned a model that was SOTA and worth publishing about in October, but procrastinated. I'm scared to check if somebody else already published on this topic.