4 ms·
If they are amazing, then ChatGPT must be God-like in your view. I've been underwhelmed by Alpaca and Alpaca-LORA and LLaMA all at 13B but I have not tried hig
by whimsicalism 4y ago
If they are amazing, then ChatGPT must be God-like in your view.
I've been underwhelmed by Alpaca and Alpaca-LORA and LLaMA all at 13B but I have not tried higher params.
- kbrkbr 4y agoIn my opinion the problem with these is engineering a good prompt. I read of lots of people only getting nonsense or repetitions, and learned a bit from what they shared. These models are not chat bots. Vicuna is more friendly in that regard. But I’m well aware of their limitations also, and I can see how one can be underwhelmed. They are not jacks of all trades
- whimsicalism 4y agoAlpaca & Alpaca-LoRA are literally trained to be chat bots. Their goal is to be instruct-tuned.
- MacsHeadroom 4y agoAlpaca-LoRA, and all LoRas, are garbage. Alpaca is horrible compared to newer finetunes. Even cleaning the Alpaca dataset and retraining a cleaned Alpaca improves its performance greatly. But newer finetunes like Vicuna go well beyond that, including hundreds of thousands of real human conversations with GPT-4 ChatGPT in the dataset (unlike Alpaca's fully synthetic dataset). Vicuna-13B in 16bit is easily comparable to ChatGPT-3.5 in capability. Newer finetunes coming out nearly every day are going beyond chatGPT-3.5 and getting closer and closer to GPT-4 performance. You don't even have to install anything to validate this for yourself. There's a live web demo of Vicuna-13B right here: https://chat.lmsys.org/ https://chat.lmsys.org/ (disable ad blocker if it does not load)
- MacsHeadroom 4y agoAlso, InstructGPT was released in January 2021 and Vicuna-13B blows it out of the water.
- yyyk 4y ago30B Alpaca is better. Not anywhere near ChatGPT level, but better than the 13B models.