3 ms·
Is the 30B model clearly better than the 7B? I played with Pi3141/alpaca-lora-7B-ggml two days ago and it was super disappointing. In percentage between 0% = a
by qwertox 4y ago
Is the 30B model clearly better than the 7B?
I played with Pi3141/alpaca-lora-7B-ggml two days ago and it was super disappointing. In percentage between 0% = alpaca-lora-7B-ggml and 100% GPT-3.5, where would LLaMA 30B be positioned?
- Rzor 4y agoI haven't been able to run it myself yet, but according to what I read so far from people who did, the 30B model is where the "magic" starts to happen.
- sp332 4y agoCheck out the graph on page 3 of this PDF: https://arxiv.org/abs/2302.13971 https://arxiv.org/abs/2302.13971 The 33B model started beating the 7B when it had been trained on only 1/3 as much data. And then they kept training it to 40% more than the total that 7B was trained on. It's better.