3 ms·
I'm inclined to think that the model is just too small, or that integer quantization damaged it badly. It has the GPT-2 trait of getting "stuck", (mode collapse
by sbierwagen 4y ago
I'm inclined to think that the model is just too small, or that integer quantization damaged it badly. It has the GPT-2 trait of getting "stuck", (mode collapse?) if it gives a bad response once, it will tend to repeat itself over and over. (Repetition was also a bad GPT-2 trait)
It's important to note that 7 billion parameters really is very small. 20 times smaller than GPT-3. Smaller still than ChatGPT or GPT-4. I find it plausible that in the future there will be distilled models substantially smaller than GPT-3 but with all its power, but GPT4ALL-LLaMA-7B isn't it.