3 ms·
I just tries the distilled 8b Llama variant, and it had very poor prompt adherence. It also reasoned its way to an incorrect answer, to a question plain Llama
by magicalhippo 2y ago
I just tries the distilled 8b Llama variant, and it had very poor prompt adherence.
It also reasoned its way to an incorrect answer, to a question plain Llama 3.1 8b got fairly correct.
So far not impressed, but will play with the qwen ones tomorrow.
- singularity2001 2y agonot adhering to system prompts is even officially mentioned as one of the caveats of the distilled models I wonder if this has to do with their censorship agenda but other report that it can be easily circumvented
- magicalhippo 2y agoI didn't have time to dig into the details of the models, but that makes sense I guess. I tried the Qwen 7B variant and it was indeed much better than the base Qwen 7B model at various math word problems.