4 ms·
They are not better, they are strictly worse in every way, and the performance characteristics show such degradation. There is a difference between distilling a
by satvikpendem 2y ago
They are not better, they are strictly worse in every way, and the performance characteristics show such degradation. There is a difference between distilling and quantizing, as while the latter does show some degradation too, it is not to the extent of distilled models and at least it's still the original model.
- DiabloD3 2y agoDepends how you define "worse in every way". With my own personal testing, DeepSeek's distillations have been able to do the task when the original upstream model either couldn't, or was marginally worse yet. You're preaching to the anti-choir on this, though: I do not think LLMs are ready for use yet. Maybe another few years, maybe another few decades, we'll find out I guess, but what we have today sure as hell isn't it.