4 ms·
Probably some combination of quantizing down from original fp16 weights and changes to the system prompt used for chat. Both can cause degraded quality, the for
by incogitor 3y ago
Probably some combination of quantizing down from original fp16 weights and changes to the system prompt used for chat. Both can cause degraded quality, the former more than the latter.