4 ms·
Do quantized models specifically prune out specific knowledge? I think they just compress things down but they're still in there. You'd most likely need to do
by KetoManx64 4mo ago
Do quantized models specifically prune out specific knowledge? I think they just compress things down but they're still in there. You'd most likely need to do that when you're doing the initial model training, but I'm not expert.
- JumpCrisscross 4mo ago> they just compress things down but they're still in there The compression is almost certainly in part specific knowledge getting fuzzed.
- DennisP 4mo agoYeah, but it's everything getting fuzzed, including the parts you care about.
- JumpCrisscross 4mo agoSure. There is a legitimate question around whether one can selectively excise “useless” knowledge. My guess is you can’t. The act of learning it encodes both the act of learning and the knowledge per se. The former is the power of the LLM. (I personally force mine to double check everything instead of going off memory.)