3 ms·
> they just compress things down but they're still in there The compression is almost certainly in part specific knowledge getting fuzzed.
by JumpCrisscross 4mo ago
> they just compress things down but they're still in there
The compression is almost certainly in part specific knowledge getting fuzzed.
- DennisP 4mo agoYeah, but it's everything getting fuzzed, including the parts you care about.
- JumpCrisscross 4mo agoSure. There is a legitimate question around whether one can selectively excise “useless” knowledge. My guess is you can’t. The act of learning it encodes both the act of learning and the knowledge per se. The former is the power of the LLM. (I personally force mine to double check everything instead of going off memory.)