8 ms·
> By definition (an LLM as a “compressor”) this shouldn’t happen. A couple problems with this. 1) That's not the definition of an LLM, it's just a useful way
by furyofantares 3y ago
> By definition (an LLM as a “compressor”) this shouldn’t happen.
A couple problems with this.
1) That's not the definition of an LLM, it's just a useful way to think about it.
2) That is exactly what I'd expect a compressor to do. That's the exact job of lossless compression.
Of course the metaphor is lossy compression, not lossless. But it's not that surprising if lossy compression reproduces some piece of what it compressed. A jpeg doesn't get every pixel or every local group of pixels wrong.