3 ms·
Very interesting, but on my initial skim through, I don’t understand how this technique is lossless? Reproduced from the methods section: 1. Select a sequence
by buildbot 3y ago
Very interesting, but on my initial skim through, I don’t understand how this technique is lossless? Reproduced from the methods section:
1. Select a sequence of tc tokens that the model has already generated or completed predicting as the reading area.
2. Insert t '<m>' tokens at once after the reading area to serve as the memory area.
3. The model performs a single inference on the memory area, but discards the model's output, retaining only the KV pairs from each layer.
4. Discard the reading area, and the model continues generating text from after the memory area.
Isn’t the memory area a lossy compression of the reading area?
- godelski 3y agoThe paper is very confusing and should have a title change. In their results (4.2.1) they say > The observation that L_Repeat converges rapidly to a value close to zero under smaller compression ratios is significant. It demonstrates that the method is highly effective in compressing information losslessly into memory tokens So I'm also unconvinced