4 ms·
They say it's infinite but then they use it to increase Llama's context length to 32k tokens instead of infinity. Still seems interesting.
by version_five 3y ago
They say it's infinite but then they use it to increase Llama's context length to 32k tokens instead of infinity. Still seems interesting.
- two_in_one 3y agoActually cool if it works. This would make many things possible on a single desktop. I'm afraid landmarking is actually a lossy compression. And the real content's size still limits model's even theoretical abilities.
- MacsHeadroom 3y agoIt's infinite but not free. Larger context still means more VRAM used and longer compute times.