4 ms·
They are indeed impractical in agentic coding. However in deep research-like products you can have a pass with LLM to compress web page text into caveman speak
by egorfine 6mo ago
They are indeed impractical in agentic coding.
However in deep research-like products you can have a pass with LLM to compress web page text into caveman speak, thus hugely compressing tokens.
- claytongulick 6mo agoI don't understand how this would work without a huge loss in resolution or "cognitive" ability. Prediction works based on the attention mechanism, and current humans don't speak like cavemen - so how could you expect a useful token chain from data that isn't trained on speech like that? I get the concept of transformers, but this isn't doing a 1:1 transform from english to french or whatever, you're fundamentally unable to represent certain concepts effectively in caveman etc... or am I missing something?
- egorfine 6mo agoGood catch actually. Okay maybe not exactly caveman dialect, but text compression using LLM is definitely possible to save on tokens in deep research.