3 ms·
I've been wondering about context compression, actually. I remember a bunch of prompt compression tricks from a couple of years ago that must be usable. If yo
by regularfry 2y ago
I've been wondering about context compression, actually. I remember a bunch of prompt compression tricks from a couple of years ago that must be usable. If you were to say "anything past half context gets compressed, including the previously-compressed context" would mean you'd still have reasonable workspace and potentially infinite recollection of the most important things. Then yes, you'd be dragging a massive context around, but you're maximising the return on using it. I presume somebody's got a toolkit for this that I don't know the right terms to google for.
- mirekrusin 2y agoCompress first half (even recursively) is a nice idea, but you'd have to sort it by importance first – ie. you don't know if the most important sentence didn't exist at the beginning (ie. the whole goal could have been stated at the beginning, you don't want to loose it – you want memory to fade in terms of importance instead).
- regularfry 2y agoI don't know that, but the LLM might be able to make that judgement. If it's involved in the compression process, it can choose to drop cruft from the middle. Maybe "summarise then compress" is a better description.
- bloomingkales 2y agoYep, let the LLM operate like functional programming functions. Explode(context) … map, reduce, flatten, explode, filter, find, filter, map, flatten, explode … reduce(context). This is the imagination loop. OpenAI sold the prior reasoning loop. But that’s all this is. Quite frankly, I wouldn't come to HN for anything less than this quality of content: https://www.youtube.com/watch?v=DX3qLIwHoUo#t=1m29s https://www.youtube.com/watch?v=DX3qLIwHoUo#t=1m29s The machine reforming visually (explode, .., reduce) is what I'm describing.