QwQ-32B on 3*1080ti does work with a 4096 window @ ~14 eval tokens/s But the lower bounds on intermediate tokens on CoT to get close to PTIME expressability +

QwQ-32B on 3*1080ti does work with a 4096 window @ ~14 eval tokens/s But the lower bounds on intermediate tokens on CoT to get close to PTIME expressability + QwQ-32B's verbosity/repeating on those tokens does eat up that pretty quickly. In theory the DAG walking QwQ-32B appears to be doing should require O(|E| log|V|) scratch space. IMHO they need to train to target that more. It seems fairly trivial to extend the context loss problem to the public UI, The Ω(n) scratch space may be hard to hit in reality but they seem almost exponential in scratch space right now, even on tasks that a traditional LLM could answer with just approximate retrieval. I may just be suffering confirmation bias, but the problem with memory in this case seems to be related to making the intermediate tokens palatable to users IMHO.