Same here. Then you see SOTA in a browser from Ex0byt, online 10x training (JIT-Lora), TurboQuant (Google), etc. Just saw KV prediction mentioned in this thread

Same here. Then you see SOTA in a browser from Ex0byt, online 10x training (JIT-Lora), TurboQuant (Google), etc. Just saw KV prediction mentioned in this thread, so looking into that too. I'm adapting all of this to Rust+WGPU with compute shaders if you want to follow along. See this repo: https://github.com/tmzt/shady-thinker https://github.com/tmzt/shady-thinker Goal is Qwen3.5 27b on a Pixel 10 Pro running GrapheneOS.