3 ms·Speculative Streaming: Fast LLM Inference Without Auxiliary Models2 points by gok 3y agoNickNickGo 3y agoTheir on device optimizations have all seeds of efficient LLMs