3 ms·Fast Inference from Transformers via Speculative Decoding2 points by Jimmc414 3y agoJimmc414 3y agoExample implementation: https://twitter.com/ggerganov/status/1697262700165013689 https://twitter.com/ggerganov/status/1697262700165013689ftxbro 3y agocomment by karpathy https://twitter.com/karpathy/status/1697318534555336961 https://twitter.com/karpathy/status/1697318534555336961