4 ms·
Is speculative decoding possible across 1/4/12/27 B Gemma 3 variants? LM Studio doesn't allow that (yet), but maybe the s/w requires some adjustments to suppor
by werediver 2y ago
Is speculative decoding possible across 1/4/12/27 B Gemma 3 variants?
LM Studio doesn't allow that (yet), but maybe the s/w requires some adjustments to support speculative decoding with Gemma 3.
- pinglin 2y agoIt's reported working but not with LM Studio: https://www.reddit.com/r/LocalLLaMA/comments/1j9reim/comment/mhq7uk5 https://www.reddit.com/r/LocalLLaMA/comments/1j9reim/comment...