3 ms·
Ok, I'll bite: no, considering these are very different domains and you don't get system access by getting the wrong speculative branch for your next text token
by rf15 1mo ago
Ok, I'll bite: no, considering these are very different domains and you don't get system access by getting the wrong speculative branch for your next text token, you just get a slightly different (but probably still related enough) text.
- StevenWaterman 1mo agoSpeculative decoding is lossless because the main model checks whether it agrees with what the drafter outputted
- ranger_danger 1mo ago> you don't get system access by getting the wrong speculative branch for your next text token I think you could if the client supports tool/MCP calls.