4 ms·
Can we expect similar issues such as spectre and meltdown that intel experienced with speculative execution.. but, in the form of prompt injection/poisoning?
by freakynit 1mo ago
Can we expect similar issues such as spectre and meltdown that intel experienced with speculative execution.. but, in the form of prompt injection/poisoning?
- rf15 1mo agoOk, I'll bite: no, considering these are very different domains and you don't get system access by getting the wrong speculative branch for your next text token, you just get a slightly different (but probably still related enough) text.
- StevenWaterman 1mo agoSpeculative decoding is lossless because the main model checks whether it agrees with what the drafter outputted
- ranger_danger 1mo ago> you don't get system access by getting the wrong speculative branch for your next text token I think you could if the client supports tool/MCP calls.
- Lerc 1mo agoNo, spectre is based upon speculative execution of code generated by another party. The breach is from the protection stopping that code from doing anything bad. Speculatve execution making things faster by varying amounts is used to turn those differences in timing into a signal. You could possibly in-pronciple detect what a model with a censorship filter was actually saying, but not really. The signal is weak and needs lots of samples to get anything worth having. You just don't have that level of control to set things up.