4 ms·

If you use the Cursor IDE: the folks that wrote it talked about their use of speculative decoding to make "Apply" faster on the Lex Friedman podcast last month.

by cbhl 2y ago

If you use the Cursor IDE: the folks that wrote it talked about their use of speculative decoding to make "Apply" faster on the Lex Friedman podcast last month. Here it is on YouTube, although you can also find it on Spotify and other podcast platforms: https://youtu.be/oFfVt3S51T4?t=1206 https://youtu.be/oFfVt3S51T4?t=1206

afro88 2y ago
For those that prefer text: it seems they use a weaker but faster model for the "predicted output" / speculation. Pretty smart.
pandada8 2y ago
https://fireworks.ai/blog/cursor https://fireworks.ai/blog/cursor Fireworks AI has a blog about it.