2 ms·
Interesting, its almost like speculative execution on a CPU. A smaller model guesses the output of the larger model, and the larger model can effectively check
by brucethemoose2 3y ago
Interesting, its almost like speculative execution on a CPU. A smaller model guesses the output of the larger model, and the larger model can effectively check and skip the correctly guessed part of the output.