> How does the target model validate the draft tokens without running the inference as normal? It does run the inference as normal, just in parallel with the o

> How does the target model validate the draft tokens without running the inference as normal? It does run the inference as normal, just in parallel with the other inferences > if it is doing just that, I don't get the point Running inferences in parallel allows you to only read the model weights out of memory only once for N parallel inferences, as opposed to reading them out of memory N times for N serial inferences. Inference is massively bottlenecked by memory bandwidth to the tune of one or two orders of magnitude compared to compute, so this helps a lot.