I think future is probably more similar to speculative execution (inference/decoding). A small LLM is used to speculate and a large LLM is used to confirm if ne

I think future is probably more similar to speculative execution (inference/decoding). A small LLM is used to speculate and a large LLM is used to confirm if needed. If the small LLM is accurate enough on N tokens it’s cheap for the large LLM to say looks good and keep moving along.