I’d wager it can probably a few percents of the full runtime. But no matter the variation in the time it takes for LLMs to generate outputs (depending on the nb

I’d wager it can probably a few percents of the full runtime. But no matter the variation in the time it takes for LLMs to generate outputs (depending on the nb of tokens produced/input size) likely drowns it completely. Like 5s+/-1s vs 4.95s+/-1s