5 ms·

I think the biggest net new recent technique is P/D disaggregation. And that spec dec is very different now especially post DSpark/DFlash. But overall yes the

I think the biggest net new recent technique is P/D disaggregation. And that spec dec is very different now especially post DSpark/DFlash. But overall yes the fundamentals of LLM performance optimization have been remarkably stable over the last few years.

brrrrrm 1mo ago
perhaps its unfair to say this in hindsight, but it's a fairly straightforward application of little's law that's been around for some time https://arxiv.org/html/2401.09670v2 https://arxiv.org/html/2401.09670v2