3 ms·

Yeah. LLMs are fundamentally a batch-based system, and we smear a veneer of liveness and autonomy on top.