3 ms·Efficient LLM Scheduling by Learning to Rank2 points by zhisbug 2y agozhisbug 2y agoWe study how to approximate the famous shortest-job-first scheduling in LLM inference!