Needs to read this in-depth over weekends. Have fully imersed into LLMs for the past 2 years, ignored system research. This appears a very logical solution, i.

Needs to read this in-depth over weekends. Have fully imersed into LLMs for the past 2 years, ignored system research. This appears a very logical solution, i.e., use estimation service quality instead of resource metrics, for scheduling. This is also more or less a known facts in the recent years, as systems are becoming so complex and so distributed intertwined that scheduling based on host load concerns a minor factors of serving requests. It's like one grew taller, and need to worry not stepping on huddles, but not bumping heads into door frame. But we do need this kind of research to fomalize the practice, and get everyone on board. Google's applied research absolutely winning here.