4 ms·
AI Discovers Algorithm That Beats NSDI'24 Best Paper
- andylizf 1y agoAuthor here. We used ADRS to automatically evolve a scheduling algorithm starting from a simple greedy policy. Surprisingly, it outperformed Uniform Progress, the NSDI'24 best paper algorithm by discovering "selective waiting" strategies and adaptive pattern recognition. Counter-intuitive finding: starting from a weaker baseline (greedy) worked better than starting from the SOTA algorithm. The AI found new insights rather than incremental improvements. The entire process took 5 hours and cost <$20. We're open-sourcing everything and have applied this to other systems problems (MoE load balancing with 5x speedup, see https://news.ycombinator.com/item?id=45688236 https://news.ycombinator.com/item?id=45688236). Happy to answer questions about the methodology or results!
- accheng 1y agoAwesome results! Did you use the simulator from the NSDI paper directly?