3 ms·
Yes — after the collapse, I ran ~200 more agent iterations across cleaner loops. Plateau settled at 0.7611 Combined ROC-AUC, up from the 0.7454 baseline. +157 b
by buildoak 7mo ago
Yes — after the collapse, I ran ~200 more agent iterations across cleaner loops. Plateau settled at 0.7611 Combined ROC-AUC, up from the 0.7454 baseline. +157 bps of improvement.
I ended up dropping WTA and focusing on ATP only — WTA data is noisier and lower quality - it was dragging the combined score. Best clean ATP-only ROC-AUC: 0.7611 (68.5% accuracy). That number has held as the gate baseline through 12+ subsequent iterations — every experiment since has regressed below it and been reverted.
Baseline accuracy was ATP 68.7%, WTA 66.6%. Ceiling seems to be right around 0.76 ROC-AUC for ATP with public data. The first 11 iterations found most of the real signal. The 200 follow-up iterations mostly confirmed the plateau rather than breaking through it - tried other fancy metrics like country of origin for tennis player, info on traumas, etc;
Planning to try the final thingy - LLM extracted motivation profile per player (based on wikipedia + public interviews) - still evaluating the hustle though. For now doing same autoresearch + ELO logic for Minecraft speed running.