3 ms·
I get it now. Benchmarks, in the end, are prompts for AI researchers. If you want a problem solved, translate it into an AGI benchmark. With enough patience,
by kelseyfrog 1y ago
I get it now. Benchmarks, in the end, are prompts for AI researchers.
If you want a problem solved, translate it into an AGI benchmark.
With enough patience, it becomes something AI researchers report on, optimize for, and ultimately saturate. Months later, the solution arrives; all you had to do was wait. AI researchers are an informal, lossy form of distributed computation - they mass-produce solutions and tools that, almost inevitably, solve the messy problem you started with.