3 ms·

This feels less like automated research and more like structured trial and error with a decent feedback loop. Still useful, but I think the real bottleneck is h

by lucasay 6mo ago

This feels less like automated research and more like structured trial and error with a decent feedback loop. Still useful, but I think the real bottleneck is how good your eval metric is. If that’s weak, the whole loop just optimizes for the wrong thing faster.

kridsdale1 6mo ago
I mean, isn’t that “the scientific method”?
- lucasay 6mo ago
  Partially—but science also questions the hypothesis and the metric. This mostly assumes both are correct and just optimizes within that box.
- svnt 6mo ago
  Only if the model is actually a human or equivalent, otherwise we don’t know what it is.
Almondsetat 6mo ago
Designing a good fitness function, a tale as old as time...