2 ms·

Any metric that can be targeted can be gamed

• kelseyfrog 9mo ago
  Then target it with metrics worth solving[1]. 1. Ex https://mppbench.com/ https://mppbench.com/
  • falcor84 9mo ago
    But that seems to be measuring "superintelligence" rather than just AI, no?
  • itemize123 9mo ago
    useless benchmark if all it shows will be fail right; At least it's a very lagging benchmark
    • kelseyfrog 9mo ago
      It's common for benchmarks to start at zero and eventually become saturated. The atari games benchmark started as such and is now a solved problem.
• positron26 9mo ago
  If the metric is a latent variable summarizing subjective judgements, yes.