3 ms·
There are very many tests, leaderboards etc. for AI models, and while none of them are perfect, in the aggregate they might not be a bad measure of intelligence
by dash2 2y ago
There are very many tests, leaderboards etc. for AI models, and while none of them are perfect, in the aggregate they might not be a bad measure of intelligence (perhaps locally, in an interval around current models' performance level, since people develop new tests as old ones get saturated).
- nialv7 2y agoYes, using those methods you can have a blurry idea of maybe this model is "smarter" than the other one, etc. But none of them may be used to anchor the claim that "intelligence is proportional to the log of compute", which is the claim sama was making.
- dash2 2y agoSuppose you take the first principal component of performance and fit it to log(compute). Seems like a fairly reasonable way to get an answer.