4 ms·
ARC leaderboard has results from semi-private data for frontier models, they have another competition for private data. It is described in their methodology: h
by andriy_koval 1mo ago
ARC leaderboard has results from semi-private data for frontier models, they have another competition for private data.
It is described in their methodology: https://arcprize.org/policy https://arcprize.org/policy
It makes sense, since once OpenAI API receive task, it is not private anymore but leaked to OpenAI.
- tintor 1mo agoWhere are results for private data? Which LLMs participate on private set? Open weight LLMs only?
- andriy_koval 1mo agoYes, they run competitions once a year amongst open weight models