9 ms·
Better link: https://iquestlab.github.io/ https://iquestlab.github.io/ But yes, sadly it looks like the agent cheated during the eval
by denysvitali 10mo ago
Better link:
https://iquestlab.github.io/ https://iquestlab.github.io/
But yes, sadly it looks like the agent cheated during the eval
- s-macke 10mo agoThe link didn’t get enough votes a few days ago.
- denysvitali 10mo agoI know - I posted it :)
- denysvitali 10mo agoAccording to https://github.com/IQuestLab/IQuest-Coder-V1/issues/14#issuecomment-3705756919 https://github.com/IQuestLab/IQuest-Coder-V1/issues/14#issue... the result is still good after fixing the cheating problem. 76.2% (from 81.4%) which still beats Opus 4.5 (74.4%)!!
- ipython 10mo agoUnfortunately they seem to have neglected to update their front page readme with this information, continuing to mislead people: https://github.com/IQuestLab/IQuest-Coder-V1 https://github.com/IQuestLab/IQuest-Coder-V1
- anamexis 10mo agoIt is updated on their actual home page, though. There is clearly no intent to mislead people. https://iquestlab.github.io https://iquestlab.github.io
- alexpop80 10mo agoWhat do you mean? Opus 4.5 and GPT 5.2 broke the 80% mark and no other models yet seem to be passing this important milestone.