4 ms·
Its 62 percent when using a neutral harness. https://arcprize.org/blog/astra https://arcprize.org/blog/astra
by Readerium 29d ago
Its 62 percent when using a neutral harness.
https://arcprize.org/blog/astra https://arcprize.org/blog/astra
- glenstein 29d agoInteresting both this and Sol got approximately a 37% boost with the custom harness.
- sbinnee 29d agoYet it is an impressive number. But yeah when you see a number 99 you have doubts. Thanks for the link
- Doohickey-d 29d agoTheir neutral harness is not very good though, if I read it right, it doesn't preserve the reasoning state between turns. No real harness discards reasoning state like that.
- AmazingTurtle 29d agoYeah that must be it. OpenAI doesn't want to disclose internal reasoning, that's why thats typically encrypted_content in OpenAI codex session ledgers etc.; leveraging responses API preserves reasoning server side all the way till a final answer is made; so that's very impressive and to me the score that matters.