Agree. Also, sometimes I intentionally want the agent to do something differently to how it would naturally solve the problem. For example, there might be a spe

Agree. Also, sometimes I intentionally want the agent to do something differently to how it would naturally solve the problem. For example, there might be a specific design decision that the agent should adhere to. Obviously, this will lead to slower task completion, higher inference costs etc. because I’m asking the agent not to take the path of least resistance. This kind of benchmark completely misses that nuance.