Experimentation makes a lot of sense as agents need to be pleasant to work with, and power users are a willing and able cohort to do that differentiation. Agent

Experimentation makes a lot of sense as agents need to be pleasant to work with, and power users are a willing and able cohort to do that differentiation. Agent capability is only part of the equation, as evidenced by the widespread frustration with the experience of working with Opus 5, despite it topping benchmarks and being objectively extremely powerful/competent. However, as a self-admitted eval true believer, I remain bullish that publicly available eval capabilities will catch up to current model intelligence, and start to be able to capture more and more of the subtle components of the agent experience, such as interactivity and code maintainability. Meta and Scale have some evals in the interactivity space that seem pretty promising https://arxiv.org/abs/2606.29957 https://arxiv.org/abs/2606.29957 https://labs.scale.com/papers/swe-interact https://labs.scale.com/papers/swe-interact. Human usability of the agent should definitely remain the top priority, but IMO we focus too much on the subjective, and not enough on the measurable/objective now, and I think/hope we'll see more companies move in the direction of Databricks