> How would "AI" even start with issues like "product sometimes runs slow" or "it crashes 1/20 times with this vague error message". By asking more questions,

> How would "AI" even start with issues like "product sometimes runs slow" or "it crashes 1/20 times with this vague error message". By asking more questions, just like a developer would. This is already available in the chat form, but that's a separate set of skills and would require the other party to give responses. It's a useful test, but I believe not in the context of what swe-bench wants to measure.