Commenting out lines, rerunning, and trying to get meaningful performance counters is pretty much my workflow, and, from what I've observed, not very uncommon.

Commenting out lines, rerunning, and trying to get meaningful performance counters is pretty much my workflow, and, from what I've observed, not very uncommon. A tool like AGI should give at least somewhat better insight than just FPS though, as that is a crude measure that can conflate a lot of different things. There's a fair amount of academic work on GPU simulators, but a lot of the activity there seemed to be a few years ago. And in any case, I'm not sure how usable these are for tuning real workloads. If you have good hardware performance counters, those can give you almost as good information as you'd get out of a simulator. I think there's tremendous potential to do better tools in this space. I'm not quite sure what the business model is, though.