I know several places where software components or systems software is tested against real-world applications, games, and whatnot, and... guess how it's often d

I know several places where software components or systems software is tested against real-world applications, games, and whatnot, and... guess how it's often done? Yup, with a combination of injecting artificial keyboard/mouse inputs and looking things at the screen. And these guys aren't beginners but professionals. You can't really automate something that doesn't have the APIs for it.