Ha, that's great. I was thinking more along the lines of a piece of software that runs alongside the Turker's browsing session and keeps track of what they see

Ha, that's great. I was thinking more along the lines of a piece of software that runs alongside the Turker's browsing session and keeps track of what they see and what they do with the mouse cursor, perhaps augmented by a video of the user's face taken by webcam, if available. But even simple feedback -- as you have built -- is extremely useful.