I agree. This sort of usability testing is valuable - more valuable than simpy A/B testing in a vacuum - but it's important to remember that this is extremely q

I agree. This sort of usability testing is valuable - more valuable than simpy A/B testing in a vacuum - but it's important to remember that this is extremely qualitative and unscientific. Throwing in the webcams, eye-tracking, or what-haves-you IMO incorrect lends an air of specificity and scientific-ness that isn't warranted. Before we launched our last product update we brought people in to use the app. It was very much as you described - we invited them to come into the office where our PM sat with them, watch them used the app, and asked good questions. The much more subtle behaviors are IMO not determinable via pseudoscientific things like watching facial expressions, eye movement, heart-rate, or whatever these things are supposed to measure. Best A/B test for the rest.