But do we react to the underlying physiognomy or the muscle tone (posture and expressions on their faces)? It would be hard to separate these two factors, but

But do we react to the underlying physiognomy or the muscle tone (posture and expressions on their faces)? It would be hard to separate these two factors, but you could start by generating 3D faces with different physiognomy but the same neutral expression.