An excellent way to figure this out would be to apply some sort of machine learning technique. Perhaps an Active-Appearance-Model or some other statistical meas

An excellent way to figure this out would be to apply some sort of machine learning technique. Perhaps an Active-Appearance-Model or some other statistical measure to mathematically represent the shape and texture of an "untrustworthy" face as a lower dimensional vector. Then train classifier on this data which can discriminate between "trustworthy" and "untrustworthiness". Of course, there are going to be ethical concerns, but this is definitely one interesting way of doing it.