I imagine that there are nuances to facial expression that might be difficult to extract from an image or video, depending on the parameters to your ML model.

I imagine that there are nuances to facial expression that might be difficult to extract from an image or video, depending on the parameters to your ML model. For example, if someone looks into my eyes, furrows their brows, and shakes their head slowly back and forth, this could mean that they are intensely sympathetic with me (when I am talking), or it could mean that they are disgusted with what I have said (when I have stopped talking), or many other things. If you add sound input to your model then maybe it can figure out how to incorporate that dimension into its results. Very cool though. Maybe you could have something like this to help colorblind people see colors, deaf people see sounds, the possibilities are exciting.