3 ms·

I’m curious why software can’t correct this now? Messenger can make me look like a sheep, can’t it move my gaze?

rwc 6y ago
Available in FaceTime: https://appleinsider.com/articles/20/06/22/facetime-eye-contact-correction-feature-to-launch-with-ios-14 https://appleinsider.com/articles/20/06/22/facetime-eye-cont...
crazygringo 6y ago
It can, I've tried gaze correction software, but it has the unfortunate side effect of making the speaker look possessed. Total uncanny valley. We humans are sensitive to every tiny detail of the eye, because we express so much emotion through it. It's no joke that the "eye is the window to the soul". Gaze correction software literally winds up looking like it messes with your soul. Maybe someday we'll invent gaze correction that actually looks realistic -- but because it involves not just moving the eyeball, but opening/closing the eyelid (the eyelid moves as our eyes move vertically, like for a webcam), therefore moving the eyelashes, involving compensation with the lower eyelid, and more... it's going to be really hard.
- blululu 6y ago
  I recall once having a conversation with someone working on computer vision at Facebook, and they mentioned that the company had spent a considerable amount of talent & resources trying to get this feature to work cleanly and they did not get past the uncanny valley problems.
- cma 6y ago
  Gaze redirection looks bad partly because head orientation ends up decoupling from eye direction. With a depth camera you can move the camera's entire vantage point as if it were behind the right area on screen. Hat's probably a better approach, along with AI filling in the disocclusions (more cameras can also do this).
namibj 6y ago
What we have today is called "teleprompter", and not even particularly recent. With some slight pre-distortion and a tilted monitor it should also handle the focus issue a classic teleprompter setup has. Basically, you just place a half-reflective mirror at a 45 degree angle, lay the screen flat facing up (I propose a slight back-tilt "so the camera can see better", but you loose contrast due to viewing angle, which would ideally be corrected by the LCD), and look at the camera that's behind the mirror, while being able to view the screen in the mirror.

rwc 6y ago

Available in FaceTime: https://appleinsider.com/articles/20/06/22/facetime-eye-contact-correction-feature-to-launch-with-ios-14 https://appleinsider.com/articles/20/06/22/facetime-eye-cont...

crazygringo 6y ago

It can, I've tried gaze correction software, but it has the unfortunate side effect of making the speaker look possessed. Total uncanny valley. We humans are sensitive to every tiny detail of the eye, because we express so much emotion through it. It's no joke that the "eye is the window to the soul". Gaze correction software literally winds up looking like it messes with your soul. Maybe someday we'll invent gaze correction that actually looks realistic -- but because it involves not just moving the eyeball, but opening/closing the eyelid (the eyelid moves as our eyes move vertically, like for a webcam), therefore moving the eyelashes, involving compensation with the lower eyelid, and more... it's going to be really hard.

blululu 6y ago
I recall once having a conversation with someone working on computer vision at Facebook, and they mentioned that the company had spent a considerable amount of talent & resources trying to get this feature to work cleanly and they did not get past the uncanny valley problems.
cma 6y ago
Gaze redirection looks bad partly because head orientation ends up decoupling from eye direction. With a depth camera you can move the camera's entire vantage point as if it were behind the right area on screen. Hat's probably a better approach, along with AI filling in the disocclusions (more cameras can also do this).

namibj 6y ago

What we have today is called "teleprompter", and not even particularly recent. With some slight pre-distortion and a tilted monitor it should also handle the focus issue a classic teleprompter setup has. Basically, you just place a half-reflective mirror at a 45 degree angle, lay the screen flat facing up (I propose a slight back-tilt "so the camera can see better", but you loose contrast due to viewing angle, which would ideally be corrected by the LCD), and look at the camera that's behind the mirror, while being able to view the screen in the mirror.