4 ms·

Sounds like Duo has a simple user experience, which is great. But it doesn't solve the eye-contact problem with video calls today. When you look at the person

Sounds like Duo has a simple user experience, which is great. But it doesn't solve the eye-contact problem with video calls today. When you look at the person on the screen whom you're talking to, that person sees you looking away, because you're not looking into the camera, which is somewhere on the edge of the screen. So you don't make eye contact with the person you're talking to. And for me, that makes video calls feel weird.

gruez 10y ago
1. build an algorithm to detect (using open cv?) and adjust eye position so they look correct to the viewer 2. patent it and license it to apple/google/microsoft 3. ??? 4. profit
- delta1 10y ago
  > build an algorithm > adjust eye position yes, those are the hard parts
  - donpark 10y ago
    [Recent ML work on Gaze Manipulation](http://www.creativeai.net/posts/6fokKr6xuytqsnT9F/deepwarp-photorealistic-image-resynthesis-for-gaze http://www.creativeai.net/posts/6fokKr6xuytqsnT9F/deepwarp-p...) [Online interactive demo](http://sites.skoltech.ru/compvision/projects/deepwarp/ http://sites.skoltech.ru/compvision/projects/deepwarp/)
    - kyriakos 10y ago
      is it just me or the examples are just creepy ? anyway I think its a minor issue. I am more worried about if I am dressed / groomed appropriately for the video call rather than if I am having constant eye contact.
      - spacemanmatt 10y ago
        confirmed creepy. not just you.
      - donpark 10y ago
        that is a major problem too, particularly in Asia and discrimination (race, sex, age, disabled) target groups. audio-only is also impacted (language, stutter, accent, pitch, etc) but to a lesser degree. but then we developers would be lost without problems.
option_greek 10y ago
Yup. Can't get over the feeling of looking down at the person you are talking to. If you look at the camera, you can't look at the person :(
- flamedoge 10y ago
  Just need 4 front facing cameras
  - anilgulecha 10y ago
    No, just place the other person as close to the camera as possible. That'll give the illusion of looking right at the person.
    - rando444 10y ago
      So, google hangouts?
  - spacemanmatt 10y ago
    Really, they need a camera a bit above center of the screen.
    - sangnoir 10y ago
      > Really, they need a camera a bit above center of the screen Somehow I don't see a phone that has a camera hole in the middle of the screen selling too well. If the camera is put in the bazel, then it replaces looking up/down with looking slightly to the side.
      - spacemanmatt 10y ago
        They'd have to solve the "hole in the screen" problem the "camera a bit above center" thing creates. You're not wrong.
    - rogerdpack 10y ago
sundvor 10y ago
Great point. Would be nice if we could integrate the lens into the display. :)
roryisok 10y ago
Can't they just put a camera behind the screen? Been waiting for this for over a decade
- wlievens 10y ago
  The screen would be in the way?
  - roryisok 10y ago
    Yes, but there must be some way to make the pixels translucent enough to work
    - wlievens 10y ago
      The screen pixels or the camera pixels?
- tintor 10y ago
  Or put 4 cameras on four sides of the screen and merge the video feed into screen center POV.
- michelb 10y ago
  apple has a patent on something similar since 2006, http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220060007222%22.PGNR.&OS=DN/20060007222&RS=DN/20060007222%5C%22 http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=... recently improved upon it: http://appleinsider.com/articles/09/01/08/apple_files_patent_for_camera_hidden_behind_display http://appleinsider.com/articles/09/01/08/apple_files_patent... Wonder why it never made it into a product.
  - dmoy 10y ago
    Maybe they tried and current screen tech sucks for it?
- amarant 10y ago
2bitencryption 10y ago
To be honest, I never found this to be a huge problem. After hundreds of Skype calls with people, I just don't notice it anymore. Sure, to fix that problem would be nice. But I don't think the engineering required to put a camera behind a screen, or creepily adjusting people's pupils in realtime, are worth it to rectify what is hardly an issue. EDIT: also, it's a bit weird to write a critique like yours for the launch of a new platform. No one else solves this issue, so why does Google not solving it suddenly make their platform pointless? It's like if Tesla releases the Model 3 and you're like "Uh, it doesn't fly? Pass."
- Alexss 10y ago
  " also, it's a bit weird to write a critique like yours for the launch of a new platform" I think this is a fair critique because the release announcement states the product should address the symptom that "nearly half of us never make video calls on mobile". If you believe the underlying problem is that lack of eye contact makes video calling low-value to many people this is not a viable solution.
nl 10y ago
I don't find the non-eye contact thing too big a deal. But sounds like it might be Skype soon: https://www.microsoft.com/en-us/research/publication/gaze-manipulation-for-one-to-one-teleconferencing/ https://www.microsoft.com/en-us/research/publication/gaze-ma...
scandox 10y ago
Entirely agree with you. This is the missing link in terms of making Video chat a good experience. I find voice is better because I concentrate on what the person is saying. With video I feel like my brain is exhausted trying to compensate for misread visual cues and trying to simulate eye-contact. Also that thing of seeing yourself is horrible...you keep monitoring your own expression which is just weird.
jeyashankher 10y ago
I have an iPhone 6S and the front camera seems to do a decent job with the eye-contact problem. I don't seem to need to do the look into the camera when taking selfies.
svantana 10y ago
To me, the obvious endgame is a 3D reconstruction of the image from the perspective of your eyes, so the screen becomes a "virtual window". I tried to pitch this to Skype a few years back, it felt like a no-brainer given that they were then recently acquired by Microsoft (which is big on similar technologies with Kinect and lately HoloLens). But they were completely uninterested! I think they were so used to Skyping that they didn't feel it was a problem.

gruez 10y ago

1. build an algorithm to detect (using open cv?) and adjust eye position so they look correct to the viewer 2. patent it and license it to apple/google/microsoft 3. ??? 4. profit

delta1 10y ago
> build an algorithm > adjust eye position yes, those are the hard parts
- donpark 10y ago
  [Recent ML work on Gaze Manipulation](http://www.creativeai.net/posts/6fokKr6xuytqsnT9F/deepwarp-photorealistic-image-resynthesis-for-gaze http://www.creativeai.net/posts/6fokKr6xuytqsnT9F/deepwarp-p...) [Online interactive demo](http://sites.skoltech.ru/compvision/projects/deepwarp/ http://sites.skoltech.ru/compvision/projects/deepwarp/)
  - kyriakos 10y ago
    is it just me or the examples are just creepy ? anyway I think its a minor issue. I am more worried about if I am dressed / groomed appropriately for the video call rather than if I am having constant eye contact.
    - spacemanmatt 10y ago
      confirmed creepy. not just you.
    - donpark 10y ago
      that is a major problem too, particularly in Asia and discrimination (race, sex, age, disabled) target groups. audio-only is also impacted (language, stutter, accent, pitch, etc) but to a lesser degree. but then we developers would be lost without problems.

option_greek 10y ago

Yup. Can't get over the feeling of looking down at the person you are talking to. If you look at the camera, you can't look at the person :(

flamedoge 10y ago
Just need 4 front facing cameras
- anilgulecha 10y ago
  No, just place the other person as close to the camera as possible. That'll give the illusion of looking right at the person.
  - rando444 10y ago
    So, google hangouts?
- spacemanmatt 10y ago
  Really, they need a camera a bit above center of the screen.
  - sangnoir 10y ago
    > Really, they need a camera a bit above center of the screen Somehow I don't see a phone that has a camera hole in the middle of the screen selling too well. If the camera is put in the bazel, then it replaces looking up/down with looking slightly to the side.
    - spacemanmatt 10y ago
      They'd have to solve the "hole in the screen" problem the "camera a bit above center" thing creates. You're not wrong.
  - rogerdpack 10y ago

sundvor 10y ago

Great point. Would be nice if we could integrate the lens into the display. :)

roryisok 10y ago

Can't they just put a camera behind the screen? Been waiting for this for over a decade

wlievens 10y ago
The screen would be in the way?
- roryisok 10y ago
  Yes, but there must be some way to make the pixels translucent enough to work
  - wlievens 10y ago
    The screen pixels or the camera pixels?
tintor 10y ago
Or put 4 cameras on four sides of the screen and merge the video feed into screen center POV.
michelb 10y ago
apple has a patent on something similar since 2006, http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220060007222%22.PGNR.&OS=DN/20060007222&RS=DN/20060007222%5C%22 http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=... recently improved upon it: http://appleinsider.com/articles/09/01/08/apple_files_patent_for_camera_hidden_behind_display http://appleinsider.com/articles/09/01/08/apple_files_patent... Wonder why it never made it into a product.
- dmoy 10y ago
  Maybe they tried and current screen tech sucks for it?
amarant 10y ago

2bitencryption 10y ago

To be honest, I never found this to be a huge problem. After hundreds of Skype calls with people, I just don't notice it anymore. Sure, to fix that problem would be nice. But I don't think the engineering required to put a camera behind a screen, or creepily adjusting people's pupils in realtime, are worth it to rectify what is hardly an issue. EDIT: also, it's a bit weird to write a critique like yours for the launch of a new platform. No one else solves this issue, so why does Google not solving it suddenly make their platform pointless? It's like if Tesla releases the Model 3 and you're like "Uh, it doesn't fly? Pass."

Alexss 10y ago
" also, it's a bit weird to write a critique like yours for the launch of a new platform" I think this is a fair critique because the release announcement states the product should address the symptom that "nearly half of us never make video calls on mobile". If you believe the underlying problem is that lack of eye contact makes video calling low-value to many people this is not a viable solution.

nl 10y ago

I don't find the non-eye contact thing too big a deal. But sounds like it might be Skype soon: https://www.microsoft.com/en-us/research/publication/gaze-manipulation-for-one-to-one-teleconferencing/ https://www.microsoft.com/en-us/research/publication/gaze-ma...

scandox 10y ago

Entirely agree with you. This is the missing link in terms of making Video chat a good experience. I find voice is better because I concentrate on what the person is saying. With video I feel like my brain is exhausted trying to compensate for misread visual cues and trying to simulate eye-contact. Also that thing of seeing yourself is horrible...you keep monitoring your own expression which is just weird.

jeyashankher 10y ago

I have an iPhone 6S and the front camera seems to do a decent job with the eye-contact problem. I don't seem to need to do the look into the camera when taking selfies.

svantana 10y ago

To me, the obvious endgame is a 3D reconstruction of the image from the perspective of your eyes, so the screen becomes a "virtual window". I tried to pitch this to Skype a few years back, it felt like a no-brainer given that they were then recently acquired by Microsoft (which is big on similar technologies with Kinect and lately HoloLens). But they were completely uninterested! I think they were so used to Skyping that they didn't feel it was a problem.