One option is to capture the video with depth as well as colour data and then just rotate it to point in the right direction(s) to maintain eye contact. This is

One option is to capture the video with depth as well as colour data and then just rotate it to point in the right direction(s) to maintain eye contact. This is what we experimented with on https://matrix.org/blog/2018/02/05/3-d-video-calling-with-matrix-webrtc-and-webvr-at-fosdem-2018 https://matrix.org/blog/2018/02/05/3-d-video-calling-with-ma...