15 ms·

Yeah, you'll have to have a way to record yourself and encode/decode/transmit in real time though.... Maybe in twenty years...

by sgt101 8y ago

deleted 8y ago
[deleted]
espadrine 8y ago
The peak of teleconferencing are the Hangouts and Facetime of this world, and they struggle to deliver almost-still-frame 1080p 2D video streams and high-quality audio. We've seen Oculus and others try to push the idea that 3D will help. The focus is wrong. The bulk of the pain is in understanding, for which the brain relies on two factors: audio and face/mouth position. Audio requires good microphones, which are too expensive to be commodity. At least, very good and fast STT autocues would help, especially with non-native speakers. Faces could use dedicated compression algorithms. Ideally, everyone would have Wacom-style devices on an online whiteboard, but that is only complementary. And it is still 2D.
- solarkraft 8y ago
  > Audio requires good microphones, which are too expensive to be commodity Huh? MEMS mics cost cents and multiple of them are frequently used in phones to get a pretty acceptable-quality (almost good!) signal.
  - espadrine 8y ago
    We maybe don't have the same scale to measure audio. What I find acceptable is “I hear the person just as well as if they were here”. Hearing is a continuum. What the top 25% of hearing people consider OK quality is often what the lower 50% really struggles with, barely keeping up with the conversation. Even in same-room audio conditions, most people ask for something to be repeated at least once in half the conversations I witness. Any further degradation rapidly increases this statistic. And I bet there are cases where people don't bother asking.
- mgkimsal 8y ago
  What webex/facetime/goto/hangouts needs is live transcripts, or... pretty live. Each person's mic/setup has much better audio of what was said, and can be configured (eventually) to learn that person's accent. When 2-3 voices talk over each other, I have to ask people to stop/repeat, wasting time, losing momentum, when the computers themselves can likely give me a good indication of what was said via text as well as audio. And if a transcription is bad, the person can 'correct' it on their end, or just verbally - "no, that's not the word I used", etc.
Qworg 8y ago
I worked on just this at MSR: https://www.microsoft.com/en-us/research/project/holoportation-3/ https://www.microsoft.com/en-us/research/project/holoportati... This is in real time - encoded, transmitted, and decoded. I hope it gets out of the lab soon.
- sgt101 8y ago
  OOh interesting. What's the blocker in terms of downstreaming? Are you looking for partners? Could be a great 5G/corporate networking use case I guess..
  - Qworg 8y ago
    Finding the right partner and how to design a business around it. If it isn't a $1B business at Microsoft, it is hard to land. :/
UncleEntity 8y ago
IDK, all you really need is the joint position vectors and then animate the avatar on the receiving side...probably don't need a realistic rendering of your plumber when a cartoon hedgehog will do since all they're doing is transmitting concepts.

Yeah, you'll have to have a way to record yourself and encode/decode/transmit in real time though.... Maybe in twenty years...

by sgt101 8y ago

Yeah, you'll have to have a way to record yourself and encode/decode/transmit in real time though.... Maybe in twenty years...

deleted 8y ago

[deleted]

espadrine 8y ago

The peak of teleconferencing are the Hangouts and Facetime of this world, and they struggle to deliver almost-still-frame 1080p 2D video streams and high-quality audio. We've seen Oculus and others try to push the idea that 3D will help. The focus is wrong. The bulk of the pain is in understanding, for which the brain relies on two factors: audio and face/mouth position. Audio requires good microphones, which are too expensive to be commodity. At least, very good and fast STT autocues would help, especially with non-native speakers. Faces could use dedicated compression algorithms. Ideally, everyone would have Wacom-style devices on an online whiteboard, but that is only complementary. And it is still 2D.

solarkraft 8y ago
> Audio requires good microphones, which are too expensive to be commodity Huh? MEMS mics cost cents and multiple of them are frequently used in phones to get a pretty acceptable-quality (almost good!) signal.
- espadrine 8y ago
  We maybe don't have the same scale to measure audio. What I find acceptable is “I hear the person just as well as if they were here”. Hearing is a continuum. What the top 25% of hearing people consider OK quality is often what the lower 50% really struggles with, barely keeping up with the conversation. Even in same-room audio conditions, most people ask for something to be repeated at least once in half the conversations I witness. Any further degradation rapidly increases this statistic. And I bet there are cases where people don't bother asking.
mgkimsal 8y ago
What webex/facetime/goto/hangouts needs is live transcripts, or... pretty live. Each person's mic/setup has much better audio of what was said, and can be configured (eventually) to learn that person's accent. When 2-3 voices talk over each other, I have to ask people to stop/repeat, wasting time, losing momentum, when the computers themselves can likely give me a good indication of what was said via text as well as audio. And if a transcription is bad, the person can 'correct' it on their end, or just verbally - "no, that's not the word I used", etc.

Qworg 8y ago

I worked on just this at MSR: https://www.microsoft.com/en-us/research/project/holoportation-3/ https://www.microsoft.com/en-us/research/project/holoportati... This is in real time - encoded, transmitted, and decoded. I hope it gets out of the lab soon.

sgt101 8y ago
OOh interesting. What's the blocker in terms of downstreaming? Are you looking for partners? Could be a great 5G/corporate networking use case I guess..
- Qworg 8y ago
  Finding the right partner and how to design a business around it. If it isn't a $1B business at Microsoft, it is hard to land. :/

UncleEntity 8y ago

IDK, all you really need is the joint position vectors and then animate the avatar on the receiving side...probably don't need a realistic rendering of your plumber when a cartoon hedgehog will do since all they're doing is transmitting concepts.