4 ms·

Why would we want an avatar vs a real video stream of the actual person?

by aantix 3y ago

kuschku 3y ago
Being able to have an avatar that fits your voice without having to actually look like that has many applications. Whether you're trans or you just want to join a video call early in the morning without dressing up, the applications are endless. In many situations we demand that people dress or present a certain way, just out of bullshit social expectations. This is one way to eat your cake and have it too.
- zamadatix 3y ago
  For those use cases you should be able to get much more accurate results using a base video stream. This more fits use cases where you're lacking a video stream but not necessarily because you just don't want to turn it on.
  - kridsdale1 3y ago
    A video stream isn’t volumetric. This is for the metaverse.
    - zamadatix 3y ago
      I think metaverse is their primary target in making it as well, as I mentioned in a sibling comment, but modeling an avatar then rendering it is probably the easiest computational way of generating a video, even if the video itself isn't volumetric. See: traditional 3D avatars already in meeting apps which use the video feed. I'm not sure if there would be any other potential use cases beyond these two. Or rather, I'm not able to think of them at so far.
- godelski 3y ago
  I get that this is to drive the avatar, but I'm curious as to why. There's stronger signals with video, which I'm certain taking in even a very low resolution image would have stronger signals to convey movement than audio does (either the network is memorizing a lot (which is fine, but limited), or this is an iteration towards a 3D high sensitivity audio driven for precise sound? Something else?). I mean the quest has cameras in it, so why not use those? Computation? They aren't big models (largest is 1.42G, smallest 0.58GB)
bigfishrunning 3y ago
You could generate the avatar clientside and save a ton of bandwidth vs a compressed video stream...
esafak 3y ago
Old recordings of people without pictures, for one!
zamadatix 3y ago
Given it's by meta I'm guessing it's related to their metaverse goals.
RobCodeSlayer 3y ago
I’m imaging video game applications where the avatars are controlled by both online users and LLMs
plaguuuuuu 3y ago
Either games or its just interesting research that mostly ties in with what FB is doing. Cause there are problems like, e.g. imagine the bandwidth requirement of streaming 3D copies of like 20 people in a room it's simply not possible within the near future, even today zoom/teams video conferencing is somehow highly compressed and shit quality with just low res 2D video.
airstrike 3y ago
Because my avatar can more easily travel to a made-up photorealistic 3D render of an exoplanet

Why would we want an avatar vs a real video stream of the actual person?

by aantix 3y ago

Why would we want an avatar vs a real video stream of the actual person?

Being able to have an avatar that fits your voice without having to actually look like that has many applications. Whether you're trans or you just want to join a video call early in the morning without dressing up, the applications are endless. In many situations we demand that people dress or present a certain way, just out of bullshit social expectations. This is one way to eat your cake and have it too.

zamadatix 3y ago
For those use cases you should be able to get much more accurate results using a base video stream. This more fits use cases where you're lacking a video stream but not necessarily because you just don't want to turn it on.
- kridsdale1 3y ago
  A video stream isn’t volumetric. This is for the metaverse.
  - zamadatix 3y ago
    I think metaverse is their primary target in making it as well, as I mentioned in a sibling comment, but modeling an avatar then rendering it is probably the easiest computational way of generating a video, even if the video itself isn't volumetric. See: traditional 3D avatars already in meeting apps which use the video feed. I'm not sure if there would be any other potential use cases beyond these two. Or rather, I'm not able to think of them at so far.
godelski 3y ago
I get that this is to drive the avatar, but I'm curious as to why. There's stronger signals with video, which I'm certain taking in even a very low resolution image would have stronger signals to convey movement than audio does (either the network is memorizing a lot (which is fine, but limited), or this is an iteration towards a 3D high sensitivity audio driven for precise sound? Something else?). I mean the quest has cameras in it, so why not use those? Computation? They aren't big models (largest is 1.42G, smallest 0.58GB)

bigfishrunning 3y ago

You could generate the avatar clientside and save a ton of bandwidth vs a compressed video stream...

esafak 3y ago

Old recordings of people without pictures, for one!

zamadatix 3y ago

Given it's by meta I'm guessing it's related to their metaverse goals.

RobCodeSlayer 3y ago

I’m imaging video game applications where the avatars are controlled by both online users and LLMs

plaguuuuuu 3y ago

Either games or its just interesting research that mostly ties in with what FB is doing. Cause there are problems like, e.g. imagine the bandwidth requirement of streaming 3D copies of like 20 people in a room it's simply not possible within the near future, even today zoom/teams video conferencing is somehow highly compressed and shit quality with just low res 2D video.

airstrike 3y ago

Because my avatar can more easily travel to a made-up photorealistic 3D render of an exoplanet