3 ms·

This is excellent! A similar paper from Meta was published two days ago on Avatars too https://shunsukesaito.github.io/rgca/ https://shunsukesaito.github.io/rg

This is excellent! A similar paper from Meta was published two days ago on Avatars too https://shunsukesaito.github.io/rgca/ https://shunsukesaito.github.io/rgca/ Here's the discussion for it(empty as of right now): https://news.ycombinator.com/item?id=38554537 https://news.ycombinator.com/item?id=38554537

lelag 3y ago
There was also this project that was posted a couple days ago: https://blog.metaphysic.ai/controllable-deepfakes-with-gaussian-avatars/ https://blog.metaphysic.ai/controllable-deepfakes-with-gauss... Why they call a virtual avatar a deepfake beats me though....
- peterleiser 3y ago
  If you can "wear" the face of someone else then that seems like deepfake territory.
- yjftsjthsd-h 3y ago
  > Why they call a virtual avatar a deepfake beats me though.... What's the difference? In both cases you're simulating somebody's face in a way that doesn't actually require having the original to drive it.
ChuckMcM 3y ago
It is, I fully expect at some point we'll get "in game" MMORPG player characters that are emotively very very effective. The immersion level will be pretty intense at that point, especially if it's VR but even a 3rd person view type like WoW using a web cam to process your facial expression? Another interesting application would be "zoom" meetings where everyone is shown around the table or in the audience and it processes their emotive state in real time. That could help speakers engage with the audience in a better way. Of course the "bad" uses of this tech are pretty out there too, from porn apps, to ripping people off by getting a "zoom" from a relative.
- godelski 3y ago
  > Another interesting application would be "zoom" meetings where everyone is shown around the table or in the audience and it processes their emotive state in real time. That could help speakers engage with the audience in a better way. See Permutation City[0]. Another application is actually masking reactions selectively. There's some interesting aspects that play out with respect to this and some other interesting aspects tech that people will now view as near Sci-Fi. The VR identity cloning is a common Sci-Fi plot, not specifically a Permutation City thing. Great book, highly recommend. [0] https://en.wikipedia.org/wiki/Permutation_City https://en.wikipedia.org/wiki/Permutation_City
- johnnymorgan 3y ago
  Star citizen has a very basic version of FoIP, it does help convey emotive state even if a little janky. Add in stuff like head tracking and you get a very natural feeling to interacting. Give me haptics and I'll be super happy. Hand gestures go a long way too
muglug 3y ago
I think the tech in that paper was demoed for this podcast: https://youtu.be/MVYrJJNdrEg https://youtu.be/MVYrJJNdrEg The big roadblock to commercialisation for the moment is the original capture — for the paper they used a 110-camera capture rig under ideal lighting conditions. In the above podcast Zuckerberg mentions that in the future people will be able to use their phones to do that same capture, but I don’t think that tech is coming next year. I wonder if there’ll be an interim period where people who want high-quality avatars will have to book an appointment.

lelag 3y ago

There was also this project that was posted a couple days ago: https://blog.metaphysic.ai/controllable-deepfakes-with-gaussian-avatars/ https://blog.metaphysic.ai/controllable-deepfakes-with-gauss... Why they call a virtual avatar a deepfake beats me though....

peterleiser 3y ago
If you can "wear" the face of someone else then that seems like deepfake territory.
yjftsjthsd-h 3y ago
> Why they call a virtual avatar a deepfake beats me though.... What's the difference? In both cases you're simulating somebody's face in a way that doesn't actually require having the original to drive it.

ChuckMcM 3y ago

It is, I fully expect at some point we'll get "in game" MMORPG player characters that are emotively very very effective. The immersion level will be pretty intense at that point, especially if it's VR but even a 3rd person view type like WoW using a web cam to process your facial expression? Another interesting application would be "zoom" meetings where everyone is shown around the table or in the audience and it processes their emotive state in real time. That could help speakers engage with the audience in a better way. Of course the "bad" uses of this tech are pretty out there too, from porn apps, to ripping people off by getting a "zoom" from a relative.

godelski 3y ago
> Another interesting application would be "zoom" meetings where everyone is shown around the table or in the audience and it processes their emotive state in real time. That could help speakers engage with the audience in a better way. See Permutation City[0]. Another application is actually masking reactions selectively. There's some interesting aspects that play out with respect to this and some other interesting aspects tech that people will now view as near Sci-Fi. The VR identity cloning is a common Sci-Fi plot, not specifically a Permutation City thing. Great book, highly recommend. [0] https://en.wikipedia.org/wiki/Permutation_City https://en.wikipedia.org/wiki/Permutation_City
johnnymorgan 3y ago
Star citizen has a very basic version of FoIP, it does help convey emotive state even if a little janky. Add in stuff like head tracking and you get a very natural feeling to interacting. Give me haptics and I'll be super happy. Hand gestures go a long way too

muglug 3y ago

I think the tech in that paper was demoed for this podcast: https://youtu.be/MVYrJJNdrEg https://youtu.be/MVYrJJNdrEg The big roadblock to commercialisation for the moment is the original capture — for the paper they used a 110-camera capture rig under ideal lighting conditions. In the above podcast Zuckerberg mentions that in the future people will be able to use their phones to do that same capture, but I don’t think that tech is coming next year. I wonder if there’ll be an interim period where people who want high-quality avatars will have to book an appointment.