4 ms·

Algorithm Clones Facial Expressions and Pastes Them onto Other Faces

59 points by tim_sw 12y ago

Mithaldu 12y ago
Oh good, more research without the tools necessary (code) to reproduce the results. Thanks, modern scientists.
- jszymborski 12y ago
  I've read a crazy amount of biomedical papers that model development or some such thing, spend the whole time talking about how cool their model is, hardly validating it in vivo or in vitro and call it a day without a single line of code released. Frankly, I don't even know how reviewers even believe them.
  - innguest 12y ago
    Academics are all cronies.
- mikejarema 12y ago
  Here's something similar which you can take for a test drive: http://auduno.github.io/clmtrackr/examples/facesubstitution.html http://auduno.github.io/clmtrackr/examples/facesubstitution.... It's open-sourced and carries the MIT license.
  - Mithaldu 12y ago
    That is both funny and horrifying. :D
bsenftner 12y ago
We have that and quite a bit more at www.3D-Avatar-Store.com. Expression cloning with local target fitting is quite easy... to the degree we don't even mention it. It's sorta expected when mapping expressions from one character to another, otherwise the mapping is painfully obvious.
pointernil 12y ago
How about using a stripped down version of such a system to re-target just the eyes of a person in a video chat? Could this be used to solve the "not looking into cam, thus not looking at me" issues in video chats today?
- kszx 12y ago
  Is this "issue" a bug or a feature? Isn't it very liberating that people can't see whether you're looking at them or not?
  - NeutronBoy 12y ago
    Pointernil is referring to the effect when you video conference with someone and they're talking, and looking at the TV screen (ie 'at' the other people). Since the camera is off-centre from the TV, it looks like they're not looking directly at the audience. While it's unintended, it's quite distracting.
    - pointernil 12y ago
      Yes. Thanks for the better description of the issue. Cheers
- mistercow 12y ago
  Seems like what you need for that is a lot simpler. You really just need to be able to cut out and redraw the iris and pupil in a different position. The rest of the face shouldn't need to change at all.
  - Mithaldu 12y ago
    That isn't all that's needed. When i skype with my family i do notice that the entire shape of my eyes looks different when i look at the screen vs. looking at the camera.
    - pointernil 12y ago
thret 12y ago
The application I see for this is avatar or even character faces in games, reflecting their user's expression in real time. Also probably cat memes.
- natrius 12y ago
  Any advancement in avatars makes it easier for us to virtualize our real selves. You might be able to get higher-fidelity video chat if you relied more on a base avatar and sent transformations of the face over the wire. Virtual reality avatars are going to be pretty good assuming this technique transfers well to three dimensions.

59 points by tim_sw 12y ago

Mithaldu 12y ago

Oh good, more research without the tools necessary (code) to reproduce the results. Thanks, modern scientists.

jszymborski 12y ago
I've read a crazy amount of biomedical papers that model development or some such thing, spend the whole time talking about how cool their model is, hardly validating it in vivo or in vitro and call it a day without a single line of code released. Frankly, I don't even know how reviewers even believe them.
- innguest 12y ago
  Academics are all cronies.
mikejarema 12y ago
Here's something similar which you can take for a test drive: http://auduno.github.io/clmtrackr/examples/facesubstitution.html http://auduno.github.io/clmtrackr/examples/facesubstitution.... It's open-sourced and carries the MIT license.
- Mithaldu 12y ago
  That is both funny and horrifying. :D

bsenftner 12y ago

We have that and quite a bit more at www.3D-Avatar-Store.com. Expression cloning with local target fitting is quite easy... to the degree we don't even mention it. It's sorta expected when mapping expressions from one character to another, otherwise the mapping is painfully obvious.

pointernil 12y ago

How about using a stripped down version of such a system to re-target just the eyes of a person in a video chat? Could this be used to solve the "not looking into cam, thus not looking at me" issues in video chats today?

kszx 12y ago
Is this "issue" a bug or a feature? Isn't it very liberating that people can't see whether you're looking at them or not?
- NeutronBoy 12y ago
  Pointernil is referring to the effect when you video conference with someone and they're talking, and looking at the TV screen (ie 'at' the other people). Since the camera is off-centre from the TV, it looks like they're not looking directly at the audience. While it's unintended, it's quite distracting.
  - pointernil 12y ago
    Yes. Thanks for the better description of the issue. Cheers
mistercow 12y ago
Seems like what you need for that is a lot simpler. You really just need to be able to cut out and redraw the iris and pupil in a different position. The rest of the face shouldn't need to change at all.
- Mithaldu 12y ago
  That isn't all that's needed. When i skype with my family i do notice that the entire shape of my eyes looks different when i look at the screen vs. looking at the camera.
  - pointernil 12y ago

thret 12y ago

The application I see for this is avatar or even character faces in games, reflecting their user's expression in real time. Also probably cat memes.

natrius 12y ago
Any advancement in avatars makes it easier for us to virtualize our real selves. You might be able to get higher-fidelity video chat if you relied more on a base avatar and sent transformations of the face over the wire. Virtual reality avatars are going to be pretty good assuming this technique transfers well to three dimensions.