4 ms·

Note that this is just one recent work in a well established research problem that has been worked on for decades. The interesting bit here is that they appear

Note that this is just one recent work in a well established research problem that has been worked on for decades. The interesting bit here is that they appear to achieve state of the art results by doing something simpler than other approaches - instead of fitting a well thought out generic morphable face model, they use a pretty standard deep learning model to map a 2d image to a discretized 3d model of the face (voxels). Intuitively this feels a bit off, since outputting a discretized 3d model instead of fitting a continuous model has inherent resolution limitations, but the benchmark results are pretty impressive.

blt 9y ago
Maybe it is comparatively easy to fit the morphable model to the voxels?
- bd 9y ago
  This new method seems to produce nicer results when the input deviates from what would generic model expect (e.g. stronger facial expressions). ----- Here is the same image used as input for 3d face reconstruction using both methods: https://imgur.com/a/bZ012 https://imgur.com/a/bZ012 Left side is morphable face model (EOS by Patrik Huber). Right side is this new CNN voxels method. ----- Here is live 3D model on the authors' site: http://www.cs.nott.ac.uk/~psxasj/3dme/view.php?name=../59b419de805ba http://www.cs.nott.ac.uk/~psxasj/3dme/view.php?name=../59b41...
amelius 9y ago
I guess this means that the method easily generalizes to any type of object (?) > Intuitively this feels a bit off, since outputting a discretized 3d model instead of fitting a continuous model has inherent resolution limitations, but the benchmark results are pretty impressive. Isn't this because deep learning is in fact a type of interpolation? And another question: are there any techniques based on neural networks that combine multiple images into the most plausible model?
- Hydraulix989 9y ago
  > Isn't this because deep learning is in fact a type of interpolation? No The output is still discrete. > And another question: are there any techniques based on neural networks that combine multiple images into the most plausible model? Could work.

blt 9y ago

Maybe it is comparatively easy to fit the morphable model to the voxels?

bd 9y ago
This new method seems to produce nicer results when the input deviates from what would generic model expect (e.g. stronger facial expressions). ----- Here is the same image used as input for 3d face reconstruction using both methods: https://imgur.com/a/bZ012 https://imgur.com/a/bZ012 Left side is morphable face model (EOS by Patrik Huber). Right side is this new CNN voxels method. ----- Here is live 3D model on the authors' site: http://www.cs.nott.ac.uk/~psxasj/3dme/view.php?name=../59b419de805ba http://www.cs.nott.ac.uk/~psxasj/3dme/view.php?name=../59b41...

amelius 9y ago

I guess this means that the method easily generalizes to any type of object (?) > Intuitively this feels a bit off, since outputting a discretized 3d model instead of fitting a continuous model has inherent resolution limitations, but the benchmark results are pretty impressive. Isn't this because deep learning is in fact a type of interpolation? And another question: are there any techniques based on neural networks that combine multiple images into the most plausible model?

Hydraulix989 9y ago
> Isn't this because deep learning is in fact a type of interpolation? No The output is still discrete. > And another question: are there any techniques based on neural networks that combine multiple images into the most plausible model? Could work.