3 ms·

The key innovation is an accurate, reliable method for rotating faces so they're 'looking straight at the camera' before feeding them to a deep neural network.

by cs702 13y ago

The key innovation is an accurate, reliable method for rotating faces so they're 'looking straight at the camera' before feeding them to a deep neural network. They call this 3D photo rotation process "frontalization." Figure 1 on page 2 of the paper shows at a very high level how this is being done. Very nice!

weezer 13y ago
Just a little background the paper itself doesn't provide: The 3-d modeling and rotation is building on the work Yaniv did as part of Face.com (face recognition startup), which was acquired by Facebook.com. Studied here: http://vis-www.cs.umass.edu/lfw/results.html http://vis-www.cs.umass.edu/lfw/results.html Also Marc'Aurelio was just hired away from Google and is a deep learning expert.
thearn4 13y ago
Yeah, generally speaking for object detection - Scale, position, rotation: typically you can identify something based on 2 of these really efficiently (for example, wavelets give you a transform that is sufficient for object identification under scale and position transformations in O(N) operations). If you'd rather have orientation identification (ie. rotation angle) and scale in that mix but don't care about position, the radon transform is nice and easy to work with. But beyond inverting 1-2 key transformations, one usually has to pay a pretty hefty computational cost which often precludes online (near real-time) use.
apu 13y ago
Actually, that's only one of the contributions, and I'm not so sure it's the "key innovation". Every other recent face recognition method also tries to do some kind of alignment to make faces more similar in pose/expression/lighting prior to classifying them; and of these, several also fit faces to a 3-d model to rotate to frontal (with varying quality). See my other comment for my guess on what's actually providing the boost: https://news.ycombinator.com/item?id=7393378 https://news.ycombinator.com/item?id=7393378
- cs702 13y ago
  Yes, other attempts "also fit faces to a 3-d model to rotate to frontal (with varying quality)," as you put it, but this method for rotating faces appears to be superior -- more accurate and reliable. It looks like the main contribution to me :-)

by cs702 13y ago

weezer 13y ago

Just a little background the paper itself doesn't provide: The 3-d modeling and rotation is building on the work Yaniv did as part of Face.com (face recognition startup), which was acquired by Facebook.com. Studied here: http://vis-www.cs.umass.edu/lfw/results.html http://vis-www.cs.umass.edu/lfw/results.html Also Marc'Aurelio was just hired away from Google and is a deep learning expert.

thearn4 13y ago

Yeah, generally speaking for object detection - Scale, position, rotation: typically you can identify something based on 2 of these really efficiently (for example, wavelets give you a transform that is sufficient for object identification under scale and position transformations in O(N) operations). If you'd rather have orientation identification (ie. rotation angle) and scale in that mix but don't care about position, the radon transform is nice and easy to work with. But beyond inverting 1-2 key transformations, one usually has to pay a pretty hefty computational cost which often precludes online (near real-time) use.

apu 13y ago

Actually, that's only one of the contributions, and I'm not so sure it's the "key innovation". Every other recent face recognition method also tries to do some kind of alignment to make faces more similar in pose/expression/lighting prior to classifying them; and of these, several also fit faces to a 3-d model to rotate to frontal (with varying quality). See my other comment for my guess on what's actually providing the boost: https://news.ycombinator.com/item?id=7393378 https://news.ycombinator.com/item?id=7393378

cs702 13y ago
Yes, other attempts "also fit faces to a 3-d model to rotate to frontal (with varying quality)," as you put it, but this method for rotating faces appears to be superior -- more accurate and reliable. It looks like the main contribution to me :-)