5 ms·

Distortion-Free Wide-Angle Portraits on Camera Phones [video]

71 points by Ultramanoid 7y ago

antpls 7y ago
Wow, Let's hope it will be included in the next release of Camera apps of major phone OSes
- Ultramanoid 7y ago
  Given that the authors are all from Google, I guess we can expect this to be working on the coming Pixel 4 at the very least.
  - makr 7y ago
    This is actually already working on my Pixel 3 right now.
    - nitrogen 7y ago
      Samsung's camera app as well on the newer Galaxy phones
- app4soft 7y ago
  And add "Distortion" effect for Camera apps, that would give good-old behavior.
bitL 7y ago
I always wondered why Zorin's original work [1] didn't make it to photography nor 3D rendering/gaming; we basically had 2 decades of ugly perspective distortions everybody got used to. [1] http://graphics.stanford.edu/~dzorin/perception/sig95/index.html http://graphics.stanford.edu/~dzorin/perception/sig95/index....
- dTal 7y ago
  I can't quite see why this is a hard problem. Surely simply converting to a spherical projection would do the job?
  - Luc 7y ago
    But then straight lines become curves. This maintains straight lines.
app4soft 7y ago
What about testing it for aerial imagery?
- ris 7y ago
  It works by detecting faces and using a different projection for them. How exactly would this be useful?
- opencl 7y ago
  Plain old barrel distortion correction already works fine for that.
bufferoverflow 7y ago
I think some of their examples are either fake or not shown in full. https://i.imgur.com/eRP0fZc.jpg https://i.imgur.com/eRP0fZc.jpg Look at the top-right corner.
- yorwba 7y ago
  The examples are not shown in full, because they also correct for nonlinearities in the lens projection. https://people.csail.mit.edu/yichangshih/wide_angle_portrait/shih_sig19_supp.pdf https://people.csail.mit.edu/yichangshih/wide_angle_portrait... The image on the left is a rectangular crop of the image after lens correction, the one on the right combines lens correction and their warping method. That combination allows them to pull in a few pixels that would've been outside the rectangular region otherwise.
diydsp 7y ago
I would have liked to see the results compared to ground truth. ie a picture of the person taken from the center the lense. And i would have like to see more false-positive rejection. e.g. stuff that gets detected as a face but isn't- at the edges of photos, but really that relies on the robustness of the face detect heuristic, so it's a short and sweet heuristic that will make people look more normal at the edges of photos.
jimbo1qaz 7y ago
Going into this paper, i expected they would be correcting for barrel distortion. I was disappointed to read that they instead: >we formulate an optimization problem to create a content-aware warping mesh which locally adapts to the stereographic projection on facial regions, and seamlessly evolves to the perspective projection over the background. Reminds me of this person who wrote a youtube tutorial with the factually incorrect title "Manually correct perspective in Photoshop". He did not correct the (already correct) perspective, but rather selectively distorted parts of scenery photographs so bridges looked "vertical". In fact, he was making those parts of the image no longer conform to the mathematical photo projection. Instead of picking a better-suited image projection, that video and this paper selectively fudge parts of the image to use a different projection. https://youtu.be/BocAGkS8yRQ https://youtu.be/BocAGkS8yRQ
randyrand 7y ago
what projection do our eyes use?

71 points by Ultramanoid 7y ago

antpls 7y ago

Wow, Let's hope it will be included in the next release of Camera apps of major phone OSes

Ultramanoid 7y ago
Given that the authors are all from Google, I guess we can expect this to be working on the coming Pixel 4 at the very least.
- makr 7y ago
  This is actually already working on my Pixel 3 right now.
  - nitrogen 7y ago
    Samsung's camera app as well on the newer Galaxy phones
app4soft 7y ago
And add "Distortion" effect for Camera apps, that would give good-old behavior.

bitL 7y ago

I always wondered why Zorin's original work [1] didn't make it to photography nor 3D rendering/gaming; we basically had 2 decades of ugly perspective distortions everybody got used to. [1] http://graphics.stanford.edu/~dzorin/perception/sig95/index.html http://graphics.stanford.edu/~dzorin/perception/sig95/index....

dTal 7y ago
I can't quite see why this is a hard problem. Surely simply converting to a spherical projection would do the job?
- Luc 7y ago
  But then straight lines become curves. This maintains straight lines.

app4soft 7y ago

What about testing it for aerial imagery?

ris 7y ago
It works by detecting faces and using a different projection for them. How exactly would this be useful?
opencl 7y ago
Plain old barrel distortion correction already works fine for that.

bufferoverflow 7y ago

I think some of their examples are either fake or not shown in full. https://i.imgur.com/eRP0fZc.jpg https://i.imgur.com/eRP0fZc.jpg Look at the top-right corner.

yorwba 7y ago
The examples are not shown in full, because they also correct for nonlinearities in the lens projection. https://people.csail.mit.edu/yichangshih/wide_angle_portrait/shih_sig19_supp.pdf https://people.csail.mit.edu/yichangshih/wide_angle_portrait... The image on the left is a rectangular crop of the image after lens correction, the one on the right combines lens correction and their warping method. That combination allows them to pull in a few pixels that would've been outside the rectangular region otherwise.

diydsp 7y ago

I would have liked to see the results compared to ground truth. ie a picture of the person taken from the center the lense. And i would have like to see more false-positive rejection. e.g. stuff that gets detected as a face but isn't- at the edges of photos, but really that relies on the robustness of the face detect heuristic, so it's a short and sweet heuristic that will make people look more normal at the edges of photos.

jimbo1qaz 7y ago

Going into this paper, i expected they would be correcting for barrel distortion. I was disappointed to read that they instead: >we formulate an optimization problem to create a content-aware warping mesh which locally adapts to the stereographic projection on facial regions, and seamlessly evolves to the perspective projection over the background. Reminds me of this person who wrote a youtube tutorial with the factually incorrect title "Manually correct perspective in Photoshop". He did not correct the (already correct) perspective, but rather selectively distorted parts of scenery photographs so bridges looked "vertical". In fact, he was making those parts of the image no longer conform to the mathematical photo projection. Instead of picking a better-suited image projection, that video and this paper selectively fudge parts of the image to use a different projection. https://youtu.be/BocAGkS8yRQ https://youtu.be/BocAGkS8yRQ

randyrand 7y ago

what projection do our eyes use?