4 ms·

As a camera nerd, I have to weigh in with some minor annoyances: there are errors in this article that reflect common misconceptions or elisions that computer v

As a camera nerd, I have to weigh in with some minor annoyances: there are errors in this article that reflect common misconceptions or elisions that computer vision engineers often believe in. Let me debunk a few... "Cameras, by design, also introduce a level of distortion": distortion is an artifact that is usually minimized in lens design if you are trying to create a rectilinear lens, though you are typically forced to trade it off versus other design criteria (like resolving power, or optical complexity). If you're trying to design a rectilinear lens, you explicitly try to remove any distortion; for "normal" focal lengths (focal length ~= image circle diagonal) you can often get vanishingly low distortion, even with relatively simple lenses. (There was a 19th century lens design that achieved "zero" distortion with just a few elements, and a precisely-chosen aperture position; the name escapes me at the moment.) If you are not designing a rectilinear lens, there are other lens mappings, in which case it's not really proper to describe the effect as distortion, though in technical literature it's often still described this way. (For further reading, check out F-theta vs. F-tan-theta lenses: https://en.wikipedia.org/wiki/Fisheye_lens#Focal_length https://en.wikipedia.org/wiki/Fisheye_lens#Focal_length; https://www.thorlabs.com/newgrouppage9.cfm?objectgroup_id=10766 https://www.thorlabs.com/newgrouppage9.cfm?objectgroup_id=10....) "... the proportions of objects closer to the edges of the frame are distorted." This is technically incorrect; objects subtend the same number of pixels in an f*theta fisheye image whether they're at the center of the frame or at the edge, for a given camera-subject distance. The apparent distortion exists because you're viewing it further away than the focal length implies, so the viewer at a "normal" viewing distance is applying a transform to the image that is not distance-preserving. If you get extremely close to the image, and if it were displayed on a curved surface (so the image's angular extent to the viewer matched the camera's FOV), you'd see no distortion. Also, we think of arranging subject matter for, e.g., a portrait, along a plane that's orthogonal to the camera's axis -- but then people closer to the edge are farther from the camera, and would naturally appear smaller. It's a weird artifact of rectilinear lenses that they're actually magnified in the resulting image so people standing in a line are all equal size; to get the same effect with a fisheye lens, you just have people arranged in an arc centered on the camera. (Most portraits are not shot on fisheye lenses though, for very good reasons, so this is more of an edge case.) Related: "there are no straight lines" -- if you're using a camera to make an image, you're operating in projective space. Whether a line is straight in 3D space, and is then straight in the projective space, is a function of your projection. Rectilinear lenses -- for which the relation f*tan(theta) applies -- straight lines remain straight. But for most (all?) other mappings they do not; for example, f*theta lenses preserve angular extent: an object that subtends a particular angle from the camera's perspective will occupy the same number of pixels regardless of where they appear in the image. This is arguably superior for many computer vision tasks. ... I need to stop here, because this is already getting tedious. For anyone interested in developing a deep intuition for lenses and imaging, I would recommend playing around with a view camera, understanding the Scheimpflug relationship (https://en.wikipedia.org/wiki/Scheimpflug_principle https://en.wikipedia.org/wiki/Scheimpflug_principle), and really thinking through imaging from first principles if you want to understand these things more deeply. Or just, you know, use Colmap and don't look back. That works too.

dcanelhas 2y ago
Mathematically, the camera most often used in computer vision is a pinhole camera. When we talk about "distortions" I think it's usually with regards to how the real device systematically deviates from that model. Calibration in this context is essentially the task of finding the optimal parameters of some (usually nonlinear) function (u,v)=f(x,y) that remaps positions in the original image frame to a rectified frame, where all straight lines in the world appear straight in the image. Technically, a skewed and squashed image would also fulfill those requirements, too. But this is a customer-oriented blog post, to give someone enough of an understanding to convince them of the importance of calibration, it's not a rigorous technical paper, so I actually think it's fine to skip/simplify some details.
- ryandamm 2y ago
  You are, of course, absolutely right. I nonetheless think it may have value for some people to understand where the deviations from that simplifying assumption lie, and at least understand the stakes. Now, I may be biased, because I work in imaging-for-humans, and I've had many a conversations with engineers about why a particular simplification doesn't work for, e.g., filmmakers, but I think that even for purely technical disciplines, understanding the assumptions that go into the pinhole model can be useful. At the margins. Which sometimes matter.
- ryandamm 2y ago
  A specific example of where it's useful to know the underlying mechanics: for very wide angle lenses, you will typically get brightness falloff at the edges due to a phenomenon called the cos^4 phenomenon (https://nvlpubs.nist.gov/nistpubs/jres/39/jresv39n3p213_A1b.pdf https://nvlpubs.nist.gov/nistpubs/jres/39/jresv39n3p213_A1b....). This is often elided by camera systems, that apply a gain to peripheral pixels to correct for this phenomenon. If you understand imaging, you will expect that, and understand why, for example, your wide angle lens displays a lower signal-to-noise ratio for a given illumination value than you might otherwise expect at the edges of the image. This is a really specific example, but there are dozens. Imaging is its own deep, technical field that is abstracted, and occasionally obscured, by the pinhole model.

dcanelhas 2y ago

Mathematically, the camera most often used in computer vision is a pinhole camera. When we talk about "distortions" I think it's usually with regards to how the real device systematically deviates from that model. Calibration in this context is essentially the task of finding the optimal parameters of some (usually nonlinear) function (u,v)=f(x,y) that remaps positions in the original image frame to a rectified frame, where all straight lines in the world appear straight in the image. Technically, a skewed and squashed image would also fulfill those requirements, too. But this is a customer-oriented blog post, to give someone enough of an understanding to convince them of the importance of calibration, it's not a rigorous technical paper, so I actually think it's fine to skip/simplify some details.

ryandamm 2y ago
You are, of course, absolutely right. I nonetheless think it may have value for some people to understand where the deviations from that simplifying assumption lie, and at least understand the stakes. Now, I may be biased, because I work in imaging-for-humans, and I've had many a conversations with engineers about why a particular simplification doesn't work for, e.g., filmmakers, but I think that even for purely technical disciplines, understanding the assumptions that go into the pinhole model can be useful. At the margins. Which sometimes matter.
ryandamm 2y ago
A specific example of where it's useful to know the underlying mechanics: for very wide angle lenses, you will typically get brightness falloff at the edges due to a phenomenon called the cos^4 phenomenon (https://nvlpubs.nist.gov/nistpubs/jres/39/jresv39n3p213_A1b.pdf https://nvlpubs.nist.gov/nistpubs/jres/39/jresv39n3p213_A1b....). This is often elided by camera systems, that apply a gain to peripheral pixels to correct for this phenomenon. If you understand imaging, you will expect that, and understand why, for example, your wide angle lens displays a lower signal-to-noise ratio for a given illumination value than you might otherwise expect at the edges of the image. This is a really specific example, but there are dozens. Imaging is its own deep, technical field that is abstracted, and occasionally obscured, by the pinhole model.