4 ms·

Mathematically, the camera most often used in computer vision is a pinhole camera. When we talk about "distortions" I think it's usually with regards to how the

Mathematically, the camera most often used in computer vision is a pinhole camera. When we talk about "distortions" I think it's usually with regards to how the real device systematically deviates from that model. Calibration in this context is essentially the task of finding the optimal parameters of some (usually nonlinear) function (u,v)=f(x,y) that remaps positions in the original image frame to a rectified frame, where all straight lines in the world appear straight in the image. Technically, a skewed and squashed image would also fulfill those requirements, too. But this is a customer-oriented blog post, to give someone enough of an understanding to convince them of the importance of calibration, it's not a rigorous technical paper, so I actually think it's fine to skip/simplify some details.

ryandamm 2y ago
You are, of course, absolutely right. I nonetheless think it may have value for some people to understand where the deviations from that simplifying assumption lie, and at least understand the stakes. Now, I may be biased, because I work in imaging-for-humans, and I've had many a conversations with engineers about why a particular simplification doesn't work for, e.g., filmmakers, but I think that even for purely technical disciplines, understanding the assumptions that go into the pinhole model can be useful. At the margins. Which sometimes matter.
ryandamm 2y ago
A specific example of where it's useful to know the underlying mechanics: for very wide angle lenses, you will typically get brightness falloff at the edges due to a phenomenon called the cos^4 phenomenon (https://nvlpubs.nist.gov/nistpubs/jres/39/jresv39n3p213_A1b.pdf https://nvlpubs.nist.gov/nistpubs/jres/39/jresv39n3p213_A1b....). This is often elided by camera systems, that apply a gain to peripheral pixels to correct for this phenomenon. If you understand imaging, you will expect that, and understand why, for example, your wide angle lens displays a lower signal-to-noise ratio for a given illumination value than you might otherwise expect at the edges of the image. This is a really specific example, but there are dozens. Imaging is its own deep, technical field that is abstracted, and occasionally obscured, by the pinhole model.
- midjji 2y ago
  Or you know, vignetting
tripletao 2y ago
The "rectilinear" or "f*tan(theta)" projection mentioned in the grandparent comment is equivalent to that pinhole camera. The former name is because the pinhole camera preserves straight lines, as noted. The latter is because if theta is the angle between the incoming ray of light and the lens's optical axis, and f is the focal length, then the pixel illuminated by that ray of light is at a distance f*tan(theta) from the center of the imager. That rectilinear projection is indeed the most popular choice, but all projections involve tradeoffs as the FOV gets bigger, in the same way that all planar cartographic projections involve tradeoffs as the depicted region gets bigger. For example, the magnification of objects at the edges of a rectilinear projection gets extreme as the FOV approaches 180 degrees, and the projection stops existing entirely at or beyond that. That magnification is sometimes called "perspective distortion", even though it's inherent to the rectilinear projection. Wide-angle lenses or multi-lens arrays will often deliberately choose f*theta instead, to avoid that "perspective distortion" or support FOV >= 180 degrees. Other projections (e.g. equirectangular) are also used, especially for stuff like panoramas and VR. The concept of distortion is meaningful only with respect to a desired baseline, which is often but not always rectilinear.
- midjji 2y ago
  Rectification is not always the best choice, its a simple one for data processing in some cases, but often limiting. Keeping to a simple model is much more important.
backes 2y ago
> When we talk about "distortions" I think it's usually with regards to how the real device systematically deviates from that model. IMO this is the correct definition of distortion. However, as the parent comment said: > If you are not designing a rectilinear lens, there are other lens mappings, in which case it's not really proper to describe the effect as distortion, though in technical literature it's often still described this way. I think many people confuse mapping and distortion. When a fisheye lens is used, it's often seen as "heavy distortion". But a more accurate way should be to say that it's a different mapping/projection, and the _distortion_ a calibration measures is the difference of this ideal projection (ftheta (e.g. Kannala Brandt) rather than fthan*theta (pinhole) ), and the actual image. This can be a minuscule amount. This means that, "undistorting" a fisheye image doesn't give you a rectilinear image, but still a fisheye image. You can of course decide to map the undistorted fisheye image to a rectilinear one, but that's conceptually a different operation than (un)distortion.

ryandamm 2y ago

You are, of course, absolutely right. I nonetheless think it may have value for some people to understand where the deviations from that simplifying assumption lie, and at least understand the stakes. Now, I may be biased, because I work in imaging-for-humans, and I've had many a conversations with engineers about why a particular simplification doesn't work for, e.g., filmmakers, but I think that even for purely technical disciplines, understanding the assumptions that go into the pinhole model can be useful. At the margins. Which sometimes matter.

ryandamm 2y ago

A specific example of where it's useful to know the underlying mechanics: for very wide angle lenses, you will typically get brightness falloff at the edges due to a phenomenon called the cos^4 phenomenon (https://nvlpubs.nist.gov/nistpubs/jres/39/jresv39n3p213_A1b.pdf https://nvlpubs.nist.gov/nistpubs/jres/39/jresv39n3p213_A1b....). This is often elided by camera systems, that apply a gain to peripheral pixels to correct for this phenomenon. If you understand imaging, you will expect that, and understand why, for example, your wide angle lens displays a lower signal-to-noise ratio for a given illumination value than you might otherwise expect at the edges of the image. This is a really specific example, but there are dozens. Imaging is its own deep, technical field that is abstracted, and occasionally obscured, by the pinhole model.

midjji 2y ago
Or you know, vignetting

tripletao 2y ago

The "rectilinear" or "f*tan(theta)" projection mentioned in the grandparent comment is equivalent to that pinhole camera. The former name is because the pinhole camera preserves straight lines, as noted. The latter is because if theta is the angle between the incoming ray of light and the lens's optical axis, and f is the focal length, then the pixel illuminated by that ray of light is at a distance f*tan(theta) from the center of the imager. That rectilinear projection is indeed the most popular choice, but all projections involve tradeoffs as the FOV gets bigger, in the same way that all planar cartographic projections involve tradeoffs as the depicted region gets bigger. For example, the magnification of objects at the edges of a rectilinear projection gets extreme as the FOV approaches 180 degrees, and the projection stops existing entirely at or beyond that. That magnification is sometimes called "perspective distortion", even though it's inherent to the rectilinear projection. Wide-angle lenses or multi-lens arrays will often deliberately choose f*theta instead, to avoid that "perspective distortion" or support FOV >= 180 degrees. Other projections (e.g. equirectangular) are also used, especially for stuff like panoramas and VR. The concept of distortion is meaningful only with respect to a desired baseline, which is often but not always rectilinear.

midjji 2y ago
Rectification is not always the best choice, its a simple one for data processing in some cases, but often limiting. Keeping to a simple model is much more important.

backes 2y ago

> When we talk about "distortions" I think it's usually with regards to how the real device systematically deviates from that model. IMO this is the correct definition of distortion. However, as the parent comment said: > If you are not designing a rectilinear lens, there are other lens mappings, in which case it's not really proper to describe the effect as distortion, though in technical literature it's often still described this way. I think many people confuse mapping and distortion. When a fisheye lens is used, it's often seen as "heavy distortion". But a more accurate way should be to say that it's a different mapping/projection, and the _distortion_ a calibration measures is the difference of this ideal projection (ftheta (e.g. Kannala Brandt) rather than fthan*theta (pinhole) ), and the actual image. This can be a minuscule amount. This means that, "undistorting" a fisheye image doesn't give you a rectilinear image, but still a fisheye image. You can of course decide to map the undistorted fisheye image to a rectilinear one, but that's conceptually a different operation than (un)distortion.