4 ms·

I've got a smartphone with a not so good camera. Some time ago I heard about an app that could take a picture of a template and then would calculate the distort

by thdrdt 6y ago

I've got a smartphone with a not so good camera. Some time ago I heard about an app that could take a picture of a template and then would calculate the distortions of your camera and fix those (with ML). Does anyone know what app/research this is? I can't imagine why smartphone maker don't use this for their crappy camera's so they can still embed cheap camera's but enhance the picture quality.

numpad0 6y ago
They do? Just costs engineer hours to implement. So only fancier of potatoes get them. The area of research that covers from HDR to smartphone array cameras is called Computational Photography, I think.
- thdrdt 6y ago
  Well the demo I saw showed it was just a matter of taking a picture of a printed template and the software would figure out the distortion. It shouldn't take too much effort to implement the correction data in the camera software.
  - ddek 6y ago
    Does the issue become individual corrections? On the one hand, Lightroom corrects my lens distortion and vignette from a saved profile of that lens; but I’d think at a smartphone level, where the parts are smaller, they’re also less even? In other words, camera lenses are small so manufacturing defects are more noticeable. Therefore, each smartphone requires a different correction. I don’t know, I’m just guessing.
    - csharptwdec19 6y ago
      > Does the issue become individual corrections? > In other words, camera lenses are small so manufacturing defects are more noticeable. Therefore, each smartphone requires a different correction. Well, yes, and no, and yes. Small camera lenses are probably going to be more evident in certain defects. On the other hand, there are typically less elements in smaller camera lenses, which is nice as the less of those there are, the less the chance of other defects. Taking a picture like mentioned above could absolutely create a 'per-camera/phone' correction profile. May not be able to correct every type of defect but I could see it being useful for some lenses. I know for Sony cameras, there have been a couple models where a large percentage of the copies produced have -one- soft corner.
      - jacobush 6y ago
        Some of the smallest plastic lenses have geometries and optical properties which are impossible to make in glass or in larges plastic pieces. So smaller lenses don't have only disadvantages to them. (Even though I myself personally rather take a pound of glass any day over these things.)
    - Kye 6y ago
      Not 100% equivalent, but from what I’ve heard, this can be an issue when using full frame lenses on a crop sensor DSLR. The sensor gets everything from a smaller part of the lens, so defects that don’t show on a 35mm equivalent sensor can show at APS-C.
      - CarVac 6y ago
        When you use a full-frame lens on a crop sensor: * Vignetting always improves * First-order distortion improves, second-order distortion sometimes gets worse. * Corner sharpness usually improves, unless midway out on the frame was the weakest performance on full frame. * Center sharpness (image-scale) drops due to the additional enlargement. * Lateral chromatic aberration generally doesn't get significantly better or worse. * Longitudinal chromatic aberration becomes more visible because of the additional enlargement.
    - CarVac 6y ago
      I think Google Pixels are individually calibrated.
phonon 6y ago
Adobe has pre-configured profiles for most major phones fyi https://helpx.adobe.com/camera-raw/kb/supported-lenses.html https://helpx.adobe.com/camera-raw/kb/supported-lenses.html
hikarudo 6y ago
> (with ML) Nitpick, that's just camera calibration. No ML needed.
- thdrdt 6y ago
  Well I thought they used ML to calibrate it (the correction matrix) but I am not sure about this.
  - chapplap 6y ago
    It's not too complicated, at least in theory. Typically you decide a lens model ahead of time that has some unknown calibration parameters (eg. focal length, skew, radial/tangential distortion). For a given lens/sensor system, the calibration parameters can be determined ahead of time by taking pictures of a calibration pattern (checkerboard, AprilGrid, etc.). The calibration pattern provides some ground truth distances between multiple points in the image. An off-the-shelf non-linear solver is then used to solve for the unknown calibration parameters. This usually provides acceptable results for applications like correcting lens distortion, even across multiple instances of the same lens. However, sometimes better accuracy is needed - either because the lens is very cheap and inconsistent between different examples or for applications like SLAM where better camera calibration translates directly into better results. In that case there are techniques like online calibration to tweak the calibration parameters on the fly.
jimfleming 6y ago
This functionality is built into OpenCV[0]. If you're using a reference image (or if you know the lens properties) it doesn't require ML. It's mostly just a matrix transform. [0] https://docs.opencv.org/2.4/doc/tutorials/calib3d/camera_calibration/camera_calibration.html https://docs.opencv.org/2.4/doc/tutorials/calib3d/camera_cal...
- thdrdt 6y ago
  Ah yes! It looks like this is related to what I was looking for. It was a long time ago that I saw a demo about this and I was under the impression that ML was used to calculate the correction matrix.

by thdrdt 6y ago

numpad0 6y ago

They do? Just costs engineer hours to implement. So only fancier of potatoes get them. The area of research that covers from HDR to smartphone array cameras is called Computational Photography, I think.

thdrdt 6y ago
Well the demo I saw showed it was just a matter of taking a picture of a printed template and the software would figure out the distortion. It shouldn't take too much effort to implement the correction data in the camera software.
- ddek 6y ago
  Does the issue become individual corrections? On the one hand, Lightroom corrects my lens distortion and vignette from a saved profile of that lens; but I’d think at a smartphone level, where the parts are smaller, they’re also less even? In other words, camera lenses are small so manufacturing defects are more noticeable. Therefore, each smartphone requires a different correction. I don’t know, I’m just guessing.
  - csharptwdec19 6y ago
    > Does the issue become individual corrections? > In other words, camera lenses are small so manufacturing defects are more noticeable. Therefore, each smartphone requires a different correction. Well, yes, and no, and yes. Small camera lenses are probably going to be more evident in certain defects. On the other hand, there are typically less elements in smaller camera lenses, which is nice as the less of those there are, the less the chance of other defects. Taking a picture like mentioned above could absolutely create a 'per-camera/phone' correction profile. May not be able to correct every type of defect but I could see it being useful for some lenses. I know for Sony cameras, there have been a couple models where a large percentage of the copies produced have -one- soft corner.
    - jacobush 6y ago
      Some of the smallest plastic lenses have geometries and optical properties which are impossible to make in glass or in larges plastic pieces. So smaller lenses don't have only disadvantages to them. (Even though I myself personally rather take a pound of glass any day over these things.)
  - Kye 6y ago
    Not 100% equivalent, but from what I’ve heard, this can be an issue when using full frame lenses on a crop sensor DSLR. The sensor gets everything from a smaller part of the lens, so defects that don’t show on a 35mm equivalent sensor can show at APS-C.
    - CarVac 6y ago
      When you use a full-frame lens on a crop sensor: * Vignetting always improves * First-order distortion improves, second-order distortion sometimes gets worse. * Corner sharpness usually improves, unless midway out on the frame was the weakest performance on full frame. * Center sharpness (image-scale) drops due to the additional enlargement. * Lateral chromatic aberration generally doesn't get significantly better or worse. * Longitudinal chromatic aberration becomes more visible because of the additional enlargement.
  - CarVac 6y ago
    I think Google Pixels are individually calibrated.

phonon 6y ago

Adobe has pre-configured profiles for most major phones fyi https://helpx.adobe.com/camera-raw/kb/supported-lenses.html https://helpx.adobe.com/camera-raw/kb/supported-lenses.html

hikarudo 6y ago

> (with ML) Nitpick, that's just camera calibration. No ML needed.

thdrdt 6y ago
Well I thought they used ML to calibrate it (the correction matrix) but I am not sure about this.
- chapplap 6y ago
  It's not too complicated, at least in theory. Typically you decide a lens model ahead of time that has some unknown calibration parameters (eg. focal length, skew, radial/tangential distortion). For a given lens/sensor system, the calibration parameters can be determined ahead of time by taking pictures of a calibration pattern (checkerboard, AprilGrid, etc.). The calibration pattern provides some ground truth distances between multiple points in the image. An off-the-shelf non-linear solver is then used to solve for the unknown calibration parameters. This usually provides acceptable results for applications like correcting lens distortion, even across multiple instances of the same lens. However, sometimes better accuracy is needed - either because the lens is very cheap and inconsistent between different examples or for applications like SLAM where better camera calibration translates directly into better results. In that case there are techniques like online calibration to tweak the calibration parameters on the fly.

jimfleming 6y ago

This functionality is built into OpenCV[0]. If you're using a reference image (or if you know the lens properties) it doesn't require ML. It's mostly just a matrix transform. [0] https://docs.opencv.org/2.4/doc/tutorials/calib3d/camera_calibration/camera_calibration.html https://docs.opencv.org/2.4/doc/tutorials/calib3d/camera_cal...

thdrdt 6y ago
Ah yes! It looks like this is related to what I was looking for. It was a long time ago that I saw a demo about this and I was under the impression that ML was used to calculate the correction matrix.