4 ms·

Uncertainty propagation. Richer models that fit better. Lots of feedback and metrics and visualization to evaluate the quality of the solve. Flexibility of the

by dima55 2y ago

Uncertainty propagation. Richer models that fit better. Lots of feedback and metrics and visualization to evaluate the quality of the solve. Flexibility of the tool. Documentation.

midjji 2y ago
Richer models that fit better are often a trap, in particular for beginners. Use the simplest model you can get away with. Unless you know what you are doing, most people are better of with a simpler model as it will be more robust to observability issues. If you dont know what those are, use a simpler model. Uncertainty propagation is very difficult to use for vision, and largely just modelling errors in vision as the error distributions, e.g. for anything observed or reconstructed from images are either subpixel accurate, or too non linear.
- dima55 2y ago
  Avoiding rich models is a great thing to do if you don't model uncertainty: a beginner that didn't get enough useful calibration data will see poor uncertainties in the results. So I now use the splined models in pretty much all applications, and there are very few downsides. In my experience, every lens fits noticeably better with the richer model (the mrcal validation shows you this explicitly). I think you should look at the tour of mrcal; it's friendly.
amelius 2y ago
A problem I always run into with OpenCV is that I need to preprocess the checkerboard images such that the lighting is just right. This is odd because that's the sort of thing I'd expect a computer vision library to excel at. Another problem I run into is that the transforms are ill-defined just outside of the screen. That means that if I want to draw e.g. a line in world coordinates onto the image from a camera, then I often get garbage if the line starts or ends outside the image (even if I divide the line into many segments).
- midjji 2y ago
  The corner detection is bad. Not sure why. I also think its gotten worse since 3.5. I know the corner detection refinement is worse than the raw detections, so turn that shit off. Yeah thats a common problem if the calibration failed. It could also be that you are not cropping to what is in front of the camera, but if its really weird, its most likely the former. So the default, and probably most of the camera models in opencv requires a monotonic change lenses and bijective imaging. The former is common unless the lense has defects on the surface, and the latter is practically a physical constraint. The problem is that these constraints are difficult to add to the estimator, so they didnt. Meaning it will find a solution where they are not satisfied. If say the bijectiveness is not satisfied a bit outside the image, but stil valid accounting for infront and float accuracy, then that would absolutely account for the problem you describe. Is pretty obvious if you consider the function what the problem is, just hard to add the constraint in opencvs estimator. The solution is 1, verify the result is satisfied after estimation, 2 make sure you make the parameters are as observable as possible during calibration. This means spread out in the image, evenly distributed, and all the way to the edges. Also make verify it has not rotated one or more of the detected chessboards upside down, or 90 deg sideways. Finally, because it becomes harder and harder to avoid this problem with more parameters, always start with 1, then try 2, then the two variations of 3, and so on. More parameters always fit better, so use an appropriate test.
- midjji 2y ago
  Also always do calibration in good light conditions.
- backes 2y ago
  I strongly agree with you on the first point. OpenCV provides calibration primitives which look like they'd solve the problem easily and this gives you false confidence. In my experience, they're very low-level and are not more than a wrapper around the optimization routine. You need to implement most things from scratch, such as correct exposure, checking for blurred frames, verifying the extracted checkerboard,... It's weird that everyone needs to re-invent this process. Can you be more specific on the second point? Once you have the intrinsics, it's trivial to project them to an (undistorted) image.

by dima55 2y ago

Uncertainty propagation. Richer models that fit better. Lots of feedback and metrics and visualization to evaluate the quality of the solve. Flexibility of the tool. Documentation.

midjji 2y ago

Richer models that fit better are often a trap, in particular for beginners. Use the simplest model you can get away with. Unless you know what you are doing, most people are better of with a simpler model as it will be more robust to observability issues. If you dont know what those are, use a simpler model. Uncertainty propagation is very difficult to use for vision, and largely just modelling errors in vision as the error distributions, e.g. for anything observed or reconstructed from images are either subpixel accurate, or too non linear.

dima55 2y ago
Avoiding rich models is a great thing to do if you don't model uncertainty: a beginner that didn't get enough useful calibration data will see poor uncertainties in the results. So I now use the splined models in pretty much all applications, and there are very few downsides. In my experience, every lens fits noticeably better with the richer model (the mrcal validation shows you this explicitly). I think you should look at the tour of mrcal; it's friendly.

amelius 2y ago

A problem I always run into with OpenCV is that I need to preprocess the checkerboard images such that the lighting is just right. This is odd because that's the sort of thing I'd expect a computer vision library to excel at. Another problem I run into is that the transforms are ill-defined just outside of the screen. That means that if I want to draw e.g. a line in world coordinates onto the image from a camera, then I often get garbage if the line starts or ends outside the image (even if I divide the line into many segments).

midjji 2y ago
The corner detection is bad. Not sure why. I also think its gotten worse since 3.5. I know the corner detection refinement is worse than the raw detections, so turn that shit off. Yeah thats a common problem if the calibration failed. It could also be that you are not cropping to what is in front of the camera, but if its really weird, its most likely the former. So the default, and probably most of the camera models in opencv requires a monotonic change lenses and bijective imaging. The former is common unless the lense has defects on the surface, and the latter is practically a physical constraint. The problem is that these constraints are difficult to add to the estimator, so they didnt. Meaning it will find a solution where they are not satisfied. If say the bijectiveness is not satisfied a bit outside the image, but stil valid accounting for infront and float accuracy, then that would absolutely account for the problem you describe. Is pretty obvious if you consider the function what the problem is, just hard to add the constraint in opencvs estimator. The solution is 1, verify the result is satisfied after estimation, 2 make sure you make the parameters are as observable as possible during calibration. This means spread out in the image, evenly distributed, and all the way to the edges. Also make verify it has not rotated one or more of the detected chessboards upside down, or 90 deg sideways. Finally, because it becomes harder and harder to avoid this problem with more parameters, always start with 1, then try 2, then the two variations of 3, and so on. More parameters always fit better, so use an appropriate test.
midjji 2y ago
Also always do calibration in good light conditions.
backes 2y ago
I strongly agree with you on the first point. OpenCV provides calibration primitives which look like they'd solve the problem easily and this gives you false confidence. In my experience, they're very low-level and are not more than a wrapper around the optimization routine. You need to implement most things from scratch, such as correct exposure, checking for blurred frames, verifying the extracted checkerboard,... It's weird that everyone needs to re-invent this process. Can you be more specific on the second point? Once you have the intrinsics, it's trivial to project them to an (undistorted) image.