4 ms·

Ive done my bachelor's thesis in 2013 on this. We were working on a robot built for exploration of disaster sites (earthquake, fire, ...). I built: 1) Virtual

by Scea91 8y ago

Ive done my bachelor's thesis in 2013 on this. We were working on a robot built for exploration of disaster sites (earthquake, fire, ...). I built: 1) Virtual camera: The robot was equipped with 6 fixed cameras. You can position virtual camera anywhere in the coordinate space and it constructs it's image with the information from the real cameras. 2) System for semi-automatic external calibration of cameras from LIDAR sensor. External calibration is the process of estimating the position and orientation of the camera. This system was not as precise as calibration using calibration patterns in lab, but it was useful when the robot was in the wild and you wanted to adjust the camera or strap a new camera on it quickly. However, to be frank I am not sure ft the system was ever actually used that way. I left the lab after finishing my thesis. The ROS ecosystem is reasonably simple to understand and definitely makes things much easier. Apart from having some issues with compatibility between versions and incomplete functionality I remember it as quite a nice platform. On the other hand I have not tried any of it's competitors if there are any.

amarant 8y ago
that's pretty cool! I'm particularly interested in how you made the virtual camera work? can I read your thesis, or (dare i ask) source code?
- Scea91 8y ago
  The virtual camera vas done during summer project but the idea IIRC is quite simple. I will try some hasty writeup ;) 1) Create a virtual image plane (or cylinder/sphere for panoramic camera) in the 3D space. For this you need to specify the virtual camera's position, orientation, focal length and FOV. 2) Define pixels on the image plane. You need to specify the image resolution and just create grid on the the image plane. Each pixel corresponds to a 3D point in the world coordinate system. 3) To get a color of each pixel you project it's corresponding 3D point to all real cameras. Usually only some of the cameras see the point. Each camera that sees the point gives you a color according to the color of the pixel in it's image. You have to perform blending of these colors. What worked well in our case was to give more weight to cameras where the point projected close to the center of their image. As you can see, there will definitely be some artifacts, usually more the farther away the virtual camera is from the real cameras. However, it worked to my surprise quite well. It was even possible to look at the robot from 3rd person even though all of the real cameras were on it. Also it is crucial to have well calibrated cameras. Apart from their projection matrices you also need to consider their radial and tangential distortion which was very significant with our camera models. https://www.wikiwand.com/en/Pinhole_camera_model https://www.wikiwand.com/en/Pinhole_camera_model https://www.wikiwand.com/en/Camera_matrix https://www.wikiwand.com/en/Camera_matrix https://www.wikiwand.com/en/Distortion_(optics) https://www.wikiwand.com/en/Distortion_(optics)

by Scea91 8y ago

amarant 8y ago

that's pretty cool! I'm particularly interested in how you made the virtual camera work? can I read your thesis, or (dare i ask) source code?

Scea91 8y ago
The virtual camera vas done during summer project but the idea IIRC is quite simple. I will try some hasty writeup ;) 1) Create a virtual image plane (or cylinder/sphere for panoramic camera) in the 3D space. For this you need to specify the virtual camera's position, orientation, focal length and FOV. 2) Define pixels on the image plane. You need to specify the image resolution and just create grid on the the image plane. Each pixel corresponds to a 3D point in the world coordinate system. 3) To get a color of each pixel you project it's corresponding 3D point to all real cameras. Usually only some of the cameras see the point. Each camera that sees the point gives you a color according to the color of the pixel in it's image. You have to perform blending of these colors. What worked well in our case was to give more weight to cameras where the point projected close to the center of their image. As you can see, there will definitely be some artifacts, usually more the farther away the virtual camera is from the real cameras. However, it worked to my surprise quite well. It was even possible to look at the robot from 3rd person even though all of the real cameras were on it. Also it is crucial to have well calibrated cameras. Apart from their projection matrices you also need to consider their radial and tangential distortion which was very significant with our camera models. https://www.wikiwand.com/en/Pinhole_camera_model https://www.wikiwand.com/en/Pinhole_camera_model https://www.wikiwand.com/en/Camera_matrix https://www.wikiwand.com/en/Camera_matrix https://www.wikiwand.com/en/Distortion_(optics) https://www.wikiwand.com/en/Distortion_(optics)