4 ms·

I assume this uses GPS data to help with stitching the photos together. Do drones capture orientation data beyond GPS position (for example, pitch, yaw, gimbal

I assume this uses GPS data to help with stitching the photos together. Do drones capture orientation data beyond GPS position (for example, pitch, yaw, gimbal direction)? I have tried adding motion graphics to some of my drone videos in After Effects but as far as I can tell it discards all of the metadata and infers the camera position based solely on the video frames. This is really slow and imprecise; I’ve been wondering if there’s a plug-in that will use the recorded telemetry from the drone to set the camera orientation.

kaennar 8y ago
In the lab I work in we have internal tools for doing something like this for precision agriculture, but I have never heard of an open source solution. UAVs generally don't record their orientation or GPS data without setting up a separate log file (I have no idea how you do this on Pixhawks. We use BYU's autopilot or our own autopilot).
- unbearded 8y ago
  DJI drones stores that on the image itself. No need to use the logs.
ViktorV 8y ago
I don't think it uses GPS for that. You can match two partially overlapping images. From the movement of the feature points in the overlapping areas you can calculate your camera orientation, and thus the 3d point cloud of feature points ( or something close to this ). I think it is called structure from movement, the readme links a paper, maybe it's worth reading for you.
- yeldarb 8y ago
  Interesting, is there a technical reason for not using that extra data to help with the process? For example, another commenter mentioned > Some materials don't really contain any surface details that the algorithms could use to attach feature points to, so they will be blank. Large white walls and large windows are especially difficult. Seems like you might be able to position some of these with orientation data from the camera.
  - Fission 8y ago
    The GPS data is probably being used as a prior. The GPS metadata is accurate but imprecise, while SFM more precise. GPS is likely used for pose initialization, while SFM is then used to refine the pose.
- gloflo 8y ago
  Would a high-quality pre-registration via precise coordinates and orientation yield better (and faster) results though? If the algorithm did not have to guess the camera parameters I would imagine a benefit.
  - Fission 8y ago
    In practice, there are a couple of ways to handle this. In one case, EXIF data gives you a lot of information (sensor size, camera/drone make, etc.) that you can extract very good camera intrinsics from. This information can also be fed into big EXIF databases, from which you can obtain pretty accurate camera parameters. Additionally, camera intrinsic calibration is a relatively solved problem, especially if you know that every single camera has the same intrinsics.
unbearded 8y ago
I'm familiar only with the DJI drones and they do capture Pitch, Yaw and Gimbal and a few other variables as well. It stores them on the "extended" part of the EXIF.
punnerud 8y ago
OpenDroneMap use a text file where you specify image name and pixels with their corresponding GPS coordinates for those pixels. Pitch, yaw, gimbal direction can probably help you with optimizing those algorithm. I am thinking of using OpenCV for image recognition of a large “chess board”, from there it should be easy to get both the GPS-coordinates and pixels in the images seeing this calibration board.
mohaine 8y ago
All ODM needs is a centroid Lat/Lon from the EXIF tags on each photo. You can optionally add a text file that gives a pixel an exact Lat/Lon. From this it matches features from the photos to calculate the exact pitch, yaw, direction. I think you can specify those in the EXIF but it is really hard (i.e. expensive) to get these to high enough quality to render the final image. In our case the high quality gimbals where more than our drones. (~$15k each)
Leherenn 8y ago
Depends on the drones, but professional photogrammetry drones certainly do. You often have position (latitude, longitude and altitude in WGS-84), attitude (yaw, pitch, roll) which already contains the gimbal orientation (e.g. the drone applies the lever arm to go from the IMU position to the camera optical center and output those) along with their covariances (accuracy) as well as timestamping. If the camera has a rolling shutter, you might even have imu data (i.e. accelerations) to correct distortions. Another important data for the reconstruction is the focal length. Ultimately, it is the job of the reconstruction software to use those data. Usually, they use them to initialise the process, and use the covariances as constraints, then they run their optimiser based on that. If your accuracies were too tight, the reconstruction will often fail as they trust you on the accuracies. However, the better/more accurate the data (e.g. RTK data), the faster/easier it is for the software to do its job.

kaennar 8y ago

In the lab I work in we have internal tools for doing something like this for precision agriculture, but I have never heard of an open source solution. UAVs generally don't record their orientation or GPS data without setting up a separate log file (I have no idea how you do this on Pixhawks. We use BYU's autopilot or our own autopilot).

unbearded 8y ago
DJI drones stores that on the image itself. No need to use the logs.

ViktorV 8y ago

I don't think it uses GPS for that. You can match two partially overlapping images. From the movement of the feature points in the overlapping areas you can calculate your camera orientation, and thus the 3d point cloud of feature points ( or something close to this ). I think it is called structure from movement, the readme links a paper, maybe it's worth reading for you.

yeldarb 8y ago
Interesting, is there a technical reason for not using that extra data to help with the process? For example, another commenter mentioned > Some materials don't really contain any surface details that the algorithms could use to attach feature points to, so they will be blank. Large white walls and large windows are especially difficult. Seems like you might be able to position some of these with orientation data from the camera.
- Fission 8y ago
  The GPS data is probably being used as a prior. The GPS metadata is accurate but imprecise, while SFM more precise. GPS is likely used for pose initialization, while SFM is then used to refine the pose.
gloflo 8y ago
Would a high-quality pre-registration via precise coordinates and orientation yield better (and faster) results though? If the algorithm did not have to guess the camera parameters I would imagine a benefit.
- Fission 8y ago
  In practice, there are a couple of ways to handle this. In one case, EXIF data gives you a lot of information (sensor size, camera/drone make, etc.) that you can extract very good camera intrinsics from. This information can also be fed into big EXIF databases, from which you can obtain pretty accurate camera parameters. Additionally, camera intrinsic calibration is a relatively solved problem, especially if you know that every single camera has the same intrinsics.

unbearded 8y ago

I'm familiar only with the DJI drones and they do capture Pitch, Yaw and Gimbal and a few other variables as well. It stores them on the "extended" part of the EXIF.

punnerud 8y ago

OpenDroneMap use a text file where you specify image name and pixels with their corresponding GPS coordinates for those pixels. Pitch, yaw, gimbal direction can probably help you with optimizing those algorithm. I am thinking of using OpenCV for image recognition of a large “chess board”, from there it should be easy to get both the GPS-coordinates and pixels in the images seeing this calibration board.

mohaine 8y ago

All ODM needs is a centroid Lat/Lon from the EXIF tags on each photo. You can optionally add a text file that gives a pixel an exact Lat/Lon. From this it matches features from the photos to calculate the exact pitch, yaw, direction. I think you can specify those in the EXIF but it is really hard (i.e. expensive) to get these to high enough quality to render the final image. In our case the high quality gimbals where more than our drones. (~$15k each)

Leherenn 8y ago

Depends on the drones, but professional photogrammetry drones certainly do. You often have position (latitude, longitude and altitude in WGS-84), attitude (yaw, pitch, roll) which already contains the gimbal orientation (e.g. the drone applies the lever arm to go from the IMU position to the camera optical center and output those) along with their covariances (accuracy) as well as timestamping. If the camera has a rolling shutter, you might even have imu data (i.e. accelerations) to correct distortions. Another important data for the reconstruction is the focal length. Ultimately, it is the job of the reconstruction software to use those data. Usually, they use them to initialise the process, and use the covariances as constraints, then they run their optimiser based on that. If your accuracies were too tight, the reconstruction will often fail as they trust you on the accuracies. However, the better/more accurate the data (e.g. RTK data), the faster/easier it is for the software to do its job.