4 ms·

I was wondering why you choose to go with video data rather than photos. From what I know, for a given feature point, you would rather have pictures with real

I was wondering why you choose to go with video data rather than photos. From what I know, for a given feature point, you would rather have pictures with really different point of views. It seems to me video would generate a lot of redundant data, massively increasing processing times for very little gain. What's your experience on this?

Xelbair 7y ago
They probably are using drone/car/vehicle to capture the imaginery - at least that's the only reason i can think of - i used to do that back at the university. We recorded a video from a drone, split it into frames, and built the mesh using those frames. When drone is competently flown it creates a nice stripe of overlapping frames - and overlapping stripes(i don't know proper technical english translation for that term) of overlapping imaginery are basis of aerial photogrammetry. Worst case - you can keep every n-th photo, or use some more intelligent approach and select the frames based on contrast, and other parameters. Usually gathering data is the expensive part - in photogrammetry you try to take as much imaginery as possible - especially in traditional aerial photogrammetry.
- namibj 7y ago
  I'm aiming for software to digest the output of a 300fps tethered camera on a drone. Basically an Apertus Axiom beta with a fancier FPGA to use all 64 lanes of the sensor to not have only 150fps, and being able to attach a 40G QSFP+ transceiver to it for data offloading. Existing software makes hand-picked pictures worthwhile, but I want full-auto behavior. Near-term I want a construction worker to open the case with the hardware after cleaning his hands, and then showing all insides of the building to the camera, before swapping the storage and getting the data physically back to base for processing. Also, video capture gets you continuity of motion. Unordered picture collections are horribly expensive to compute with. I also prefer redundancy to holes in my textures or models ;) My drone swirls around the buildings, btw. I want VR for walking, not google earth.
  - Leherenn 7y ago
    Ok, so it's mostly a hardware issue then, a camera in burst mode continuously shooting at a lower FPS should theoretically work just as well. But indeed, if that works with a video, that's might be easier to use. You're talking about imu data, do you do geo-referencing also? It might be difficult having a GNSS fix if your target is inside buildings.
    - namibj 7y ago
      The point is that mechanical shutters in useful DSLRs have massive rolling shutter artifacts. You can get video cameras with shutter skew in the low microseconds or even sub-microsecond. Remember, the exposure has to be short due to motion blur. Concepts of resonant rotating camera heads that use torsion springs and electronically controlled clutches with a torsion oscillator that advances the camera in a few milliseconds were made, but apart from an ability to capture ~1 Gpixel effective 360° data in 2 seconds (there is overlap due to covering the sphere with rectangles and not changing the horizontal number of images per revolution (oscillation frequency) to reduce overlap at nadir and zenit), the benefits were considered insufficient to warrant more time on it before I actually _have_ the camera. And I'll probably have an fpga that is too bulky to oscillate with it, so I need fatigue-resistant cabling for the 64 + 2 clocks high-speed LVDS lanes between them. You can get e.g. the Blackmagic pocket cinema 4k that writes 30fps DCI4k DNG with an electronic global shutter to an USB-C SSD. I have no plans to rely on magnetic field sensors or GNSS. I plan to fuse bundle adjustment with constraints from the raw sensor data. Offset and drift correction for both will be done there, and thus I don't get the typical drift-off issues from cumulating a sensor offset. I _do_ plan to allow fixing feature markers in space, to handle geo-referencing and potential un-curling at the same time. (Depending on lens distortion you get variously strong tendencies for the area you scan to either reconstruct as a small hollow earth or a small spherical earth (in both cases the average surface curvature radius is up to a few km at most, often even less).)
namibj 7y ago
Just turn up the light and wave the camera faster, or, if you can't, maybe reduce fps if you need to conserve space at the time. Using only a covering subset of images with consistent redundancy for depth map reconstruction is easier after initial Structure-from-Motion than finding ways to recover from cases of under sampled geometry. And with video data the initial SfM isn't nearly as expensive as with unstructured picture collections, and it's even cheaper if you get accelerometer and gyroscope data from the camera to initialize the motion recovery part and help feature matching to skip known-not-overlapping views. You actually only use ~10 to ~40 degree view angles for depth map reconstruction, because of occlusion and texture distortion degrading results at high angles.

Xelbair 7y ago

They probably are using drone/car/vehicle to capture the imaginery - at least that's the only reason i can think of - i used to do that back at the university. We recorded a video from a drone, split it into frames, and built the mesh using those frames. When drone is competently flown it creates a nice stripe of overlapping frames - and overlapping stripes(i don't know proper technical english translation for that term) of overlapping imaginery are basis of aerial photogrammetry. Worst case - you can keep every n-th photo, or use some more intelligent approach and select the frames based on contrast, and other parameters. Usually gathering data is the expensive part - in photogrammetry you try to take as much imaginery as possible - especially in traditional aerial photogrammetry.

namibj 7y ago
I'm aiming for software to digest the output of a 300fps tethered camera on a drone. Basically an Apertus Axiom beta with a fancier FPGA to use all 64 lanes of the sensor to not have only 150fps, and being able to attach a 40G QSFP+ transceiver to it for data offloading. Existing software makes hand-picked pictures worthwhile, but I want full-auto behavior. Near-term I want a construction worker to open the case with the hardware after cleaning his hands, and then showing all insides of the building to the camera, before swapping the storage and getting the data physically back to base for processing. Also, video capture gets you continuity of motion. Unordered picture collections are horribly expensive to compute with. I also prefer redundancy to holes in my textures or models ;) My drone swirls around the buildings, btw. I want VR for walking, not google earth.
- Leherenn 7y ago
  Ok, so it's mostly a hardware issue then, a camera in burst mode continuously shooting at a lower FPS should theoretically work just as well. But indeed, if that works with a video, that's might be easier to use. You're talking about imu data, do you do geo-referencing also? It might be difficult having a GNSS fix if your target is inside buildings.
  - namibj 7y ago
    The point is that mechanical shutters in useful DSLRs have massive rolling shutter artifacts. You can get video cameras with shutter skew in the low microseconds or even sub-microsecond. Remember, the exposure has to be short due to motion blur. Concepts of resonant rotating camera heads that use torsion springs and electronically controlled clutches with a torsion oscillator that advances the camera in a few milliseconds were made, but apart from an ability to capture ~1 Gpixel effective 360° data in 2 seconds (there is overlap due to covering the sphere with rectangles and not changing the horizontal number of images per revolution (oscillation frequency) to reduce overlap at nadir and zenit), the benefits were considered insufficient to warrant more time on it before I actually _have_ the camera. And I'll probably have an fpga that is too bulky to oscillate with it, so I need fatigue-resistant cabling for the 64 + 2 clocks high-speed LVDS lanes between them. You can get e.g. the Blackmagic pocket cinema 4k that writes 30fps DCI4k DNG with an electronic global shutter to an USB-C SSD. I have no plans to rely on magnetic field sensors or GNSS. I plan to fuse bundle adjustment with constraints from the raw sensor data. Offset and drift correction for both will be done there, and thus I don't get the typical drift-off issues from cumulating a sensor offset. I _do_ plan to allow fixing feature markers in space, to handle geo-referencing and potential un-curling at the same time. (Depending on lens distortion you get variously strong tendencies for the area you scan to either reconstruct as a small hollow earth or a small spherical earth (in both cases the average surface curvature radius is up to a few km at most, often even less).)

namibj 7y ago

Just turn up the light and wave the camera faster, or, if you can't, maybe reduce fps if you need to conserve space at the time. Using only a covering subset of images with consistent redundancy for depth map reconstruction is easier after initial Structure-from-Motion than finding ways to recover from cases of under sampled geometry. And with video data the initial SfM isn't nearly as expensive as with unstructured picture collections, and it's even cheaper if you get accelerometer and gyroscope data from the camera to initialize the motion recovery part and help feature matching to skip known-not-overlapping views. You actually only use ~10 to ~40 degree view angles for depth map reconstruction, because of occlusion and texture distortion degrading results at high angles.