4 ms·

Combining open-source photogrammetry software into a system that scales reasonably to >100k images (with accelerometer/gyroscope data from the video camera) and

by namibj 7y ago

Combining open-source photogrammetry software into a system that scales reasonably to >100k images (with accelerometer/gyroscope data from the video camera) and >1G vertices in a single, consistent, high-quality mesh. It later computes a texture map based on the images, not smoothing them, but just minimizing seams and using some seam carving and color interpolation to make most seams invisible. I want VR reconstructions that don't lack details to be feasible for <1k$/acre flat area, not counting someone steering the camera. - Places need to be archived like the rest of our culture. [Current sub-part:] I'm figuring out how https://www.gcc.tu-darmstadt.de/home/proj/tsr/tsr.en.jsp https://www.gcc.tu-darmstadt.de/home/proj/tsr/tsr.en.jsp can be combined with https://igl.ethz.ch/projects/instant-meshes/ https://igl.ethz.ch/projects/instant-meshes/ to get better meshes (with less wasted vertices) and also how this can be made to scale Out-of-Core or on a HPC cluster (the kind with lots of RDMA interconnect bandwidth and low latency). AMA (I won't respond for a few hours)

jcims 7y ago
I am planning a new building on my property and really would love to have something like this to visualize it from different angles on the driveway, back porch, etc.
- namibj 7y ago
  I understand you want to visualize the current state? And have a free camera you can move around the scene? If so, this should be able to do it. If you can get your hands on a camera with electronic global shutter and more than 1080p, I'd offer you processing in exchange for being able to later use the data for testing / tuning some of the more recent multi-view stereo algorithms (well, their implementations). Contact jcims-hn-reconstruction on d10.pw (beware of the digits in the domain) and I'll get you a sample result, if you're interested. Email should work ;)
  - jcims 7y ago
    Whoa this is an awesome offer, thank you! I'm arranging for some oblique and overhead shots around the property, will hit you up once they are available to see if it makes sense to give it a go. Thanks!
    - namibj 7y ago
      If you want good results, I'd advise considering needs of photogrammetry before the day you take the photos. I just don't want to post a public link to the example I mentioned. Depending on how you take those shots/arrange for them there might be tricks to get (closer to) certain goals. Feel free to mail me some questions regarding the shots/arrangements you mention.
Leherenn 7y ago
I was wondering why you choose to go with video data rather than photos. From what I know, for a given feature point, you would rather have pictures with really different point of views. It seems to me video would generate a lot of redundant data, massively increasing processing times for very little gain. What's your experience on this?
- Xelbair 7y ago
  They probably are using drone/car/vehicle to capture the imaginery - at least that's the only reason i can think of - i used to do that back at the university. We recorded a video from a drone, split it into frames, and built the mesh using those frames. When drone is competently flown it creates a nice stripe of overlapping frames - and overlapping stripes(i don't know proper technical english translation for that term) of overlapping imaginery are basis of aerial photogrammetry. Worst case - you can keep every n-th photo, or use some more intelligent approach and select the frames based on contrast, and other parameters. Usually gathering data is the expensive part - in photogrammetry you try to take as much imaginery as possible - especially in traditional aerial photogrammetry.
  - namibj 7y ago
    I'm aiming for software to digest the output of a 300fps tethered camera on a drone. Basically an Apertus Axiom beta with a fancier FPGA to use all 64 lanes of the sensor to not have only 150fps, and being able to attach a 40G QSFP+ transceiver to it for data offloading. Existing software makes hand-picked pictures worthwhile, but I want full-auto behavior. Near-term I want a construction worker to open the case with the hardware after cleaning his hands, and then showing all insides of the building to the camera, before swapping the storage and getting the data physically back to base for processing. Also, video capture gets you continuity of motion. Unordered picture collections are horribly expensive to compute with. I also prefer redundancy to holes in my textures or models ;) My drone swirls around the buildings, btw. I want VR for walking, not google earth.
    - Leherenn 7y ago

by namibj 7y ago

jcims 7y ago

I am planning a new building on my property and really would love to have something like this to visualize it from different angles on the driveway, back porch, etc.

namibj 7y ago
I understand you want to visualize the current state? And have a free camera you can move around the scene? If so, this should be able to do it. If you can get your hands on a camera with electronic global shutter and more than 1080p, I'd offer you processing in exchange for being able to later use the data for testing / tuning some of the more recent multi-view stereo algorithms (well, their implementations). Contact jcims-hn-reconstruction on d10.pw (beware of the digits in the domain) and I'll get you a sample result, if you're interested. Email should work ;)
- jcims 7y ago
  Whoa this is an awesome offer, thank you! I'm arranging for some oblique and overhead shots around the property, will hit you up once they are available to see if it makes sense to give it a go. Thanks!
  - namibj 7y ago
    If you want good results, I'd advise considering needs of photogrammetry before the day you take the photos. I just don't want to post a public link to the example I mentioned. Depending on how you take those shots/arrange for them there might be tricks to get (closer to) certain goals. Feel free to mail me some questions regarding the shots/arrangements you mention.

Leherenn 7y ago

I was wondering why you choose to go with video data rather than photos. From what I know, for a given feature point, you would rather have pictures with really different point of views. It seems to me video would generate a lot of redundant data, massively increasing processing times for very little gain. What's your experience on this?

Xelbair 7y ago
They probably are using drone/car/vehicle to capture the imaginery - at least that's the only reason i can think of - i used to do that back at the university. We recorded a video from a drone, split it into frames, and built the mesh using those frames. When drone is competently flown it creates a nice stripe of overlapping frames - and overlapping stripes(i don't know proper technical english translation for that term) of overlapping imaginery are basis of aerial photogrammetry. Worst case - you can keep every n-th photo, or use some more intelligent approach and select the frames based on contrast, and other parameters. Usually gathering data is the expensive part - in photogrammetry you try to take as much imaginery as possible - especially in traditional aerial photogrammetry.
- namibj 7y ago
  I'm aiming for software to digest the output of a 300fps tethered camera on a drone. Basically an Apertus Axiom beta with a fancier FPGA to use all 64 lanes of the sensor to not have only 150fps, and being able to attach a 40G QSFP+ transceiver to it for data offloading. Existing software makes hand-picked pictures worthwhile, but I want full-auto behavior. Near-term I want a construction worker to open the case with the hardware after cleaning his hands, and then showing all insides of the building to the camera, before swapping the storage and getting the data physically back to base for processing. Also, video capture gets you continuity of motion. Unordered picture collections are horribly expensive to compute with. I also prefer redundancy to holes in my textures or models ;) My drone swirls around the buildings, btw. I want VR for walking, not google earth.
  - Leherenn 7y ago