6 ms·

"Structure from motion" refers to the process of reconstructing camera poses and a sparse point cloud from a set of images (or video). That's the input to the s

"Structure from motion" refers to the process of reconstructing camera poses and a sparse point cloud from a set of images (or video). That's the input to the scene reconstruction process described in this paper. As far as I know, the basic approach to SfM hasn't changed much in the last decade or so. It boils down to image feature extraction (using something like SIFT feature vectors), then a heuristic matching process, then bundle adjustment and outlier rejection.

sneak 3y ago
My application is for taking videos of rooms and turning them into to-scale models in a 3d modeling software (to be quantized and re-textured manually). Basically a rough floor plan generator from walkthrough videos. Is there something off the shelf that does this, or a library that can generate these point clouds so I can go about trying to fit simple geometry to them to reconstruct the scene? Please forgive the noob questions; I've not worked with automated 3d stuff or photogrammetry before.
- blensor 3y ago
  Take a look at Meshroom from https://alicevision.org/ https://alicevision.org/ It's a pretty decent tool, although not sure if videos work well. I once tried to build a mesh from an aerial video by extracting certain frames and the result was ok-ish
- CyberDildonics 3y ago
  into to-scale models What does this mean? Why would scale matter in a 3D model when you can scale it however you want in two seconds?
  - mkl 3y ago
    "Scale model" means things are correctly sized relative to each other, e.g. your square room doesn't end up rectangular.
    - CyberDildonics 3y ago
      Scale model generally means you make something smaller than the real thing. Also they said 'to-scale', but none of this applies, since if you are doing some sort of photogrametry you are getting an accurate model of what you are photographing anyway, that's the entire point.
      - robotresearcher 3y ago
        Conventional photogrammetry gives an 'up to scale' model, i.e. the scale is arbitrary and needs to set later to match reality. This is because a real cow far away and a toy cow close up look the same in pictures. You can supplement images with lidar, stereo images, or IMU odometry to capture true scale information at the SfM stage to get (approximately) scale-correct models. Alternatively you can learn what the real world tends to look like and estimate absolute depth from images alone, if you have enough data. This might be defeated by toy cows and doll houses, but work for some applications.
        CyberDildonics 3y ago
        gives an 'up to scale' model, This is not a term that makes sense. It makes a model, there is no sense in having it scaled in any way except for being correct. Saying 'up to scale' is like saying 'feet running' or 'mouth eating' or 'face talking'. the scale is arbitrary Only if you don't measure anything. If you have the camera correct and know how high something is or just know some depth or distances photogrametry makes an accurate model. 'Scale model' is a term from when scaling the model down made it easier to make. In the computer, it just means it isn't accurate.
        robotresearcher 3y ago
        It's actually the technical term that is used in this context. It's used when you don't know the scale. You can estimate all the geometry 'up to' but not including a scale term. > and know how high something is or just know some depth or distances There you go: you just added in some extra information to constrain the scale estimate as I described.
- tomp 3y ago
  No. I recently tested exactly this use case, with iPhone apps 3D Scanner, Polycam and magicplan. The results were, how do I put it nicely, not very useful. For example, the floor wasn't even flat/straight! One would think this would be a basic constraint (or “inductive bias”) of the algorithm… I haven’t tested Luma yet; also my example wasn't literally the most basic one (square empty room), instead there was some clutter on the floor, irregularly shaped room, a chair in the middle of the room, and open doors.
- robotresearcher 3y ago
  Apple device only, but RoomPlan will do geometry estimation directly from the sensors. https://developer.apple.com/videos/play/wwdc2023/10192/ https://developer.apple.com/videos/play/wwdc2023/10192/

sneak 3y ago

My application is for taking videos of rooms and turning them into to-scale models in a 3d modeling software (to be quantized and re-textured manually). Basically a rough floor plan generator from walkthrough videos. Is there something off the shelf that does this, or a library that can generate these point clouds so I can go about trying to fit simple geometry to them to reconstruct the scene? Please forgive the noob questions; I've not worked with automated 3d stuff or photogrammetry before.

blensor 3y ago
Take a look at Meshroom from https://alicevision.org/ https://alicevision.org/ It's a pretty decent tool, although not sure if videos work well. I once tried to build a mesh from an aerial video by extracting certain frames and the result was ok-ish
CyberDildonics 3y ago
into to-scale models What does this mean? Why would scale matter in a 3D model when you can scale it however you want in two seconds?
- mkl 3y ago
  "Scale model" means things are correctly sized relative to each other, e.g. your square room doesn't end up rectangular.
  - CyberDildonics 3y ago
    Scale model generally means you make something smaller than the real thing. Also they said 'to-scale', but none of this applies, since if you are doing some sort of photogrametry you are getting an accurate model of what you are photographing anyway, that's the entire point.
    - robotresearcher 3y ago
      Conventional photogrammetry gives an 'up to scale' model, i.e. the scale is arbitrary and needs to set later to match reality. This is because a real cow far away and a toy cow close up look the same in pictures. You can supplement images with lidar, stereo images, or IMU odometry to capture true scale information at the SfM stage to get (approximately) scale-correct models. Alternatively you can learn what the real world tends to look like and estimate absolute depth from images alone, if you have enough data. This might be defeated by toy cows and doll houses, but work for some applications.
      - CyberDildonics 3y ago
        gives an 'up to scale' model, This is not a term that makes sense. It makes a model, there is no sense in having it scaled in any way except for being correct. Saying 'up to scale' is like saying 'feet running' or 'mouth eating' or 'face talking'. the scale is arbitrary Only if you don't measure anything. If you have the camera correct and know how high something is or just know some depth or distances photogrametry makes an accurate model. 'Scale model' is a term from when scaling the model down made it easier to make. In the computer, it just means it isn't accurate.
        robotresearcher 3y ago
        It's actually the technical term that is used in this context. It's used when you don't know the scale. You can estimate all the geometry 'up to' but not including a scale term. > and know how high something is or just know some depth or distances There you go: you just added in some extra information to constrain the scale estimate as I described.
tomp 3y ago
No. I recently tested exactly this use case, with iPhone apps 3D Scanner, Polycam and magicplan. The results were, how do I put it nicely, not very useful. For example, the floor wasn't even flat/straight! One would think this would be a basic constraint (or “inductive bias”) of the algorithm… I haven’t tested Luma yet; also my example wasn't literally the most basic one (square empty room), instead there was some clutter on the floor, irregularly shaped room, a chair in the middle of the room, and open doors.
robotresearcher 3y ago
Apple device only, but RoomPlan will do geometry estimation directly from the sensors. https://developer.apple.com/videos/play/wwdc2023/10192/ https://developer.apple.com/videos/play/wwdc2023/10192/