5 ms·

A hard problem, but back in 2008, Microsoft Photosynth could stitch together a user’s collection of images into a 3-D model. I remember seeing a detailed model

A hard problem, but back in 2008, Microsoft Photosynth could stitch together a user’s collection of images into a 3-D model. I remember seeing a detailed model of the exterior of the Notre Dame cathedral. For people who have physical access to a property, I wonder if you could use a variant of SLAM like the robot vacs - maybe hack one of those laser tape measures to plug into your phone as you walk around the house.

TaylorAlexander 5y ago
Well if you take video or many photos of a house you can create a 3D model using photogrammetry. I was recently served an ad showing that I think Redfin includes 3D photogrammetry captures of some homes. So it’s perhaps already being done in the wild.
- jpgleeson 5y ago
  There are a number of companies operating in the space. Matterport is one some people in my work have been exploring for documenting projects in various stages of construction. There are definitely still a bunch of rough edges, but the results so far even just using a phone to capture the spaces have been acceptable to good.
  - TaylorAlexander 5y ago
    Nice! Glad to hear it’s not just for research nerds these days. I’ve done a fair bit of experimentation with photogrammetry and it does work well for the built environment. I have an application however where I want to be able to capture an outdoor forest scene with a video camera and convert that to a video game/simulation scene which renders efficiently. It turns out that is a more challenging problem that actually no one has solved as far as I know. There’s lots of neat research tho and it will be possible some day. This bit of research is pretty cool: https://youtu.be/JuH79E8rdKc https://youtu.be/JuH79E8rdKc
    - madhawav 5y ago
      This is actually a very interesting video. Reflection effects look very realistic.
      - TaylorAlexander 5y ago
        Yah it’s cool stuff! Currently NERF takes a very long time to calculate and render but folks are working on improving that.
        madhawav 5y ago
        Do you think its possible to decouple lighting effects from the generated 3D scene? Currently it seem to simply copy the actual lighting condition in the environment. In a way, it's a limitation. It's much better if it can provide the RGB color + other material information (SVBRDF decomposition). If that's possible, then these meshes can be used in other environments.
        TaylorAlexander 5y ago
        I think its definitely possible, though I don’t know if the algorithms exist now. You would probably need to capture the scene under different lighting conditions, and probably from different angles. But assuming there is some theoretical way to go from photos to a mathematical representation of material characteristics I’d think we’ll eventually have neural nets that can do the conversion.
rpvnwnkl 5y ago
Photosynth was a great app, really disappointed when it was discontinued. Does anyone have the backstory on that? It seemed to have a leg up on Google’s street view/cardboard app at the time.
semi-extrinsic 5y ago
Bringing a laser tape measure and a pen to a physical viewing, and just doing this manually, gets you 98% of the way. It's not automatic of course, but you can do it pretty discretely and cover a whole house with measurements in ~ 10 minutes. I did it every single time when we were looking at houses. Also always brought a powerful flashlight and a small spirit level.

TaylorAlexander 5y ago

Well if you take video or many photos of a house you can create a 3D model using photogrammetry. I was recently served an ad showing that I think Redfin includes 3D photogrammetry captures of some homes. So it’s perhaps already being done in the wild.

jpgleeson 5y ago
There are a number of companies operating in the space. Matterport is one some people in my work have been exploring for documenting projects in various stages of construction. There are definitely still a bunch of rough edges, but the results so far even just using a phone to capture the spaces have been acceptable to good.
- TaylorAlexander 5y ago
  Nice! Glad to hear it’s not just for research nerds these days. I’ve done a fair bit of experimentation with photogrammetry and it does work well for the built environment. I have an application however where I want to be able to capture an outdoor forest scene with a video camera and convert that to a video game/simulation scene which renders efficiently. It turns out that is a more challenging problem that actually no one has solved as far as I know. There’s lots of neat research tho and it will be possible some day. This bit of research is pretty cool: https://youtu.be/JuH79E8rdKc https://youtu.be/JuH79E8rdKc
  - madhawav 5y ago
    This is actually a very interesting video. Reflection effects look very realistic.
    - TaylorAlexander 5y ago
      Yah it’s cool stuff! Currently NERF takes a very long time to calculate and render but folks are working on improving that.
      - madhawav 5y ago
        Do you think its possible to decouple lighting effects from the generated 3D scene? Currently it seem to simply copy the actual lighting condition in the environment. In a way, it's a limitation. It's much better if it can provide the RGB color + other material information (SVBRDF decomposition). If that's possible, then these meshes can be used in other environments.
        TaylorAlexander 5y ago
        I think its definitely possible, though I don’t know if the algorithms exist now. You would probably need to capture the scene under different lighting conditions, and probably from different angles. But assuming there is some theoretical way to go from photos to a mathematical representation of material characteristics I’d think we’ll eventually have neural nets that can do the conversion.

rpvnwnkl 5y ago

Photosynth was a great app, really disappointed when it was discontinued. Does anyone have the backstory on that? It seemed to have a leg up on Google’s street view/cardboard app at the time.

semi-extrinsic 5y ago

Bringing a laser tape measure and a pen to a physical viewing, and just doing this manually, gets you 98% of the way. It's not automatic of course, but you can do it pretty discretely and cover a whole house with measurements in ~ 10 minutes. I did it every single time when we were looking at houses. Also always brought a powerful flashlight and a small spirit level.