3 ms·

I'm disappointed they didn't demonstrate what happens when they take this system and expose it to a few real-world photos. Can it handle that, or has it been v

I'm disappointed they didn't demonstrate what happens when they take this system and expose it to a few real-world photos. Can it handle that, or has it been very much fitted to these shapes?

ankeshanand 8y ago
"Our method still has many limitations when compared to more traditional computer vision techniques, and has currently only been trained to work on synthetic scenes. However, as new sources of data become available and advances are made in our hardware capabilities, we expect to be able to investigate the application of the GQN framework to higher resolution images of real scenes."
- VikingCoder 8y ago
  "As new sources of data"... Like, StreetView?
  - amiga-workbench 8y ago
    That could be really interesting, StreetView but with this system imagining interpolated points between the original photos, giving the user more free and natural movement.
  - ehsankia 8y ago
    More important, more hardware, aka TPUs.
yagyu 8y ago
It has very much been fitted. The supplementary material describes the training set as 5 pictures each of 2M different scenes of the type you can see in the paper (square room with random objects). So to extrapolate wildly, it seems reasonable that getting similar results for, say, real-world bedrooms, you'd need to take around 5 pics each of 2M bedrooms, and record the location and angle of the camera for each picture. Edit: and I didn't mean to sound negative, using artificially generated rooms to develop the method is a great idea, and the next step will be narrow domain specific applications (they mention eg robotic arms) where it's feasible to automatically collect enough data for a task, and somewhere in the future we may have the data and compute to sample the distribution of real world environments in decent resolution..
- chrisfosterelli 8y ago
  I'd say much more than that, since real-world bedrooms likely have a much more complicated representation than more simple generated rooms.
  - yagyu 8y ago
    I agree with you that the estimate is conservative, and would depend strongly on image resolution and how broad your distribution of bedrooms is - only modern US style, or also 40 year old Japanese houses?
- visarga 8y ago
  This could be achieved with a drone camera and positioning sensors.
- maffydub 8y ago
  FYI, for artificially generated rooms, there is already the SceneNet RGB-D dataset - "5M Photorealistic Images of Synthetic Indoor Trajectories with Ground Truth" https://robotvault.bitbucket.io/scenenet-rgbd.html https://robotvault.bitbucket.io/scenenet-rgbd.html I think there are only 15k different rooms, rather than 2M you suggested, though!

I'm disappointed they didn't demonstrate what happens when they take this system and expose it to a few real-world photos. Can it handle that, or has it been very much fitted to these shapes?

ankeshanand 8y ago

"Our method still has many limitations when compared to more traditional computer vision techniques, and has currently only been trained to work on synthetic scenes. However, as new sources of data become available and advances are made in our hardware capabilities, we expect to be able to investigate the application of the GQN framework to higher resolution images of real scenes."

VikingCoder 8y ago
"As new sources of data"... Like, StreetView?
- amiga-workbench 8y ago
  That could be really interesting, StreetView but with this system imagining interpolated points between the original photos, giving the user more free and natural movement.
- ehsankia 8y ago
  More important, more hardware, aka TPUs.

yagyu 8y ago

It has very much been fitted. The supplementary material describes the training set as 5 pictures each of 2M different scenes of the type you can see in the paper (square room with random objects). So to extrapolate wildly, it seems reasonable that getting similar results for, say, real-world bedrooms, you'd need to take around 5 pics each of 2M bedrooms, and record the location and angle of the camera for each picture. Edit: and I didn't mean to sound negative, using artificially generated rooms to develop the method is a great idea, and the next step will be narrow domain specific applications (they mention eg robotic arms) where it's feasible to automatically collect enough data for a task, and somewhere in the future we may have the data and compute to sample the distribution of real world environments in decent resolution..

chrisfosterelli 8y ago
I'd say much more than that, since real-world bedrooms likely have a much more complicated representation than more simple generated rooms.
- yagyu 8y ago
  I agree with you that the estimate is conservative, and would depend strongly on image resolution and how broad your distribution of bedrooms is - only modern US style, or also 40 year old Japanese houses?
visarga 8y ago
This could be achieved with a drone camera and positioning sensors.
maffydub 8y ago
FYI, for artificially generated rooms, there is already the SceneNet RGB-D dataset - "5M Photorealistic Images of Synthetic Indoor Trajectories with Ground Truth" https://robotvault.bitbucket.io/scenenet-rgbd.html https://robotvault.bitbucket.io/scenenet-rgbd.html I think there are only 15k different rooms, rather than 2M you suggested, though!