25 ms·

> plus data on the camera angles they were taken from Doesn't seem like much of a stretch to determine the angles as well. E.g. a semi brute forced way with G

by simsla 5y ago

> plus data on the camera angles they were taken from Doesn't seem like much of a stretch to determine the angles as well. E.g. a semi brute forced way with GANs

c4wrd 5y ago
I've spent a lot of time thinking about this (i.e. taking a video and creating a 3D scene) and I don't think that it is feasible in most cases to have good accuracy. If you need to infer the angle, you need make a lot of biased assumptions about things like velocity, position, etc., of the camera and even if you were 99.9% accurate, that 0.1% inaccuracy is compounded over time. Now I'm not saying it's not possible, but I'd believe that if you want an accurate 3D scene, you'd rather be spending your computation budget on things other than determining those angles when it can be simply be provided by hardware.
- krasin 5y ago
  https://github.com/NVLabs/instant-ngp https://github.com/NVLabs/instant-ngp has a script that converts a video into frames and then uses COLMAP ([1]) to compute camera poses. You can then train a NeRF model within a few seconds. It all works pretty well. Trying it on your own video is pretty straightforward. 1. https://colmap.github.io/ https://colmap.github.io/
- riotnrrd 5y ago
  You're far too pessimistic (or maybe you don't know the field well). The problem of estimating the relative poses of the cameras responsible for a set of photos is a long standing and essentially "solved" problem in computer vision. I say "solved" because there is still active research (increasing accuracy, faster, more robust, etc.) but there are decades-old, well known techniques that any dedicated programmer could implement in a week. If you're genuinely curious, look into structure from motion, visual odometry, or SLAM.
- doliveira 5y ago
  > even if you were 99.9% accurate, that 0.1% inaccuracy is compounded over time Not really, with SLAM there are various algorithms to keep inaccuracy in check. Basically it works by a feedback loop of guessing an estimate for position and then updating it using landmarks.
riotnrrd 5y ago
You don't even need anything that fancy. Traditional structure-from-motion, or visual odometry gives accurate enough position estimations. If you want to experiment, take a bunch (~100) of photos of an object, and use COLMAP to generate the poses. COLMAP implements a global SfM technique, so it will be very accurate but very slow.

by simsla 5y ago

> plus data on the camera angles they were taken from Doesn't seem like much of a stretch to determine the angles as well. E.g. a semi brute forced way with GANs

c4wrd 5y ago

I've spent a lot of time thinking about this (i.e. taking a video and creating a 3D scene) and I don't think that it is feasible in most cases to have good accuracy. If you need to infer the angle, you need make a lot of biased assumptions about things like velocity, position, etc., of the camera and even if you were 99.9% accurate, that 0.1% inaccuracy is compounded over time. Now I'm not saying it's not possible, but I'd believe that if you want an accurate 3D scene, you'd rather be spending your computation budget on things other than determining those angles when it can be simply be provided by hardware.

krasin 5y ago
https://github.com/NVLabs/instant-ngp https://github.com/NVLabs/instant-ngp has a script that converts a video into frames and then uses COLMAP ([1]) to compute camera poses. You can then train a NeRF model within a few seconds. It all works pretty well. Trying it on your own video is pretty straightforward. 1. https://colmap.github.io/ https://colmap.github.io/
riotnrrd 5y ago
You're far too pessimistic (or maybe you don't know the field well). The problem of estimating the relative poses of the cameras responsible for a set of photos is a long standing and essentially "solved" problem in computer vision. I say "solved" because there is still active research (increasing accuracy, faster, more robust, etc.) but there are decades-old, well known techniques that any dedicated programmer could implement in a week. If you're genuinely curious, look into structure from motion, visual odometry, or SLAM.
doliveira 5y ago
> even if you were 99.9% accurate, that 0.1% inaccuracy is compounded over time Not really, with SLAM there are various algorithms to keep inaccuracy in check. Basically it works by a feedback loop of guessing an estimate for position and then updating it using landmarks.

riotnrrd 5y ago

You don't even need anything that fancy. Traditional structure-from-motion, or visual odometry gives accurate enough position estimations. If you want to experiment, take a bunch (~100) of photos of an object, and use COLMAP to generate the poses. COLMAP implements a global SfM technique, so it will be very accurate but very slow.