4 ms·
It's striking but it's far more believable when you realize that they need to play at a much faster speed than the source, so they have tons of extra samples fr
by pron 12y ago
It's striking but it's far more believable when you realize that they need to play at a much faster speed than the source, so they have tons of extra samples from which to extract information. They basically use all that data in the extra frames (that would otherwise simply get tossed away in a regular time-lapse) to construct a 3D scene. This wouldn't look nearly as good if they had to play it at normal speed.
- z303 12y agoTake a look at an older paper 'Content-Preserving Warps for 3D Video Stabilization'. That seems to do an ok job http://createdigitalmotion.com/2009/06/magical-3d-warping-techniques-steadies-your-videos/ http://createdigitalmotion.com/2009/06/magical-3d-warping-te...
- vanderZwan 12y agoEDIT: After clicking through to their website, I found this updated technique which shares one principal author that seems to work even better: http://people.cs.nctu.edu.tw/~yushuen/VideoStabilization/ http://people.cs.nctu.edu.tw/~yushuen/VideoStabilization/ Wow. Actually, if they that add that technique to the mix it might solve the deformed "pop"-effect you see in some videos, like the deformed building you see around 16 seconds into this videos: http://research.microsoft.com/en-us/um/redmond/projects/hyperlapse/supplementary/html/bike1_original_speed.html http://research.microsoft.com/en-us/um/redmond/projects/hype...
- z303 12y agoCool, I'd not seen the updated work. I wonder how much can be done in realtime. I have no idea what the compute split is between between the different processes. With sensors (gyros etc) the camera path would be trivial, instead of recovering that from the video. Rendering the results would be possible on a mobile GPU. Just leaving the frame conversion to a point cloud in terms of compute and memory. Maybe some scheme where you down sample the input frames to create the deformation mesh, then apply that to the full size frame would be the way to go
- vanderZwan 12y ago> With sensors (gyros etc) the camera path would be trivial, instead of recovering that from the video. Well... not quite trivial. They're calibrated differently per model, and it's actually quite tricky to reconstruct the path based on accelerometers and gyroscope alone. There's also the likely issue of synchronising the data from these sensors with the video input. If you solve that second issue however, it could in theory at least help with recovering the path from the video, creating better predictions where the point cloud has moved to for example.
- z303 12y agoI was being a little cheeky with that. I know position and orientation data can be quite noisy and getting clean good quality data is not simple
- bane 12y agoOn the flip side, it sounds like shooting at higher fps and using this to end up with a 24 or 60 fps result might end up with some cool results.
- deleted 12y ago[deleted]
- Osmium 12y agoCamera movement between each frame would be minimal though so there'd be a lot of overlap between frames so minimal extra information. I'd guess the key to improving this result would be multiple cameras at different angles; I imagine it only works as well as it does because the GoPro uses a fisheye lens.
- derengel 12y ago"This wouldn't look nearly as good if they had to play it at normal speed." For normal speed you wouldn't need this :)
- schafx 12y agoThis technique actually works great for normal speed and can mimic the smoothness of a cinematographer's expensive dolly: https://www.youtube.com/watch?v=3TlCGh5Pc90 https://www.youtube.com/watch?v=3TlCGh5Pc90