3 ms·

I wonder when will we get this quality http://pages.cs.wisc.edu/~fliu/project/3dstab.htm http://pages.cs.wisc.edu/~fliu/project/3dstab.htm

by hamoid 14y ago

MrMike 14y ago
This is amazing. Wonder if there is a commercial implementation yet?
- hedgehog 14y ago
  These guys do some pretty amazing stuff, video stabilization is one of the applications: http://www.2d3.com/ http://www.2d3.com/
  - gghh 14y ago
    Exactly, I also though of 2d3 as soon as I have seen that blog post. I came to know the company since it is linked by the (very valuable, if I may) book "Multiple View Geometry in Computer Vision", http://www.amazon.com/Multiple-View-Geometry-Computer-Vision/dp/0521540518/ http://www.amazon.com/Multiple-View-Geometry-Computer-Vision... . IIRC the company is run by Andrew Fitzgibbon http://research.microsoft.com/en-us/um/people/awf/ http://research.microsoft.com/en-us/um/people/awf/ , which now seems odd to me since he's a full time researcher at Microsoft. I wander how Andrew's algorithm in the 2d3.com product differs from the one published by the youtube guys in the post.
ChuckMcM 14y ago
Thanks for that link, this should have a submission of its own. I can easily see how something like this would make a 'point and shoot' video camera really useful. Think "Flip Camera meets James Cameron"
modeless 14y ago
Have you tried it? I have and I'd say the quality is pretty close if not the same. The much bigger problem to solve now is that shaky videos shot in less-than-perfect lighting contain motion blur, which is extremely hard to remove. You'll notice that all of these demo videos were conveniently shot outside in direct sunlight and contain no motion blur at all.
- hamoid 14y ago
  I have only tried the one offered by YouTube, and not recently. I don't know if they have improved the algorithm in this respect, but for what I've seen in the past, the filter often creates a very eerie wobbly effect on the video, an effect that makes it look fake, like being underwater or drunk. It is slightly visible in the demo video if you observe the borders. This strange effect is totally absent on the link I posted, which I believe it's on a different level of quality. But I imagine it's computationally very expensive and can't be offered to millions of users for free.
  - modeless 14y ago
    An eerie wobbly effect is definitely visible to me in the videos from that paper.
chriszf 14y ago
If you read the Google paper, you'll notice that they actually refer to this and other work by Liu et al. The overall technique is the same, estimate the original camera path, calculate an optimal camera path, retarget the input frames to a crop window that fits the optimal path. The primary difference seems to be estimation and calculation technique. Liu's work does a structure-from-motion reconstruction, ie: rebuild a 3d model of the original scene. Google's work uses something called pyramidal Lucas-Kanade to do 'feature tracking' instead. This is sort of localized reconstruction, it seems to only care about the viewport differences from frame to frame. They then feed it through some linear programming voodoo to get the best path. I don't understand either well enough to say why one is better than the other, although I'd guess it's because Lucas-Kanade is temporally and physically localized, it's easier to farm out to a parallel cluster than an SfM technique. There also seems to be a difference on the rear end of the technique, having feature detection allows them to add 'saliency' constraints, ie: retarget based on the inclusion of certain features, like a person's face. Again, the math is beyond my understanding, but it seems like this isn't part of Liu's work.

I wonder when will we get this quality http://pages.cs.wisc.edu/~fliu/project/3dstab.htm http://pages.cs.wisc.edu/~fliu/project/3dstab.htm

by hamoid 14y ago

I wonder when will we get this quality http://pages.cs.wisc.edu/~fliu/project/3dstab.htm http://pages.cs.wisc.edu/~fliu/project/3dstab.htm

MrMike 14y ago

This is amazing. Wonder if there is a commercial implementation yet?

hedgehog 14y ago
These guys do some pretty amazing stuff, video stabilization is one of the applications: http://www.2d3.com/ http://www.2d3.com/
- gghh 14y ago
  Exactly, I also though of 2d3 as soon as I have seen that blog post. I came to know the company since it is linked by the (very valuable, if I may) book "Multiple View Geometry in Computer Vision", http://www.amazon.com/Multiple-View-Geometry-Computer-Vision/dp/0521540518/ http://www.amazon.com/Multiple-View-Geometry-Computer-Vision... . IIRC the company is run by Andrew Fitzgibbon http://research.microsoft.com/en-us/um/people/awf/ http://research.microsoft.com/en-us/um/people/awf/ , which now seems odd to me since he's a full time researcher at Microsoft. I wander how Andrew's algorithm in the 2d3.com product differs from the one published by the youtube guys in the post.

ChuckMcM 14y ago

Thanks for that link, this should have a submission of its own. I can easily see how something like this would make a 'point and shoot' video camera really useful. Think "Flip Camera meets James Cameron"

modeless 14y ago

Have you tried it? I have and I'd say the quality is pretty close if not the same. The much bigger problem to solve now is that shaky videos shot in less-than-perfect lighting contain motion blur, which is extremely hard to remove. You'll notice that all of these demo videos were conveniently shot outside in direct sunlight and contain no motion blur at all.

hamoid 14y ago
I have only tried the one offered by YouTube, and not recently. I don't know if they have improved the algorithm in this respect, but for what I've seen in the past, the filter often creates a very eerie wobbly effect on the video, an effect that makes it look fake, like being underwater or drunk. It is slightly visible in the demo video if you observe the borders. This strange effect is totally absent on the link I posted, which I believe it's on a different level of quality. But I imagine it's computationally very expensive and can't be offered to millions of users for free.
- modeless 14y ago
  An eerie wobbly effect is definitely visible to me in the videos from that paper.

chriszf 14y ago

If you read the Google paper, you'll notice that they actually refer to this and other work by Liu et al. The overall technique is the same, estimate the original camera path, calculate an optimal camera path, retarget the input frames to a crop window that fits the optimal path. The primary difference seems to be estimation and calculation technique. Liu's work does a structure-from-motion reconstruction, ie: rebuild a 3d model of the original scene. Google's work uses something called pyramidal Lucas-Kanade to do 'feature tracking' instead. This is sort of localized reconstruction, it seems to only care about the viewport differences from frame to frame. They then feed it through some linear programming voodoo to get the best path. I don't understand either well enough to say why one is better than the other, although I'd guess it's because Lucas-Kanade is temporally and physically localized, it's easier to farm out to a parallel cluster than an SfM technique. There also seems to be a difference on the rear end of the technique, having feature detection allows them to add 'saliency' constraints, ie: retarget based on the inclusion of certain features, like a person's face. Again, the math is beyond my understanding, but it seems like this isn't part of Liu's work.