7 ms·
Fused Video Stabilization on the Pixel 2 and Pixel 2 XL
- isoprophlex 9y agoI still think most software video stabilization looks terrible and unnatural. What's wrong with focus breathing and perspective shifts?
- zo1 9y agoIt looks unnatural because we're used to something different. If stabilized video was the norm, I'm sure we'd feel differently.
- readams 9y agoThe terribleness and unnaturalness is precisely the problem they are trying to solve with this solution. The results look very impressive compared to previous solutions.
- xster 9y agoIt's like fisheye and ultra-HDRs. Nothing wrong with them. Just too different from how our natural lenses and retinas work.
- BugsJustFindMe 9y ago> Fused Video Stabilization on the Pixel 2 and Pixel 2 XL Aside from gatekeeping, what does any of this have to do with the Pixel 2? Why is the headline not "Fused video stabilization for any Android device with a camera"? I mean, Microsoft published inertial image deblurring using entirely unintegrated devices in 2010 (https://www.microsoft.com/en-us/research/publication/image-deblurring-using-inertial-measurement-sensors/ https://www.microsoft.com/en-us/research/publication/image-d...), though I swear I remember seeing a poster from them earlier than that. (Also, I can honestly say that I've never seen a video recorded on a phone that had motion blur or shutter wobble anywhere near as bad as what they show in their examples, so those examples seem contrived, but obviously it's great to have an algorithm that can handle exceptionally bad cases.)
- TheAdamist 9y agoNeeds output from what the ois is correcting too for this to work, which I don't think most ois systems output.
- BugsJustFindMe 9y agoSee, I don't believe that that is true. There might be some additional information gained there, but it's not _needed_, as evidence by a decade of publications on the subject of correcting motion artifacts.
- CarVac 9y agoWithout knowing where the lens is in relation to the sensor, you can't correct for the time-varying perspective distortion. Perhaps the decade of publications have never involved optical image stabilization in the loop. What you could have, however, is a simulation of where the OIS moves the optical axis of the lens to, when given the gyroscope inputs. In control theory terms, an open loop observer. But that would need to be calibrated to each model of phone.
- BugsJustFindMe 9y ago> But that would need to be calibrated to each model of phone. We needn't suddenly forget that every android phone asks you to wave it around in figure eights to calibrate the compass (and then still only seems to be accurate to within +/- 180 degrees). Making every video you take come out better on every Android device with sufficient CPU power after a few seconds of recording would be significantly preferable to "Only works on our devices, neener neener." To me, obviously, maybe not to Google.
- cromwellian 9y agoThat kind of calibration isn't the same as factory calibration of IMU, camera, OIS systems against some known ground truth.
- codinghorror 9y agoThis -- is super freakin' cool. Real value add from Google, considering everyone is on the same very mediocre Qualcomm Snapdragon 835 platform. (Please PLEASE for the love of God, Google, build your own mobile SoCs so someone can remotely catch up to Apple's 3 year lead in mobile performance already) Credible too because Pixel 2 had such stellar picture quality in the reviews.
- efraim 9y agoI'm curious why you think Google would be better at building SoC's than Qualcomm?
- codinghorror 9y agoAnyone has to be better than Qualcomm, I hope. Their performance is just embarrassing at this point, relative to where Apple is. Compare: https://browser.geekbench.com/android-benchmarks https://browser.geekbench.com/android-benchmarks https://browser.geekbench.com/ios-benchmarks https://browser.geekbench.com/ios-benchmarks 4214 versus 1965. And that's assuming you own the very very very latest Android device from this year. It gets far worse the deeper / older you go in years.
- efraim 9y agoQualcomm is seriously lagging Apple but so are Samsung and HiSilicon[0][1], they seem to be about the same performance. Android Authority [2] tried to explain why Apple is faster and their best argument (among a few) seems to be that Apple's silicon is just larger. Qualcomm could probably make larger chips if they get payed for it. [0] https://www.anandtech.com/show/11540/samsung-galaxy-s8-exynos-versus-snapdragon https://www.anandtech.com/show/11540/samsung-galaxy-s8-exyno... [1] https://www.anandtech.com/show/11088/hisilicon-kirin-960-performance-and-power https://www.anandtech.com/show/11088/hisilicon-kirin-960-per... [2] https://www.androidauthority.com/why-are-apples-chips-faster-than-qualcomms-gary-explains-802738/ https://www.androidauthority.com/why-are-apples-chips-faster...
- 9y ago
- demarq 9y agoI knew there was quite a bit of processing going on in flagship cameras but I never expected this level. Really impressed!
- vizzah 9y agoiPhone user looking with envy. Is anything of a similar magnitude available in a standalone software package? For already recorded videos?
- ssully 9y agoIdk about videos, but Google Photos automatically stableizes live photos on iOS. Didn't realize it did so until about a month ago. The results aren't always great, but it's cool when it works well. When it doesn't it's one button click to turn it off.
- CharlesW 9y ago> iPhone user looking with envy. I'm not sure FOMO is warranted, since Apple's so-called 'cinematic video stabilization' (introduced in the iPhone 6 days) is quite good. Additionally, the iPhone 6 Plus, 7 Plus, and both iPhone 8 models have optical image stabilization.
- xster 9y agoI think the whole point of the article is that simple OIS isn't enough.
- nileshk 9y agoBoth iPhone 7 models have OIS as well
- ClassyJacket 9y agoAlso the iPhone 7 has OIS.
- Klathmon 9y agoThat motorcycle video near the end of the article is simply amazing.
- rasz 9y agoso were the demos of Youtube video stabilization, in reality Youtube EIS is absolute garbage.
- boundlessdreamz 9y agoIt seems to work quite well. See https://youtu.be/0Mp5Y3UTWH8?t=93 https://youtu.be/0Mp5Y3UTWH8?t=93
- sandGorgon 9y agoWay too many things that are Pixel only. Does this mean that the Android open source era is over ?
- mozumder 9y agoTwo things left to improve for mobile phone video that would massively increase image quality: increased dynamic range and higher bit rate. Increasing dynamic range in going to be tough. The only way to to do that would be to make the pixel's electron well bigger, but phone cameras have limited pixel size. Image sensors will probably have to provide a separate stacked electron well capacitor below each pixel, instead of using the photosensor PN junction. Also, the industry really needs a standard raw video format that allows for high-quality post exposure correction and white-balance.
- exodust 9y agoCropping is still happening, and it can gobble up your content if you're not careful. Even in their demo video of guy jumping down sand dunes, we see the destructive price of 'fused cropping stabilisation'. The most important shot of guy jumping misses the moment his feet hit sand at 2 sec in when he does his biggest leap. It's details like these, and the clouds of sand he kicks up behind him which make an interesting shot. I like my focal length untouched, thanks anyway, optical stabilisation all the way for me. Here's the sand dune video. https://youtu.be/kaO7Gs-F2vA https://youtu.be/kaO7Gs-F2vA I wonder if there's any manual control over crop factor in the Pixel settings.
- microcolonel 9y agoSomething I'd like to see is a square sensor, so you can take horizontal video in portrait if you forget.