4 ms·

> Fused Video Stabilization on the Pixel 2 and Pixel 2 XL Aside from gatekeeping, what does any of this have to do with the Pixel 2? Why is the headline not "F

> Fused Video Stabilization on the Pixel 2 and Pixel 2 XL Aside from gatekeeping, what does any of this have to do with the Pixel 2? Why is the headline not "Fused video stabilization for any Android device with a camera"? I mean, Microsoft published inertial image deblurring using entirely unintegrated devices in 2010 (https://www.microsoft.com/en-us/research/publication/image-deblurring-using-inertial-measurement-sensors/ https://www.microsoft.com/en-us/research/publication/image-d...), though I swear I remember seeing a poster from them earlier than that. (Also, I can honestly say that I've never seen a video recorded on a phone that had motion blur or shutter wobble anywhere near as bad as what they show in their examples, so those examples seem contrived, but obviously it's great to have an algorithm that can handle exceptionally bad cases.)

TheAdamist 9y ago
Needs output from what the ois is correcting too for this to work, which I don't think most ois systems output.
- BugsJustFindMe 9y ago
  See, I don't believe that that is true. There might be some additional information gained there, but it's not _needed_, as evidence by a decade of publications on the subject of correcting motion artifacts.
  - CarVac 9y ago
    Without knowing where the lens is in relation to the sensor, you can't correct for the time-varying perspective distortion. Perhaps the decade of publications have never involved optical image stabilization in the loop. What you could have, however, is a simulation of where the OIS moves the optical axis of the lens to, when given the gyroscope inputs. In control theory terms, an open loop observer. But that would need to be calibrated to each model of phone.
    - BugsJustFindMe 9y ago
      > But that would need to be calibrated to each model of phone. We needn't suddenly forget that every android phone asks you to wave it around in figure eights to calibrate the compass (and then still only seems to be accurate to within +/- 180 degrees). Making every video you take come out better on every Android device with sufficient CPU power after a few seconds of recording would be significantly preferable to "Only works on our devices, neener neener." To me, obviously, maybe not to Google.
      - cromwellian 9y ago
        That kind of calibration isn't the same as factory calibration of IMU, camera, OIS systems against some known ground truth.
est 9y ago
I wonder how long will they release the android version of this https://www.microsoft.com/en-us/research/product/microsoftpix/ https://www.microsoft.com/en-us/research/product/microsoftpi...
puzzle 9y ago
There's a difference between running algorithms and running them without lag or battery drain. You need to tune the whole pipeline and keep every single step in sync, without unnecessary data copies or conversions, which is why Google spent the last few years designing its own image sensor processor, among others. It wasn't until recently that phones exposed raw pixels through the Camera2 API, for example. Some manufacturers don't want to go through the motions of exposing camera internals because of the extra work involved, some just want to keep their secret sauce secret.

TheAdamist 9y ago

Needs output from what the ois is correcting too for this to work, which I don't think most ois systems output.

BugsJustFindMe 9y ago
See, I don't believe that that is true. There might be some additional information gained there, but it's not _needed_, as evidence by a decade of publications on the subject of correcting motion artifacts.
- CarVac 9y ago
  Without knowing where the lens is in relation to the sensor, you can't correct for the time-varying perspective distortion. Perhaps the decade of publications have never involved optical image stabilization in the loop. What you could have, however, is a simulation of where the OIS moves the optical axis of the lens to, when given the gyroscope inputs. In control theory terms, an open loop observer. But that would need to be calibrated to each model of phone.
  - BugsJustFindMe 9y ago
    > But that would need to be calibrated to each model of phone. We needn't suddenly forget that every android phone asks you to wave it around in figure eights to calibrate the compass (and then still only seems to be accurate to within +/- 180 degrees). Making every video you take come out better on every Android device with sufficient CPU power after a few seconds of recording would be significantly preferable to "Only works on our devices, neener neener." To me, obviously, maybe not to Google.
    - cromwellian 9y ago
      That kind of calibration isn't the same as factory calibration of IMU, camera, OIS systems against some known ground truth.

est 9y ago

I wonder how long will they release the android version of this https://www.microsoft.com/en-us/research/product/microsoftpix/ https://www.microsoft.com/en-us/research/product/microsoftpi...

puzzle 9y ago

There's a difference between running algorithms and running them without lag or battery drain. You need to tune the whole pipeline and keep every single step in sync, without unnecessary data copies or conversions, which is why Google spent the last few years designing its own image sensor processor, among others. It wasn't until recently that phones exposed raw pixels through the Camera2 API, for example. Some manufacturers don't want to go through the motions of exposing camera internals because of the extra work involved, some just want to keep their secret sauce secret.