6 ms·

Gyroflow: Video stabilization using gyroscope data

179 points by nateb2022 5mo ago

Gigachad 5mo ago
I wanted to try this but sadly it seems my A7 iii doesn't record gyro data.
- Zanfa 5mo ago
  There are external devices that can be attached to the camera to record gyro data, e.g https://docs.gyroflow.xyz/app/advanced-usage/using-external-gyro-source/niyien-a1 https://docs.gyroflow.xyz/app/advanced-usage/using-external-... I just ordered one a few weeks ago and haven't received it yet, so can't talk about personal experience, but there's no technical reason it couldn't work well.
  - JosephRedfern 5mo ago
    Out of curiosity - how do you sync gyro data with video timecode?
- nnevatie 5mo ago
  My a7CR seems to be supported - will try this out later.
  - tecleandor 5mo ago
    Some months ago I tested footage of my A7C2 on Gyroflow expecting to do visual stabilization, and found my camera has a gyro (didn't know about it) and automatically adds that info to the videos. The result was pretty good and it was super easy to do it.
    - mikae1 5mo ago
      Just be sure to turn off any in camera stabilization to get the best results.
      - Gigachad 5mo ago
        In the guide videos they show you just have to sync it up by pressing the record button and end record button on both the gyroscope and the camera at the same time and this is apparently enough to calculate the time difference between the two clocks.
        tecleandor 5mo ago
        I think mikae1 refers to the lens/sensor stabilization, as it can be problematic if you combine it with other stabilization systems. My camera already embeds gyro information automatically in the video files without intervention (I didn't even know about it).
nerderloo 5mo ago
Such a fine piece of software.
jauntywundrkind 5mo ago
That this also helps so much with rolling shutter correction is wild! Epic.
rixed 5mo ago
How is this different from usual sensor stabilisation techniques? Is it because it can adjust for a wider range of motion?
- Gigachad 5mo ago
  Normal sensor stabilization only moves a tiny amount, It's more useful for photography for reducing micro jitters to get sharp photos. For video you need much more aggressive cropping and warping to undo the massive shake of walking with the camera.
  - Mars008 5mo ago
    [dead]
po1nt 5mo ago
As a drone pilot I used this before davinci studio implemented something similar. I highly recommend.
FrostKiwi 5mo ago
So So useful <3 Stabilization = you have to zoom-in, loss of FOV. Depending on Action Cam and undistortion parameters, this can be different, sometimes too little, sometimes too much. Gyroflow allows you to dial it in. Lot's Stabilization in a particularly shaky spot and widest FOV everywhere else, smoothed between to be unnoticeable.
hdersch 5mo ago
I wonder how this compares to purely vision-based systems which use nothing but the images themselves for stabilization. Here are some quite old results of stabilization using image-based 3d-reconstruction of the scene which I wrote more than 10years ago, compared with other stabilization programs of that time (Deshaker, Adobe After Effects, Youtube). With todays improved hardware and progress in 3d-algorithms you may not need any additional gyroscopic data. https://www.youtube.com/watch?v=-m3fwhx3Z5g https://www.youtube.com/watch?v=-m3fwhx3Z5g
- _flux 5mo ago
  I think gyroscopic data still can have the edge if it has higher sampling rate than the video: then it could be used for removing blur from individual frames. I also expect purely gyroscopic approach to be much lighter compute-wise.
  - 7e 5mo ago
    I expect you can recover most if not all of the gyroscopic data even from a blurred image, possibly at higher accuracy.
    - _flux 5mo ago
      I actually don't think that's possible, if you assume the objects in the video can also shake and the intent is to eliminate only the camera shaking.
- UltraSane 5mo ago
  gyroscope data is much higher time resolution and is the ground truth vs guessing motion from pixel.
  - deleted 5mo ago
    [deleted]
  - Mars008 5mo ago
    [dead]
matsemann 5mo ago
Haven't really used much other stabilization in post after modern gopros have gotten so good. Especially with the 360 variants (MAX and now MAX 2), it's buttery smooth (and infinite FOV means no cropping). Sometimes too smooth, I want to show how rough the cycling trail really was!
- greatgib 5mo ago
  With very good daylight, Hypersmooth of Gopro is ok, but as soon as the conditions are a little bit less than ideal, watching the videos that we get out of the Gopro makes nauseous very fast.
RicoElectrico 5mo ago
What's the deal with mdk-sdk? It is closed source, so there must be a very good reason to use it. For context, the webpage: https://www.qtav.org/blog/new-sdk.html https://www.qtav.org/blog/new-sdk.html
mikae1 5mo ago
I use this for near 100% of my video projects at work (with Sony cameras) and it's absolutely amazing. Sony is not exactly class leading when it comes to stabilization (like Panasonic). There's a Premiere and Resolve plugin-in these days. Fast guide: * Be sure to turn off any in camera stabilization in Sony cameras. * Be sure to take the added crop into account when composing. * The faster shutter speed, the better. Forget about 1/50 for 25p. There will be the most horrible artifacts. For 25p, use 1/100 or preferably use 1/200. For 50p, use 1/200 or preferably 1/400 etc.
- JKCalhoun 5mo ago
  > Forget about 1/50 for 25p. There will be the most horrible artifacts. (My ignorance,) is motion blur no longer a thing in modern digital? (I'm also confused: 1/50 is the shutter speed and 25p the frame rate?)
  - Gigachad 5mo ago
    Normally you double the shutter speed compared to the frame rate to include a motion blur that looks good. But if you then apply stabilisation post processing, you end up with a shot that has motion blur yet isn’t moving. So you want to set a very fast shutter speed, and then you might introduce fake motion blur in post processing later that matches the movement of the stabilised video.
    - JKCalhoun 5mo ago
      Interesting—and makes sense. (Perhaps motion blue becomes a post-process as well then, ha ha.) Kind of a tangent: One of the remarkable things to me about the "Dykstra flex" cameras developed for Star Wars was that the dolly/stepper-motors moved the camera while the shutter was open giving those fly-by shots full motion blur. Freeze any frame where there is a space battle and it is obvious. That small detail was not small at all in selling the effects of the film. But one of the effects guys joked that some team had borrowed the camera for some effects they were doing for a TV show or film—and they used Dykstra-flex in sort of a "stop motion" manner. He was dumbfounded why someone would move the camera, pause to expose a frame, move again to the next location, pause to expose. Just walking away, leaving motion-blur on the table…
brad7 5mo ago
My friend built this which allows you to use Gyroflow natively within Final Cut, worth a look if that’s your NLE of choice: https://gyroflowtoolbox.fcp.cafe/ https://gyroflowtoolbox.fcp.cafe/

179 points by nateb2022 5mo ago

Gigachad 5mo ago

I wanted to try this but sadly it seems my A7 iii doesn't record gyro data.

Zanfa 5mo ago
There are external devices that can be attached to the camera to record gyro data, e.g https://docs.gyroflow.xyz/app/advanced-usage/using-external-gyro-source/niyien-a1 https://docs.gyroflow.xyz/app/advanced-usage/using-external-... I just ordered one a few weeks ago and haven't received it yet, so can't talk about personal experience, but there's no technical reason it couldn't work well.
- JosephRedfern 5mo ago
  Out of curiosity - how do you sync gyro data with video timecode?
nnevatie 5mo ago
My a7CR seems to be supported - will try this out later.
- tecleandor 5mo ago
  Some months ago I tested footage of my A7C2 on Gyroflow expecting to do visual stabilization, and found my camera has a gyro (didn't know about it) and automatically adds that info to the videos. The result was pretty good and it was super easy to do it.
  - mikae1 5mo ago
    Just be sure to turn off any in camera stabilization to get the best results.
    - Gigachad 5mo ago
      In the guide videos they show you just have to sync it up by pressing the record button and end record button on both the gyroscope and the camera at the same time and this is apparently enough to calculate the time difference between the two clocks.
      - tecleandor 5mo ago
        I think mikae1 refers to the lens/sensor stabilization, as it can be problematic if you combine it with other stabilization systems. My camera already embeds gyro information automatically in the video files without intervention (I didn't even know about it).

nerderloo 5mo ago

Such a fine piece of software.

jauntywundrkind 5mo ago

That this also helps so much with rolling shutter correction is wild! Epic.

rixed 5mo ago

How is this different from usual sensor stabilisation techniques? Is it because it can adjust for a wider range of motion?

Gigachad 5mo ago
Normal sensor stabilization only moves a tiny amount, It's more useful for photography for reducing micro jitters to get sharp photos. For video you need much more aggressive cropping and warping to undo the massive shake of walking with the camera.
- Mars008 5mo ago
  [dead]

po1nt 5mo ago

As a drone pilot I used this before davinci studio implemented something similar. I highly recommend.

FrostKiwi 5mo ago

So So useful <3 Stabilization = you have to zoom-in, loss of FOV. Depending on Action Cam and undistortion parameters, this can be different, sometimes too little, sometimes too much. Gyroflow allows you to dial it in. Lot's Stabilization in a particularly shaky spot and widest FOV everywhere else, smoothed between to be unnoticeable.

hdersch 5mo ago

I wonder how this compares to purely vision-based systems which use nothing but the images themselves for stabilization. Here are some quite old results of stabilization using image-based 3d-reconstruction of the scene which I wrote more than 10years ago, compared with other stabilization programs of that time (Deshaker, Adobe After Effects, Youtube). With todays improved hardware and progress in 3d-algorithms you may not need any additional gyroscopic data. https://www.youtube.com/watch?v=-m3fwhx3Z5g https://www.youtube.com/watch?v=-m3fwhx3Z5g

_flux 5mo ago
I think gyroscopic data still can have the edge if it has higher sampling rate than the video: then it could be used for removing blur from individual frames. I also expect purely gyroscopic approach to be much lighter compute-wise.
- 7e 5mo ago
  I expect you can recover most if not all of the gyroscopic data even from a blurred image, possibly at higher accuracy.
  - _flux 5mo ago
    I actually don't think that's possible, if you assume the objects in the video can also shake and the intent is to eliminate only the camera shaking.
UltraSane 5mo ago
gyroscope data is much higher time resolution and is the ground truth vs guessing motion from pixel.
- deleted 5mo ago
  [deleted]
- Mars008 5mo ago
  [dead]

matsemann 5mo ago

Haven't really used much other stabilization in post after modern gopros have gotten so good. Especially with the 360 variants (MAX and now MAX 2), it's buttery smooth (and infinite FOV means no cropping). Sometimes too smooth, I want to show how rough the cycling trail really was!

greatgib 5mo ago
With very good daylight, Hypersmooth of Gopro is ok, but as soon as the conditions are a little bit less than ideal, watching the videos that we get out of the Gopro makes nauseous very fast.

RicoElectrico 5mo ago

What's the deal with mdk-sdk? It is closed source, so there must be a very good reason to use it. For context, the webpage: https://www.qtav.org/blog/new-sdk.html https://www.qtav.org/blog/new-sdk.html

mikae1 5mo ago

I use this for near 100% of my video projects at work (with Sony cameras) and it's absolutely amazing. Sony is not exactly class leading when it comes to stabilization (like Panasonic). There's a Premiere and Resolve plugin-in these days. Fast guide: * Be sure to turn off any in camera stabilization in Sony cameras. * Be sure to take the added crop into account when composing. * The faster shutter speed, the better. Forget about 1/50 for 25p. There will be the most horrible artifacts. For 25p, use 1/100 or preferably use 1/200. For 50p, use 1/200 or preferably 1/400 etc.

JKCalhoun 5mo ago
> Forget about 1/50 for 25p. There will be the most horrible artifacts. (My ignorance,) is motion blur no longer a thing in modern digital? (I'm also confused: 1/50 is the shutter speed and 25p the frame rate?)
- Gigachad 5mo ago
  Normally you double the shutter speed compared to the frame rate to include a motion blur that looks good. But if you then apply stabilisation post processing, you end up with a shot that has motion blur yet isn’t moving. So you want to set a very fast shutter speed, and then you might introduce fake motion blur in post processing later that matches the movement of the stabilised video.
  - JKCalhoun 5mo ago
    Interesting—and makes sense. (Perhaps motion blue becomes a post-process as well then, ha ha.) Kind of a tangent: One of the remarkable things to me about the "Dykstra flex" cameras developed for Star Wars was that the dolly/stepper-motors moved the camera while the shutter was open giving those fly-by shots full motion blur. Freeze any frame where there is a space battle and it is obvious. That small detail was not small at all in selling the effects of the film. But one of the effects guys joked that some team had borrowed the camera for some effects they were doing for a TV show or film—and they used Dykstra-flex in sort of a "stop motion" manner. He was dumbfounded why someone would move the camera, pause to expose a frame, move again to the next location, pause to expose. Just walking away, leaving motion-blur on the table…

brad7 5mo ago

My friend built this which allows you to use Gyroflow natively within Final Cut, worth a look if that’s your NLE of choice: https://gyroflowtoolbox.fcp.cafe/ https://gyroflowtoolbox.fcp.cafe/