8 ms·

Gyroflow: Advanced open-source gyro-assisted video stabilization

360 points by jonbaer 5y ago

aidenn0 5y ago
That rolling-shutter correction alone on FP is amazing.
jmrobles 5y ago
It's awesome and open source! I made this video comparison between GoPro6 4k video vs the stabilized with gyroflow https://youtu.be/KWiqF8Z8lEg https://youtu.be/KWiqF8Z8lEg
dekhn 5y ago
please fix your https cert
- qwertox 5y ago
  What was wrong with it? It's a Let's Encrypt issued on Feb 1. Maybe you better talk to your MITM. https://crt.sh/?q=www.gyroflow.xyz https://crt.sh/?q=www.gyroflow.xyz
Kelteseth 5y ago
This is actually really cool from a code perspective, as this is the first app I've seen that uses Qt/QML in combination with Rust. I was always searching for alternative cross-platform GUI toolkits that use Rust and some form of QML like language for the UI. sixtyfps (now slint) is a promising candidate that is developed by some ex Qt guys, but their QML style syntax is still a bit off in my taste. [1] https://github.com/slint-ui/slint https://github.com/slint-ui/slint
Ancapistani 5y ago
I use this regularly, with good results. Here’s an example comparison of flight footage from a RunCam 5 Orange on a 5” freestyle quadcopter: https://youtu.be/1ZORzfJ1sog https://youtu.be/1ZORzfJ1sog
- WithinReason 5y ago
  It does a really good job! Thanks for posting an example. One decision that such a program needs to make is how to smooth out the movement, and what this one seems to be doing is a sort of a "piecewise linear" function which is a bit jarring to watch when it suddenly transitions between 2 speeds.
  - Ancapistani 5y ago
    > One thing that such a program needs to make is how to smooth out the movement There are several different smoothing algorithms that you can choose from - I don’t recall the name of the one I used here, but it was specifically called out in the docs to be most suitable for acrobatic flight. You can set up key frames, which can give a lot more control over where and how the stabilization is applied. I just used the ones that gyroflow chose for me here. > what this one seems to be doing is a sort of a "piecewise linear" function which is a bit jarring to watch when it suddenly transitions between 2 speeds. I made this example specifically to learn how to stabilize that kind of footage, because that’s what I do the most of and that’s where I think I’ll find the biggest benefit. Also… this is my favorite quadcopter at the moment, and can do some pretty crazy stuff. It’s got 5x5x3 props, 1,950KV motors, and uses 6S batteries. Assuming 24V and no load, that’s around 47,000 RPM. I don’t have a GPS receiver on it, but I’ve handily outrun a friend’s flying wing that was moving at 120MPH according to its onboard telemetry. All of that is to say that a lot of that speed transition isn’t an artifact of the stabilization. :) For slow, “cinematic” footage, Premier and DaVinci Resolve’s image-based stabilization does an adequate job with less work.
    - WithinReason 5y ago
      After a second look it's probably down to the movement of the drone rather than the smoothing algorithm
  - Ancapistani 5y ago
    Here’s another example - one of the first ones I did, before I understood most of the options: https://youtu.be/X4R-8BchTd4 https://youtu.be/X4R-8BchTd4 That one uses the default smoothing algorithm, which isn’t really suitable for anything acrobatic.
londons_explore 5y ago
Does the gyro really give much extra information that can't be inferred from the video feed itself? The complexity of simultaneously capturing the gyro output and time syncing it to the video seems pretty high if the benefit is only marginal.
- WithinReason 5y ago
  I remember a colleague working on such a project about 7 years ago (he put together a prototype in 2 weeks). The reason the gyro makes a huge difference is that the image is not produced at the same time, the sensor rows are scanned out gradually, and a gyro allows you to assign a correction to each row of pixels instead only to the whole image at the same time. Doing the same through vision is close to impossible. edit: A gyro-based algorithm is quite simple, while computer vision is still not a fully solved problem in general
- Ancapistani 5y ago
  Definitely. Syncing “blackbox logs” from the flight controller’s gyroscope is a pain, for sure. There are some cameras on the market now that have their own gyroscope and automatically produce sidecar files for the same time period as the video, like the RunCam 5 Orange. For those, syncing the gyro readings is totally a non-issue.
- zcw100 5y ago
  You could infer the camera position and angle using Structure from Motion (SfM) and then stabilize the video from that but I'm guessing you'd get a lot of error built up in the process and it would be questionable how much stabilization you'd get in the end but it would be interesting to see how well it would work.
rvnx 5y ago
Result looks great! Much better than the "stabilisation" of YouTube that easily makes you sick (I think they shut it down ?).
- abbaselmas 5y ago
  this option is removed from youtube afaik, however, google photos now this option on local devices. If you edit videos you can see stabilise option, which is actually the youtube option.
qwertox 5y ago
That promo video is production grade stuff.
floatrock 5y ago
Not in the graphic arts, but this is the most appropriate use of an .xyz domain I've yet seen.
- echelon 5y ago
  .xyz is blowing up and gaining a lot of reputation. It's the new .io and .ai. Incredibly hip for your startup. abc.xyz block.xyz Look at all the HN articles pouring in with this TLD: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=xyz&sort=byDate&type=story https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu... But you're right. Spatial domain products should jump all in on this.
  - rambojazz 5y ago
    They have a lot of catching up to do if they want to improve reputation.
  - birdman3131 5y ago
    And nigh worthless seeing as a lot of phone carriers will siliently delete your text if they spot a .xyz domain in it.
    - rafale 5y ago
      Why would they do that? They will change policy anyway if the TLD keeps getting popular. You gotta see things for what they are headed not for what they are.
      - LeifCarrotson 5y ago
        Unless you want people to send each other texts about your domain today, rather than 2 years in the future. Yeah, register the .xyz domain on top of .com now, but don't put the TLD in your trademark...
  - soylentgraham 5y ago
    They certainly blew up 6/7 years ago when they were a few quid/year to register
quadrature 5y ago
Does gyroflow solve for the camera parameters ?. Im wondering if i could attach sensors to my DSLR and then use the sensor data to stabilize the camera footage even thought the sensors would be offset from the camera.
- brokenmachine 5y ago
  It would be difficult to sync your gyro data to the video timecode.
pzo 5y ago
> with Android and iOS apps coming For iOS that’s not gonna happen at least with distributing via App Store considering GPL-3 license.
- Steltek 5y ago
  Depends on who's GPL3 code is being distributed. If it's 100% Gyroflow's code, couldn't they distribute in the app store under whatever license is necessary while also hosting a GPL3 repos somewhere? Admittedly, not a great solution and Android is just always going to be more friendly for apps and use cases that fall outside of mainstream.
  - pzo 5y ago
    That’s true. It seems a young project but already got some PRs so those contributors potentially could always claim their rights to apple to remove from App Store. Some alternative could be just instructions how to build yourself or distribution via AltStore.
- jcelerier 5y ago
  https://en.m.wikipedia.org/wiki/List_of_free_and_open-source_iOS_applications https://en.m.wikipedia.org/wiki/List_of_free_and_open-source...
liminal 5y ago
But is it better than the ChickenHead S8000? (joke) https://www.youtube.com/watch?v=hle5AHZbc8U https://www.youtube.com/watch?v=hle5AHZbc8U
Luc 5y ago
Nice, but in some shots adding back in a sensible amount of camera shake would make it look more real and exciting, I feel. See also - action movies, games.
_s_a_m_ 5y ago
Gotta love the word "advanced"
jcims 5y ago
Are there any video formats that allow you to stream additional data in auxiliary channels.
- brokenmachine 5y ago
  As a hacky solution, you could use a subtitle track.
mrandish 5y ago
Learning about this app just made my day! As an owner of a Sony Alpha 1 mirrorless camera it'll be super useful. The A1 is one of the first cameras of its type to capture gyro metadata natively. While Sony ships some stabilization software for the A1, a quick look at it left me less than enthusiastic about the workflow of Sony's app and its rather limited capabilities. While the drone demos are cool, the non-drone creative possibilities for amateur filmmakers will be equally impactful. Imagine being able to get perfect crane-style interior tracking shots just using a long pole. As great as they are, drones aren't allowed in a lot of environments due to noise, distraction, insurance, etc. Flying a four pound stabilized cinema rig usually requires a BIG hexacopter that is loud and, frankly, pretty scary to be close to. A lot of us low-to-no budget filmmakers (and students) rely on shooting in improvised locations because we don't have the budget to shut down a city street to get a shot. I'm excited to try this on some handheld run-and-gun action footage. The Alpha 1 camera is amazingly small and light for the pro cinema level quality it delivers which is why it would be a shame to stick it in on a big stabilizing gimbal rig. Capturing gyro metadata natively is an inexpensive 'no-brainer' feature all 'serious' cameras should include.
ozfive 5y ago
Oh God my juices are flowing to build a gyro extension for BMPCC cameras...
ChrisMarshallNY 5y ago
This is cool. I’d like to see how cameras that use this, stack up against the Osmo[0]. That’s a pretty awesome little camera. [0] https://www.dji.com/mobile/pocket-2 https://www.dji.com/mobile/pocket-2
skyde 5y ago
for DSLR that doesn’t have gyro, can i just put a gopro on top of it and stabilize both out of the gyro data from the gopro?
- skyde 5y ago
  Actually found a video of someone on youtube doing exactly this. I will need to try it myself this is awesome!

360 points by jonbaer 5y ago

That rolling-shutter correction alone on FP is amazing.

It's awesome and open source! I made this video comparison between GoPro6 4k video vs the stabilized with gyroflow https://youtu.be/KWiqF8Z8lEg https://youtu.be/KWiqF8Z8lEg

dekhn 5y ago

please fix your https cert

qwertox 5y ago
What was wrong with it? It's a Let's Encrypt issued on Feb 1. Maybe you better talk to your MITM. https://crt.sh/?q=www.gyroflow.xyz https://crt.sh/?q=www.gyroflow.xyz

Kelteseth 5y ago

This is actually really cool from a code perspective, as this is the first app I've seen that uses Qt/QML in combination with Rust. I was always searching for alternative cross-platform GUI toolkits that use Rust and some form of QML like language for the UI. sixtyfps (now slint) is a promising candidate that is developed by some ex Qt guys, but their QML style syntax is still a bit off in my taste. [1] https://github.com/slint-ui/slint https://github.com/slint-ui/slint

Ancapistani 5y ago

I use this regularly, with good results. Here’s an example comparison of flight footage from a RunCam 5 Orange on a 5” freestyle quadcopter: https://youtu.be/1ZORzfJ1sog https://youtu.be/1ZORzfJ1sog

WithinReason 5y ago
It does a really good job! Thanks for posting an example. One decision that such a program needs to make is how to smooth out the movement, and what this one seems to be doing is a sort of a "piecewise linear" function which is a bit jarring to watch when it suddenly transitions between 2 speeds.
- Ancapistani 5y ago
  > One thing that such a program needs to make is how to smooth out the movement There are several different smoothing algorithms that you can choose from - I don’t recall the name of the one I used here, but it was specifically called out in the docs to be most suitable for acrobatic flight. You can set up key frames, which can give a lot more control over where and how the stabilization is applied. I just used the ones that gyroflow chose for me here. > what this one seems to be doing is a sort of a "piecewise linear" function which is a bit jarring to watch when it suddenly transitions between 2 speeds. I made this example specifically to learn how to stabilize that kind of footage, because that’s what I do the most of and that’s where I think I’ll find the biggest benefit. Also… this is my favorite quadcopter at the moment, and can do some pretty crazy stuff. It’s got 5x5x3 props, 1,950KV motors, and uses 6S batteries. Assuming 24V and no load, that’s around 47,000 RPM. I don’t have a GPS receiver on it, but I’ve handily outrun a friend’s flying wing that was moving at 120MPH according to its onboard telemetry. All of that is to say that a lot of that speed transition isn’t an artifact of the stabilization. :) For slow, “cinematic” footage, Premier and DaVinci Resolve’s image-based stabilization does an adequate job with less work.
  - WithinReason 5y ago
    After a second look it's probably down to the movement of the drone rather than the smoothing algorithm
- Ancapistani 5y ago
  Here’s another example - one of the first ones I did, before I understood most of the options: https://youtu.be/X4R-8BchTd4 https://youtu.be/X4R-8BchTd4 That one uses the default smoothing algorithm, which isn’t really suitable for anything acrobatic.

londons_explore 5y ago

Does the gyro really give much extra information that can't be inferred from the video feed itself? The complexity of simultaneously capturing the gyro output and time syncing it to the video seems pretty high if the benefit is only marginal.

WithinReason 5y ago
I remember a colleague working on such a project about 7 years ago (he put together a prototype in 2 weeks). The reason the gyro makes a huge difference is that the image is not produced at the same time, the sensor rows are scanned out gradually, and a gyro allows you to assign a correction to each row of pixels instead only to the whole image at the same time. Doing the same through vision is close to impossible. edit: A gyro-based algorithm is quite simple, while computer vision is still not a fully solved problem in general
Ancapistani 5y ago
Definitely. Syncing “blackbox logs” from the flight controller’s gyroscope is a pain, for sure. There are some cameras on the market now that have their own gyroscope and automatically produce sidecar files for the same time period as the video, like the RunCam 5 Orange. For those, syncing the gyro readings is totally a non-issue.
zcw100 5y ago
You could infer the camera position and angle using Structure from Motion (SfM) and then stabilize the video from that but I'm guessing you'd get a lot of error built up in the process and it would be questionable how much stabilization you'd get in the end but it would be interesting to see how well it would work.

rvnx 5y ago

Result looks great! Much better than the "stabilisation" of YouTube that easily makes you sick (I think they shut it down ?).

abbaselmas 5y ago
this option is removed from youtube afaik, however, google photos now this option on local devices. If you edit videos you can see stabilise option, which is actually the youtube option.

qwertox 5y ago

That promo video is production grade stuff.

floatrock 5y ago

Not in the graphic arts, but this is the most appropriate use of an .xyz domain I've yet seen.

echelon 5y ago
.xyz is blowing up and gaining a lot of reputation. It's the new .io and .ai. Incredibly hip for your startup. abc.xyz block.xyz Look at all the HN articles pouring in with this TLD: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=xyz&sort=byDate&type=story https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu... But you're right. Spatial domain products should jump all in on this.
- rambojazz 5y ago
  They have a lot of catching up to do if they want to improve reputation.
- birdman3131 5y ago
  And nigh worthless seeing as a lot of phone carriers will siliently delete your text if they spot a .xyz domain in it.
  - rafale 5y ago
    Why would they do that? They will change policy anyway if the TLD keeps getting popular. You gotta see things for what they are headed not for what they are.
    - LeifCarrotson 5y ago
      Unless you want people to send each other texts about your domain today, rather than 2 years in the future. Yeah, register the .xyz domain on top of .com now, but don't put the TLD in your trademark...
- soylentgraham 5y ago
  They certainly blew up 6/7 years ago when they were a few quid/year to register

quadrature 5y ago

Does gyroflow solve for the camera parameters ?. Im wondering if i could attach sensors to my DSLR and then use the sensor data to stabilize the camera footage even thought the sensors would be offset from the camera.

brokenmachine 5y ago
It would be difficult to sync your gyro data to the video timecode.

pzo 5y ago

> with Android and iOS apps coming For iOS that’s not gonna happen at least with distributing via App Store considering GPL-3 license.

Steltek 5y ago
Depends on who's GPL3 code is being distributed. If it's 100% Gyroflow's code, couldn't they distribute in the app store under whatever license is necessary while also hosting a GPL3 repos somewhere? Admittedly, not a great solution and Android is just always going to be more friendly for apps and use cases that fall outside of mainstream.
- pzo 5y ago
  That’s true. It seems a young project but already got some PRs so those contributors potentially could always claim their rights to apple to remove from App Store. Some alternative could be just instructions how to build yourself or distribution via AltStore.
jcelerier 5y ago
https://en.m.wikipedia.org/wiki/List_of_free_and_open-source_iOS_applications https://en.m.wikipedia.org/wiki/List_of_free_and_open-source...

liminal 5y ago

But is it better than the ChickenHead S8000? (joke) https://www.youtube.com/watch?v=hle5AHZbc8U https://www.youtube.com/watch?v=hle5AHZbc8U

Luc 5y ago

Nice, but in some shots adding back in a sensible amount of camera shake would make it look more real and exciting, I feel. See also - action movies, games.

_s_a_m_ 5y ago

Gotta love the word "advanced"

jcims 5y ago

Are there any video formats that allow you to stream additional data in auxiliary channels.

brokenmachine 5y ago
As a hacky solution, you could use a subtitle track.

mrandish 5y ago

Learning about this app just made my day! As an owner of a Sony Alpha 1 mirrorless camera it'll be super useful. The A1 is one of the first cameras of its type to capture gyro metadata natively. While Sony ships some stabilization software for the A1, a quick look at it left me less than enthusiastic about the workflow of Sony's app and its rather limited capabilities. While the drone demos are cool, the non-drone creative possibilities for amateur filmmakers will be equally impactful. Imagine being able to get perfect crane-style interior tracking shots just using a long pole. As great as they are, drones aren't allowed in a lot of environments due to noise, distraction, insurance, etc. Flying a four pound stabilized cinema rig usually requires a BIG hexacopter that is loud and, frankly, pretty scary to be close to. A lot of us low-to-no budget filmmakers (and students) rely on shooting in improvised locations because we don't have the budget to shut down a city street to get a shot. I'm excited to try this on some handheld run-and-gun action footage. The Alpha 1 camera is amazingly small and light for the pro cinema level quality it delivers which is why it would be a shame to stick it in on a big stabilizing gimbal rig. Capturing gyro metadata natively is an inexpensive 'no-brainer' feature all 'serious' cameras should include.

ozfive 5y ago

Oh God my juices are flowing to build a gyro extension for BMPCC cameras...

ChrisMarshallNY 5y ago

This is cool. I’d like to see how cameras that use this, stack up against the Osmo[0]. That’s a pretty awesome little camera. [0] https://www.dji.com/mobile/pocket-2 https://www.dji.com/mobile/pocket-2

skyde 5y ago

for DSLR that doesn’t have gyro, can i just put a gopro on top of it and stabilize both out of the gyro data from the gopro?

skyde 5y ago
Actually found a video of someone on youtube doing exactly this. I will need to try it myself this is awesome!