You could infer the camera position and angle using Structure from Motion (SfM) and then stabilize the video from that but I'm guessing you'd get a lot of error

You could infer the camera position and angle using Structure from Motion (SfM) and then stabilize the video from that but I'm guessing you'd get a lot of error built up in the process and it would be questionable how much stabilization you'd get in the end but it would be interesting to see how well it would work.