8 ms·
Flying planes in Microsoft Flight Simulator with a JavaScript autopilot (2023)
A (lengthy!) tutorial on how to write your own JS-based autopilot code to fly planes in Microsoft Flight Simulator with a web page to both visualize the flight and control the AP.
Note that this is a complete do-over of a previous page that used a combination of Python and JavaScript, and only covered basic autopilot functionality. This rewrite threw all of that out, and goes much, much further:
you just want to have the plane take off without having to do anything? Check, this tutorial covers auto-takeoff.
Flying with waypoint-based navigation, not by "programming airports and radio beacons one letter at a time with a jog-dial" but by just placing markers on a map and dragging those around as you please, mid-flight, Google Maps style? Check. After all, why would we not, putting a map on a webpage is super easy (barely an inconvenience).
In fact, let's throw in flight plan saving and loading, so others can fly the same route!
And maybe you want the plane to fly a fixed distance above the ground, instead of at a fixed altitude... well: check. You'll learn about terrain-follow using free Digital Elevation Model datasets for planet Earth.
And of course the big one, for the ultimate "click button, JS flies the plane start to finish" experience: ever wished the in-game autopilot could just land the plane for you? Well, it's a video game, and we know programming, so: why not? We're also implementing auto-landing and it's going to be amazing.
If you play MSFS, and you're a developer/programmer, and you've always thought how cool it would be if you could just do some scripting for it... you can, there's a lot of cool stuff you can do!
And of course if you don't want to follow along and just want to have everything set up and ready to fly your plane, just clone https://github.com/Pomax/are-we-flying https://github.com/Pomax/are-we-flying and have fun.
- cadecairos 2y agoJust dropping by to say: Hi Pomax! Long time no see, hope you're doing well! _Fantastic_ tutorial!
- TheRealPomax 2y agoGlad to hear you liked it! Buying that HOTAS from you "to see if that makes MSFS fun or whether I should just get it refunded" may have sent me down a rabbit hole that was deeper than I could have ever expected =D
- cadecairos 2y agoHaha! that's so great, I'm glad the hardware found a good home :)
- deleted 2y ago[deleted]
- t0mas88 2y agoNice! X-plane also has a great API to do these things with. It was created by NASA for research: https://github.com/nasa/XPlaneConnect https://github.com/nasa/XPlaneConnect
- TheRealPomax 2y agoThere's even a pretty great autopilot for x-plane post that makes it to HN every now and then! https://austinsnerdythings.com/2021/10/15/creating-an-autopilot-in-x-plane-using-python-part-1/ https://austinsnerdythings.com/2021/10/15/creating-an-autopi...
- auspiv 2y agoAustin from the referenced post here - always love to see the shoutout! What you did is pretty much what I wanted to do when I set out. But definitely not in JS haha, not a huge fan of it Anyways, I chose to go with PID control for my control systems, which worked really well right "out of the box" with some somewhat generic values for P, I, and D. I read/skimmed all the way through your post. It looks like your control system is more or less a P & D controller? Any reason you didn't just go with PID loops? Is this part of needing things to be stateless to work for JS (I am no JS expert, just saw a few mentions of that in the conclusion)?
- TheRealPomax 2y agoIt's basically P-D-and-a-little-second-derivative, but the straight PD controller code just doesn't like the whole "you only get to run every half second" timing aspect. Things take far too long to ramp up, and the amount of overshoot you get is just ridiculous. There's no way (at least, not that I've been able to mitigate in anything even close to acceptably) to avoid super obvious oscillation, even if it eventually converges, at the low rate we're running. Someone could, if they wanted (as one of the "what's left" options =D) replace all of this with PD code but they'll probably need to run at a much higher data rate for things to actually behave, and optimize the SimConnect calls to get the dataframes necessary without tying up a thread.
- userbinator 2y agoA bit clickbaity of a title, but the material was far more interesting and in-depth than I expected, although there was no mention of "hunting" which can often occur in control systems like this.
- deleted 2y ago[deleted]
- TheRealPomax 2y agoWhat's clickbaity about it? The title is literally what we're doing: we're flying planes in MSFS using an autopilot that we're (progressively) writing ourselves, using JS.
- userbinator 2y agoThe title was changed --- it just said "Flying Planes with JavaScript" or similar when I first saw it.
- wkjagt 2y agoThat's still the title of the actual post. I like it though.
- TheRealPomax 2y agoAhh, gotcha.
- deleted 2y ago[deleted]
- bottlepalm 2y ago> To allay any concerns: this is not about running JavaScript software to control an actual aircraft. That would kill people. SpaceX: Hold my beer (https://os-system.com/blog/javascript-in-space-spacex-devs-have-shared-crewdragons-tech-stack/ https://os-system.com/blog/javascript-in-space-spacex-devs-h...)
- TheRealPomax 2y agoMental note: hold off on considering taking SpaceX flights O_O!
- mattsan 2y agoTo be fair all critical systems can be controlled by hardware buttons below the displays so they don't need a 100% guarantee, worst case rebooting would be fine
- boffinAudio 2y agoYes, this .. and anyway, isn't the astronauts interaction during critical flight phases completely superfluous to the mission? The screens are there to tell the occupants how close they are to death, it seems, and little else. Heck, for a long time - decades, even - the principle 'interface' to the rocket systems being used to regularly launch humans to space was, literally, a stick. Javascript seems like a huge upgrade from a stick. ;)
- rad_gruchalski 2y agoIt ain’t flying civilians so who cares.
- TheRealPomax 2y agoAlso note that this isn't a "let's learn to implement this thing that I already know works", where I already worked out all the problems and am just giving you a write-up that lets you skip to the final result... We're going to get things wrong, planes are going to crash in myriad ways, and we'll discover how and why things go wrong, to hopefully figure out how we can navigate those problems. And we're going learn from our mistakes. Including learning to throw away all the work we did because that's the right thing to do =)
- kosolam 2y agoSomething in the page kept reloading it when scrolled down trying to read on mobile firefox
- TheRealPomax 2y agoYeah, you really don't want to read this on mobile. It's 200 print pages, I have no idea how long you'd be scrolling. (I need a better nav menu solution for mobile, at the very least)
- kosolam 2y agoI imagine you put a lot of time into this. That’s why I pointed out about the critical bug in this page - it sporadically reloads the page completely preventing it’s use on mobile. 50% of users are mobile users in the general case. Just my 2 cents
- TheRealPomax 2y agoOkay that's.... there isn't anything on the page itself that would even do that so that should definitely not be happening. Could I ask you to file an issue about that over on https://github.com/Pomax/are-we-flying/issues https://github.com/Pomax/are-we-flying/issues ?
- ForOldHack 2y agoIf you are running MSFS on mobile... Seriously?
- Waterluvian 2y agoWe need to integrate this with Twitch so viewers can be my copilot… with the expectation of amusingly disastrous results!
- TheRealPomax 2y agoThat'd be hilarious =P
- withinrafael 2y agoAlso cool, GTA V autopilot using OpenCV, Xbox input, etc. https://github.com/davuxcom/GTA-Pilot https://github.com/davuxcom/GTA-Pilot
- Stevvo 2y agoAwesome. It would be a nice challenge to make a helicopter autopilot, because MSFS doesn't include one like it does for fixed wing.
- TheRealPomax 2y agoI feel like one of the well respected add-on folks made a helicopter with autopilot, but yeah, it would certainly be an interesting exercise!
- sagasu007 2y agogreat sharing
- avinash 2y agoThis is an awesome read. Really enjoyed the level of detail and the various tries before things work.
- Aspos 2y agoA question: I skimmed through MSFS and Xplane API docs but could not find a way to fetch full camera image. Does anyone know of a flight sim (except MS AirSim) which would allow that? Please?
- TheRealPomax 2y agoGiven how expensive that would be (it would be a staggering amount of data!), I doubt anything would allow for that. However, what you might be able to do is use the values that you read from one MSFS/XPlane session and then use the APIs available to replicate the same flight and flight properties in another, separate instance of MSFS/XPlane (on another computer, for example), and then capture its HDMI/DP-out
- safeharbourio 2y agomaybe the OSS Flightgear[1] flight sim might be a good place to start [2] [1]https://www.flightgear.org/ https://www.flightgear.org/ [2] https://wiki.flightgear.org/Portal:Developer https://wiki.flightgear.org/Portal:Developer
- fho 2y ago> (barely an inconvenience) You need to get aaaall the way of my back about why flying planes with JS sounds like a bad idea :-)
- TheRealPomax 2y agoPlus it's in the script, so...
- albertzeyer 2y agoWow, that is a lengthy tutorial. I only skimmed over it now, so maybe I missed it: How do all these developed methods compare to existing real autopilot systems? Are there even existing autopilot systems which work similarly? Did the author come up with these methods just by themselves, or were they adapted from existing systems? Would those developed methods make sense in a real autopilot system? Also, it seems that this involved a lot of trial-and-error, and the solutions come with a number of heuristics and hyper parameters to tune. I wonder if you can do better and avoid that as much as possible. First of all, you would need to have an actual metric which covers all the things you would care about. Obviously no fatal crash, but then also maybe somewhat smooth flight (you can measure that) and how close you want to get to each waypoint, while maybe taking not too much fuel. And given those, let it find the best possible path. I guess this is optimal control theory then. When you have an actual metric which you can measure, then you can quantitatively compare all the methods, also automatically optimize any parameters.
- 8372049 2y agoI haven't read the tutorial (yet), but I know a bit about autopilots. Autopilots depend a lot on which kind of plane it is in, but for the most part, autopilots aren't really "auto pilots". They consist of various fairly basic primitives rather than more complex decision making. As a rule of thumb, the complexity of the autopilot is proportional to the size of the plane. Basic autopilots can maintain headings, maintain (barometric) altitudes (i.e. above sea level) and similar very basic operations. For these autopilots it is the pilot's job to ensure the plane is operating within the envelope and does not depart from controlled flight and the throttle is controlled by the pilot. As an example, the autopilot in an F-16 fighter jet has four things it can do: maintain attitude, maintain altitude, fly toward nav point and fly a compass heading. That's it. Autopilots in airliner jets are more complex and can handle a set of nav point, ascent/descent, arrival and, depending on the airfield, approach and landing. They also feature autothrottles. Their implementation is still fairly simple, since it's a matter of matching throttle, attitude and heading with the position and altitude of each nav point. The autopilot doesn't have to do any pathfinding or more complex decision making, in order to keep it simple, predictable and bug free.
- throwup238 2y agoThe best thing about Hacker News is that it provides all the knowledge you need to make your own suburban weapons system. Last month it was radar [1], this month navigation! Can you extend the tutorial to cover positioning from the elevation data? :-) [1] https://news.ycombinator.com/item?id=39930195 https://news.ycombinator.com/item?id=39930195
- morkalork 2y agoYes, an overview of INS, terrain matching, and vSLAM algorithms would be interesting. You know, just in case you're flying around somewhere GPS is being jammed right? On a serious note, I hope the Ukrainians who designed and built long range drones that can accurately hit the distillation columns of Russian oil refineries after flying hundreds km through contested air space get the chance to write some books about their accomplishments one day. It's quite the achievement isn't it?
- idontwantthis 2y agoI’m sure it’s quite a challenge, but I don’t think they are autonomously navigating directly to their targets. I’ve read they are using autonomous flight until a hand off to an operator behind enemy lines.
- TheRealPomax 2y agoI think I'll leave that as an exercise for the reader =P
- Stevvo 2y agoFor anyone interested, Asobo studio is currently hiring an Autopilot Programmer: https://jobs.eu.lever.co/asobostudio/1a940a26-31c4-4720-8271-ccee5553529e https://jobs.eu.lever.co/asobostudio/1a940a26-31c4-4720-8271... Honestly would never see something like this writeup from MSFS add-on developers. They are a protective bunch who share nothing with each other. This is in contrast to X-Plane add-on developers, who are friendly and collaborative.
- TheRealPomax 2y agoI guess it also has to do with whether or not someone will go "oh cool, cool, okay I copied everything you did and quickly released a shittier version that made me a few bucks but turned the flightsim community off of the plane you're still working on" =(
- Stevvo 2y agoI think a lot of it has to do with how MS has dealt with add-on developers in the past. After FSX was abandoned by MS, if you wanted new functionality in the sim you had to build it yourself. So add-on developers would all have their own propriety tech for things like Avionics, nav and the flight-model. Today MSFS is constantly updated with new functionality, so you can just focus on your project, but old attitudes remain.
- ephaeton 2y agohehe. "wasn't that the guy where I got my baguette recipe from?" goes searching "it WAS! beautiful. Then this guide here surely will also be fun"
- TheRealPomax 2y agoHaha, that's fantastic. Trying to see if there is any way to tie baguettes, Bezier curves, and autopilots into a single thing, but until Microsoft releases Bread Simulator 2025 I might be out of luck there =D