4 ms·

I imagine the system cannot account for where all the planes, staff, passengers are and where they want to go economically.

bumby 4y ago
According the the article, the system makes some non-sensical tasking: "In one example during the storm, the system assigned a pilot to deadhead on a flight from Baltimore to Manchester, N.H., and then back to Baltimore the next day, without ever flying a plane"* * The article defines deadheading as sending a pilot as a passenger to get to another location. It would be interesting to look at what the system is trying to optimize for to make such choices.
- brazzy 4y ago
  Sounds to me more like a bug in an edge case rather than trying to optimize for the wrong thing. Or (more likely, I think) the first deadhead was planned to have the pilot take over a flight which was then cancelled after the pilot was already underway.
  - smaudet 4y ago
    Also sounds like a case of not throwing enough adversarial data at the system - you can't just code coverage your code, you can't even establish KPIs, you have to establish its performance under system failure (does it freeze or gracefully shut down, does it persist to disk, what happens when the disk is yanked out of the system), etc. Very few software shops I am aware of that do anything like this.
- twobitshifter 4y ago
  If the solver is updating based on current events, it’s completely reasonable that something like that could happen. Manchester was projected to be short a pilot, but the flight made it on time, and Baltimore unexpectedly ended up short.
- prewett 4y ago
  This sounds like what you might expect from a using a random process (genetic algorithms, simulated annealing) to solve the NP-complete problem. The randomness injects suboptimal routes like this, as well as more optimal ones, but the fitness/cost function has to distill a whole bunch of things into one scalar. In my experience what happens is that the different things you want to value sort of compete with each other. I'm guessing the current state of the system is quite suboptimal, and it might not be able to remove the randomness. It's quite feasible to get within a factor of 2 of the optimal solution (with much less processing power), which sounds great from a CS algorithm analysis standpoint, but a factor of 2 looks like an awful schedule from a human standpoint.
- bell-cot 4y ago
  Compare the cost of keeping one surplus pilot forever deadheading around the system to the cost & disruption of canceling (say) one flight a month because "a pilot got sick, it was not legal to fly short-handed". Having a (relative) bunch of surplus critical-task workers, forever being shuffled to where they system guesses they're most likely suddenly be needed, makes perfect sense. (Yes, you'd have to be a bit more sophisticated, so your surplus pilots got enough flying time to stay certified. And didn't get pissed and quit. Assume that I know why RAID 5 is better than RAID 4:)
- taway22323 4y ago
  "non-sensical tasking" That could also be the result of thrashing after they were really far into the mess. That is, the solver maybe did output something sensible, but that solution has to get into the system that's used at runtime. Someone may have run another solution, or did manual updates while it was too overwhelmed to get all the changes posted. The solvers also depend on other context that's changing underneath them.

bumby 4y ago

According the the article, the system makes some non-sensical tasking: "In one example during the storm, the system assigned a pilot to deadhead on a flight from Baltimore to Manchester, N.H., and then back to Baltimore the next day, without ever flying a plane"* * The article defines deadheading as sending a pilot as a passenger to get to another location. It would be interesting to look at what the system is trying to optimize for to make such choices.

brazzy 4y ago
Sounds to me more like a bug in an edge case rather than trying to optimize for the wrong thing. Or (more likely, I think) the first deadhead was planned to have the pilot take over a flight which was then cancelled after the pilot was already underway.
- smaudet 4y ago
  Also sounds like a case of not throwing enough adversarial data at the system - you can't just code coverage your code, you can't even establish KPIs, you have to establish its performance under system failure (does it freeze or gracefully shut down, does it persist to disk, what happens when the disk is yanked out of the system), etc. Very few software shops I am aware of that do anything like this.
twobitshifter 4y ago
If the solver is updating based on current events, it’s completely reasonable that something like that could happen. Manchester was projected to be short a pilot, but the flight made it on time, and Baltimore unexpectedly ended up short.
prewett 4y ago
This sounds like what you might expect from a using a random process (genetic algorithms, simulated annealing) to solve the NP-complete problem. The randomness injects suboptimal routes like this, as well as more optimal ones, but the fitness/cost function has to distill a whole bunch of things into one scalar. In my experience what happens is that the different things you want to value sort of compete with each other. I'm guessing the current state of the system is quite suboptimal, and it might not be able to remove the randomness. It's quite feasible to get within a factor of 2 of the optimal solution (with much less processing power), which sounds great from a CS algorithm analysis standpoint, but a factor of 2 looks like an awful schedule from a human standpoint.
bell-cot 4y ago
Compare the cost of keeping one surplus pilot forever deadheading around the system to the cost & disruption of canceling (say) one flight a month because "a pilot got sick, it was not legal to fly short-handed". Having a (relative) bunch of surplus critical-task workers, forever being shuffled to where they system guesses they're most likely suddenly be needed, makes perfect sense. (Yes, you'd have to be a bit more sophisticated, so your surplus pilots got enough flying time to stay certified. And didn't get pissed and quit. Assume that I know why RAID 5 is better than RAID 4:)
taway22323 4y ago
"non-sensical tasking" That could also be the result of thrashing after they were really far into the mess. That is, the solver maybe did output something sensible, but that solution has to get into the system that's used at runtime. Someone may have run another solution, or did manual updates while it was too overwhelmed to get all the changes posted. The solvers also depend on other context that's changing underneath them.