31 ms·

Southwest cancels 5,400 flights in less than 48 hours

584 points by edward 4y ago

avalys 4y ago
Astonishing. Based on reports from employees on Reddit (https://www.reddit.com/r/SouthwestAirlines/comments/zw32yt/psa_from_a_swa_employee_since_the_company_wont/ https://www.reddit.com/r/SouthwestAirlines/comments/zw32yt/p...), the actual story here is that Southwest has had a complete operational failure and is simply incapable of determining where their crews are, what flights they are eligible for, and associating them with flights on the schedule. It sounds like there is a semi-automated system which broke somehow, or more likely failed to support the load of cascading changes that resulted from weather disruptions, and they simply don't have the capacity or flexibility to deal with this other than by cancelling flights for three days until they can sort everything out manually. Reading between the lines a bit, one possible root cause is that their semi-automated system required crew members to update their status by phone when something goes wrong (perhaps with another human in the loop), and the sheer volume of disruption overloaded their phone system and resulted in the automated system becoming completely decoupled from the actual state of reality without sufficient bandwidth in the phone channel to get it back in sync in a reasonable amount of time.
- 0x445442 4y ago
  Yeah the storm angle is a lie.
  - BitwiseFool 4y ago
    This is hearsay, but I believe any cancellations due to weather are not subject to fines and fees from the FAA and the airline is not obligated to provide more generous compensation/accommodation for customers. So if there's any reasonable way to blame a failure on inclement weather airline companies will do so without hesitation.
    - 0x445442 4y ago
      You’re wrong dude. https://www.reddit.com/r/SouthwestAirlines/comments/zw6upo/hey_guys_swa_pilot_with_a_little_information/?utm_source=share&utm_medium=ios_app&utm_name=iossmf https://www.reddit.com/r/SouthwestAirlines/comments/zw6upo/h...
      - ipython 4y ago
        How does that post in any way refute the parents point? I agree with the parent that management will try like hell to blame this on the weather- heck they already are. Just check out their press releases.
        0x445442 4y ago
        Yeah I’m a bit confused how the comment even relates to mine. I took it to mean that my assertion that laying blame on the weather for this mess is bogus and a lie. But that assumption may be wrong.
      - ThunderSizzle 4y ago
        He's not wrong. Every airline will attempt to blame operational issues on non-finable causes, such as weather.
        tedunangst 4y ago
        So what are the finable causes for flight cancellation?
    - bumby 4y ago
      >cancellations due to weather are not subject to fines and fees from the FAA I thought this too, but apparently there are no federal regulations other than reimbursement for the direct costs. "Airlines are not required to provide passengers with money or other compensation for costs that fall outside of the cancelled airline ticket and fees tied directly to the airline ticket (such as baggage fees, seat upgrades, etc.) when flights are cancelled." [1] "Each airline has its own policies about what it will do for customers on bumped or cancelled flights. There are no federal requirements." [2] However, there are rules around "bumping" passengers on a flight. "An airline is required to compensate you after involuntarily bumping you from an oversold flight in certain situations. However, there are many situations where you are not entitled to compensation." [3] [1] https://www.transportation.gov/individuals/aviation-consumer-protection/flight-delays-cancellations https://www.transportation.gov/individuals/aviation-consumer... [2] https://www.faa.gov/faq/what-are-policies-bumped-or-cancelled-flights-0 https://www.faa.gov/faq/what-are-policies-bumped-or-cancelle... [3] https://www.transportation.gov/individuals/aviation-consumer-protection/bumping-oversales https://www.transportation.gov/individuals/aviation-consumer...
  - kube-system 4y ago
    It is likely both. Scheduling is quite easy when it goes normally. Scheduling becomes a more difficult task when more exceptions happen. That reddit post is by is some boots-on-the-ground employee, not someone who understands the algorithmic details of SWAs somewhat unique routing challenges.
    - phpisthebest 4y ago
      It seems they do their scheduling like they do their seating... it is all Free for All
      - bombcar 4y ago
        Should just text all pilots to bum-rush any plane they’re equipped to fly. Why not add the two front seats into the free for all?
        kube-system 4y ago
        You joke, but being able to swap any pilot to any plane is part of the reason their fleet is 100% 737s.
    - EFreethought 4y ago
      > Scheduling is quite easy when it goes normally. Scheduling becomes a more difficult task when more exceptions happen. I think some slack/spare capacity or redundancy somewhere would make a LOT of problems less likely to happen. Running lean only works out when nothing goes wrong. I know I should not be shocked by the extent of human stupidity, but why the "business" types do not get this is bizarre to me. And paying people more would help too.
      - HDThoreaun 4y ago
        The airline business is too competitive for labor slack. People put up with spirit’s bullshit because it’s $5 cheaper. Any airline that incorporates slack will immediately find itself without customers. If they make it to a catastrophe it would work out, but it’s extremely unlikely that they will be on business that long.
- CharlesW 4y ago
  According to that post they also have no idea where bags are: "Checked bags are currently a disaster. Plan to not see your checked luggage for at least a month. In the interest of 100% transparency, some bags will be 30+ days lost in the system."
- deleted 4y ago
  [deleted]
- shmatt 4y ago
  im kind of doubting that thread, not necessarily fraud but not every employee has the full picture * this memo[1] from the SWA VP thats circulating dated December 21st calling a staffing emergency saying all hell is about to break loose. I'm pretty sure that guy didn't know of the software crash before it happened * Every single piece of checked in luggage is going to take 30+ days to be found? Weird none of the extensive press has mentioned people not getting their bag. Also its pretty much an airport thing and not an airline thing. Airports are in charge of deplaning your luggage and bringing it to you * They're running 40% of flights completely manually with 0 scheduling software? That's extremely impressive [1] https://mobile.twitter.com/ruthschmidt/status/1607460985861988352 https://mobile.twitter.com/ruthschmidt/status/16074609858619...
  - phpisthebest 4y ago
    There have been plently of reporting about the bag issue, including a Chicago local reporter posting a video of 100's or maybe even 1000's of bags in chicago most of southwest tags on them It appears there was a MASSIVE disconnect between baggage and people, bags are ending up all over the place even though the people never left.
    - tiahura 4y ago
      Sounds the the perfect setup for an AirTag commercial.
      - bombcar 4y ago
        iPhone alert: there are ten thousand airtags near you
        giantrobot 4y ago
        Well that at least narrows it down to near you. /s
        joezydeco 4y ago
        I'm kind of curious about that. If AirTags rely on nearby iPhones to capture tag IDs as they pass by and relay them to Apple, will a ramp worker with an iPhone in his pocket start seeing shorter battery life?
- caycep 4y ago
  Granted my usually routine flight in 3 days, from BUR to SJC got canceled. And no way to get thru and the usual handy dandy "full refund!" button has been replaced w just an email form with "we'll get back to you!" I sort of believe it
- hinkley 4y ago
  Somewhere there’s a retired guy sipping his morning coffee and saying, “Told ya.”
  - CyberDildonics 4y ago
    There are people actively doing interviews saying that.
- pembrook 4y ago
  Incidents like this go a long way toward teaching corporate America that software shouldn't be some forgotten "IT" vertical on your org chart that you staff with random contractors. Software is core to your business. If you don't invest in it, you're going to pay the price.
  - bmitc 4y ago
    People mistaking software as something that just does something is the problem. Software is stored and executable knowledge, so if some component of that breaks down, you need to be able to pull it from somewhere else. Otherwise, you get situations like this.
  - lezojeda 4y ago
    They are already pretty resentful at us because we have such high salaries compared to the rest of the population... I bet that nobody is willing to give any penny more to their IT departments
- hintymad 4y ago
  > the actual story here is that Southwest has had a complete operational failure and is simply incapable of determining where their crews are, what flights they are eligible for, and associating them with flights on the schedule. This is amazing. Southwest was famous for its operational efficiency and quality. Companies eagerly learned from them. I wonder what has changed.
  - iamtheworstdev 4y ago
    it's definitely epic. I know this is anecdotal but a friend of mine and pilot for Southwest had to pay for his own hotel room after he captained a flight a couple of days ago. SW apparently thought he was in another state (and it wasn't a neighboring one) even though he just flew the flight that they scheduled him on. Not long (hours) after that he was told to drive to the nearest Southwest physical location and check in. Rumor is that all Southwest pilots were told that a physical presence was required to verify location.
  - GavinMcG 4y ago
    Nothing necessarily changed. Efficiency can become fragility, and quality is rarely measured in a way that includes rare events—partly by definition, and partly because we tend to ignore externalities and unlikely scenarios when we make evaluations.
- achow 4y ago
  The Reddit post is now deleted. Saved screenshot: https://i.imgur.com/bchgmm7.jpeg https://i.imgur.com/bchgmm7.jpeg
- pwillia7 4y ago
  Unremoved reddit post: On behalf of all employees: WE ARE SORRY! I will give it to you straight- this meltdown was caused entirely by Southwest. It was triggered by the storm, but the failure to recover quickly is on Southwest 100%. If you are still hearing “weather” almost a week after the storm, it’s not true. Couple main points: 1. Please be patient with us. We desperately want to do everything we can to get you where you’re going. 2. This shitstorm is because the crew scheduling software went belly up and it almost all has to be unraveled over the phone with crew members calling scheduling. If we had better technology which eliminated the need for phone calls, this would have been fixed by now. 3. If you are able to find alternative transportation to your final destination- DO IT. Another airline, bus, train, Lyft, rental car, ANYTHING. Southwest WILL NOT be able to get you to your destination anytime in the next few days. 4. Like I said, it’s gonna take at least a week to get back to normal operations for Southwest. If anyone has questions, I will try to answer them. I work ground ops at one of SWA’s hubs. EDIT FOR FAQs—— Checked bags are currently a disaster. Plan to not see your checked luggage for at least a month. In the interest of 100% transparency, some bags will be 30+ days lost in the system. Will my flight for X date go out? Next 3 days- plan on a cancellation. 4-7 days- likely to go as scheduled. 7+ days- should see operational recovery.
- wwweston 4y ago
  Other Reddit discussion (with the occasional insider chiming in): https://www.reddit.com/r/bestof/comments/zwd1fq/u4sammich_explains_why_southwest_airlines_is/ https://www.reddit.com/r/bestof/comments/zwd1fq/u4sammich_ex... https://www.reddit.com/r/flying/comments/zw5lsl/southwest_pilots_hows_it_going/ https://www.reddit.com/r/flying/comments/zw5lsl/southwest_pi...
- indymike 4y ago
  > Southwest has had a complete operational failure and is simply incapable of determining where their crews are, what flights they are eligible for, and associating them with flights on the schedule. This has to be really hard as most systems were probably built assuming something that smelled like full staffing. Pilots can only fly X hours, and flight attendants usually can't be forced to take overtime ... there have to be similar constraints on ground crew, maintenance and gate workers, too. So when you are planning for an understaffed airline, it has to be extremely difficult to optimize where the failures will occur.
- mysterydip 4y ago
  Somewhere buried in a million lines of code is the comment "// this should never happen" that is in the block currently being executed.
  - shapefrog 4y ago
    // add fix for snowstorm here
0x445442 4y ago
This is BS. The storm was last week.
- pimlottc 4y ago
  Effects from the storm linger. Buffalo’s airport shut down due to weather issues and remains closed through at least Wednesday.
  - josephcsible 4y ago
    There's also the issue that a storm, no matter how bad, shouldn't make them forget where they left their crew members and your bags.
  - 0x445442 4y ago
    In the case of Southwest the scope of the issues is not because of the weather.
    - EMM_386 4y ago
      It actually is because of the weather. The storms left planes and crew scattered around at all the wrong airports, so they essentially have to reboot the system. Other airlines are not affected by this as much as they fly out of hubs.
  - JustLurking2022 4y ago
    Seems they only linger for Southwest, as they made up 90% of all flight cancellations today.
deleted 4y ago
[deleted]
sys32768 4y ago
Our local airport closed yesterday due to icy runways. Planes would circle a while then refuel 100s of miles away, then return to try again. We don't often see freezing rain here. It creates a hard, bubbly layer of ice that cannot be scraped off with the usual tool.
- Kon-Peki 4y ago
  Get out the curling rocks, eh!
mensetmanusman 4y ago
A software bug can destroy a company these days.
- derwiki 4y ago
  Check out the Knight Capital software bug in 2012 that ate a cool 440m and almost made them go bankrupt
awillen 4y ago
I can't for the life of me understand how they're allowed to say this is due to weather (and thus not provide the same kind of compensation they'd have to if it were "Southwest's fault"). Sure, weather is the precipitating (pun intended, sorry) cause, but given that other airlines are almost all at <10% cancellations, clearly Southwest's total lack of operational robustness and competence are the real issue. The CEO himself emailed Southwest staff and explained that one big problem is their antiquated computer systems - that is unequivocally their fault (and very personally the CEO's fault, since it's very much his job to direct the sort of large scale financial investment to do something like overhaul their system). I hope that DOT follows through on their earlier statement that they're going to investigate this. The CEO of Southwest has ruined more Christmases than the Grinch. Failures like this should trigger incredibly severe consequences (tickets refunded at 4x cost, $20/day paid to any passenger whose luggage is lost until they get it, lost slots in airports) that make it absolutely irrational to operate in the manner Southwest is.
- deleted 4y ago
  [deleted]
- fairity 4y ago
  > Failures like this should trigger incredibly severe consequences (tickets refunded at 4x cost, $20/day paid to any passenger whose luggage is lost until they get it, lost slots in airports) that make it absolutely irrational to operate in the manner Southwest is No need to implement the penalties you described if you’re simply trying to align incentives. Their market cap has fallen by $1b, primarily due to this incident.
  - awillen 4y ago
    Their market cap today isn't what matters - if people forget about this and continue to fly Southwest (which, let's be honest, many will), their share price will recover when they announce earnings. The reality is they save a lot of money by keeping virtually no slack in their system and not doing things like scaling up customer service during the holidays. If they lose some money in flight cancellations, it's still rational for them to operate like this. Plus the reality is they probably won't actually lose much - most people whose flights were cancelled will still end up taking Southwest because it's too expensive (or not possible) to book last minute tickets on other airlines. That means they just push them into the very end of Dec/early Jan when they have open capacity anyway.
ProjectArcturis 4y ago
I have always really wondered how airline scheduling software works. Despite being pretty good with algorithms, I just have no idea how you'd make a system that's robust to weather and mechanical delays.
- castratikron 4y ago
  Sounds like they don't have much of an idea either.
- twawaaay 4y ago
  I don't have experience with airline scheduling but I have experience with software in large financial corporations like banks. The situation with banks (at least the ones I worked with) is that there is a huge amount of software maintained mostly by mediocre to bad teams. These teams fail a lot, the software fails a lot, and yet everything seems to keep going. It is not about code or algorithm quality, it is about procedural side of things -- how the organisation is "programmed" to respond to failures. I use the word "programmed" in a very broad sense -- for me setting up a paper checklist and being able to rely on people to follow it is the same as programming. I suspect the main difference between airlines and banks is that banks can afford to throw money on the problem and just get things done regardless of how inefficiently. Airlines are in the much worse position -- they were able to afford being inefficient and throwing money at the problem in the past but can't do it anymore. They work with old, outdated software that wasn't built with efficiency in mind but now don't have funds to change it and are forced to maintain what they have. This may also be the answer to why sometimes they just don't have capacity to react to problem and let it cascade to bring everything to a halt.
  - blevin 4y ago
    Instead of framing this as saying they "don't have" the funds and were "forced" into inaction, another framing that the board of directors must consider is that current SWA leadership failed in their responsibility to recognize, prioritize, and manage the actual needs of their decades old business.
htag 4y ago
Is this a problem with the weather, with the scheduling software, or with the recent shrink then expand of airline demand? Either way, I don't think I'll book a flight with SW for a long, long time.
- alistairSH 4y ago
  Are there any airlines that haven’t had similar problems over the last 5-10 years? Seems like they’ve all had issues. And if they haven’t, it’s just a matter of time.
LeifCarrotson 4y ago
Same thing happened on July 20, 2016: https://www.dallasnews.com/business/local-companies/2016/08/10/cost-of-southwest-s-tech-outage-climbs-to-at-least-54-million/ https://www.dallasnews.com/business/local-companies/2016/08/... at the time, then-CEO now-chairman Gary Kelly said: > "What's unique is the partial failure, it's never happened," he said. "This isn't a drill you can run." https://web.archive.org/web/20161112192103/http://www.dallasnews.com/business/southwest-airlines/2016/07/30/southwest-ceo-router-failure-grounded-flights-equated-thousand-year-flood https://web.archive.org/web/20161112192103/http://www.dallas... Delta had a similar outage due to a datacenter fire, grounding all domestic flights. Southwest was uniquely slow in taking days to start up again. And if the way my American Airlines ticket switched my birthdate to January 1st, 2000 is any indication, many airlines still need to modernize.
- vrc 4y ago
  Google runs Disaster Recovery Training annually (DiRT) where security teams are tasked with simulating these “black swan” events. Seems like this practice needs to expand to more industries.
  - bobkazamakis 4y ago
    >Seems like this practice needs to expand to more industries. I think you've mistaken this for something immediately increases quarterly gains with no regard to long-term strategy.
    - Analemma_ 4y ago
      In that case the government should smash Southwest with a billion dollar fine so the cost of not doing this drilling exceeds the cost of doing it.
      - hallway_monitor 4y ago
        Asking the government to step in for additional regulation is rarely helpful. For this type of failure, the free market will determine whether processes and tools improve, or whether the status quo is good enough.
AlbertCory 4y ago
From personal experience: I used to preferentially use SWA because they were great. Over the years, it seems to have become that every single flight is significantly delayed. Now I preferentially don't use them.
- el_benhameen 4y ago
  I love not being nickel and dimed by them, and the companion pass makes them significantly cheaper for family travel, but I absolutely avoid holiday travel with them, especially winter holidays.
asah 4y ago
tl;dr: distributed systems are hard.
- chasd00 4y ago
  if i remember my lore correctly it's not a distributed system. It was originally either a mainframe or like AS400 application. I want to say it was related to Saber somehow... maybe they leased it. When i interviewed there years ago they were porting it to some kind of Java stack or maybe they had already ported it and were building it out further. On a tangent, I have to admit, their interview was the best one i've ever had. Very competent people, well rounded process, it was actually more fun than stressful but super challenging too. / this was a lonng time ago, like 10+ years, it could be a very very different animal now
  - bronson 4y ago
    I don't think Southwest was ever on Sabre. Their scheduling used to be in-house, now on Amadeus. Computerized operations started with Braniff's software and have ... ah, grown since then. Name almost any technology and it's probably running somewhere in their stack.
- francisofascii 4y ago
  or maybe you meant to say logistics with little slack is hard
- horns4lyfe 4y ago
  Especially when you outsource it to the lowest bidding WITCH shop you can find. Incredible that the US is still pumping out MBAs that see IT as a cost center. If you are of a sufficient size and depend on software to function, you are a software company and need to allocate your resources accordingly.
DoingSomeThings 4y ago
Incredible to see that 14 months ago there was an almost duplicate outage. Clearly the operations teams are running too lean and don't have enough slack to handle predictable weather/sickness events. Seems like something you'd plan for. Even at the 98%+ uptime range https://news.ycombinator.com/item?id=28823774 https://news.ycombinator.com/item?id=28823774
- chasd00 4y ago
  There's no slack at an airline, it's basically a real time system. Imagine a bottling plant, if one part of the line malfunctions the rest of the line just keeps throwing bottles at the malfunctioning unit until the stop button is hit. There's no way to build in artificial delay to those kinds of systems. Even if you could do that at an airline the margins are so razor thin. I bet there's a dollar value assigned to every second a plane is not in the air.
  - kibwen 4y ago
    > Even if you could do that at an airline the margins are so razor thin. I bet there's a dollar value assigned to every second a plane is not in the air. You either pay the predictable, ongoing cost or you pay the massive, unpredictable cost like they are currently. There's no free lunch. That is, unless you're an executive looking to boost short-term profits by eliminating "redundancy" and then peacing out before the whole thing collapses from your short-sighted, greedy ineptitude.
    - dehrmann 4y ago
      People don't like to hear this now, but airlines chose the right thing to optimize for. Air travel is rarely that important to have these redundancies, and they vote with their wallets every time they buy economy tickets. Given that safe is a baseline, most people value price over reliability.
      - ThunderSizzle 4y ago
        In my experience, every airline will screw you if your not their frequent flyer. Fly once a year? No one is going to waive a fee or refund you anything. Might as well go for cheapest price. My wife's experience shows SWA as an exception in that.
chasd00 4y ago
years ago i interviewed for, and was given an offer, to work on this very system. glad i didn't take it. I feel bad for the devs trying to figure out what's going on.
- bronson 4y ago
  It's an absolute nightmare of fixed-field records and XML (Southwest uses Amadeus for scheduling but Sabre is just as bad; and AFAIK operations at all airlines are a chaotic hodge podge). Southwest performs all operations in Central Time because nobody's figured out how to reliably add time zones to all systems all at once. This means they have something like a 34 hour operational day, and every flight must take off and land in the same operational day. It's a satisfying job for the right personality type!
  - encoderer 4y ago
    My god. This is terrible but also riveting. I wish there was a website devoted to collecting stories of how these internal systems really work. Airlines. IRS. credit agencies. Etc.
    - jeffrallen 4y ago
      It's called "The Daily WTF".
    - hnu0847 4y ago
      Agreed, I hope some article, book, documentary, etc. is eventually published about this.
  - cratermoon 4y ago
    In some recent work I've been doing for an airline, I've worked with 5 different and incompatible ways to represent a passenger, depending on what part of the system you're working in.
    - bronson 4y ago
jrochkind1 4y ago
> Other issues that have exacerbated the airline's struggle to accommodate the holiday rush include problems with "connecting flight crews to their schedules," Perry said. That issue has made it difficult for employees to access crew scheduling services and get reassignments. Wait, what issue? Those words didn't actually say anything. Is this implying that it was a software failure with the scheduling software?
- code_runner 4y ago
  Sounds like it’s doing more than implying it. The scheduling software seems to be pretty hosed at the moment
  - jrochkind1 4y ago
    To me, "problems connecting flight crews to their schedules" does not clearly say it's a software issue, but maybe implies it. But you have other information that it is? Any info on the details? Like, I'd think this is what scheduling software is for...
    - acedTrex 4y ago
      Appears their scheduling software is just woefully inadequate and requires significant human intervention and it just gave up. https://www.reddit.com/r/flying/comments/zw5lsl/comment/j1tne9z/?utm_source=share&utm_medium=web2x&context=3 https://www.reddit.com/r/flying/comments/zw5lsl/comment/j1tn...
      - jrochkind1 4y ago
        I feel like this is a good illustration of the problem with "eh, why does technical debt matter, everything is still working isn't it?" It is, until it's not. And then it's too late.
    - epylar 4y ago
      Pilots literally having to call an 800 number to get their schedule, not enough capacity on the 800 number.
DrWumbo 4y ago
[flagged]
- josephcsible 4y ago
  Because trains don't need scheduling software or crews?
  - sokoloff 4y ago
    One of the problems with SWA’s case is that cabin crews can work a longer duty day than flight crews and airplanes can’t leave on passenger service without a full complement of both, so the airplane scheduling problem might be more complex.
  - lostinroutine 4y ago
    I'm guessing the GP means that there will hopefully be an increased interest for alternatives to planes that one can take when there are air travel crises like this.
  - ciphol 4y ago
    Trains have major weather issues much less frequently than planes.
    - deleted 4y ago
      [deleted]
    - josephcsible 4y ago
      This storm affected trains too <https://www.trains.com/trn/news-reviews/news-wire/amtrak-cancellations-long-departure-delays-continue/ https://www.trains.com/trn/news-reviews/news-wire/amtrak-can...> <https://chi.streetsblog.org/2022/12/28/the-polar-vortex-express-what-it-was-like-riding-amtrak-during-last-weeks-storm-disruptions/ https://chi.streetsblog.org/2022/12/28/the-polar-vortex-expr...>. What was the last weather event that resulted in significant disruption to air travel but didn't disrupt rail travel?
  - overtonwhy 4y ago
RedShift1 4y ago
Is this also one of the companies that completely outsourced their IT?
- qwertyuiop_ 4y ago
  Yup sweep it under the rug. SW outsourced all of their development and support to WITCH. The VPs who get condos in Bahamas in return will not ever mention that this is the result of a failure of crew scheduling software because they signed off on the outsourcing.
- water554 4y ago
  Data point delta airlines IT is almost entirely offshore
faangiq 4y ago
Let’s see the C suite asset clawbacks.
supernova87a 4y ago
As I understand it, Southwest is particularly badly affected by such issues because they run a lot of flights that are not in a hub-spoke model, but rather serial flights one non-hub city to the next and next (like eventually coming back in a loop). You can see this by going to Flightaware.com for example, and following back a flight's previous destinations. See for example https://flightaware.com/live/flight/SWA1092/history/20221223/2210Z/KSJC/KSAN https://flightaware.com/live/flight/SWA1092/history/20221223... and "track inbound plane" a couple times. They jump around the country, much less frequently going back to a hub as other airlines do. That means that the planes and crews have a relatively harder time recovering from system-wide disasters because they don't have as part of normal operations as much ability to centralize or pool resources and get people/planes reorganized. (everyone go back to base, consolidate passengers, crew, planes and redeploy them and sort things out in one place) Unfortunate, but that's their model. Good for some purposes, not so good for others. Maybe it's them being quirky and an active choice. I mean, up until a few years ago they did not fly to Hawaii because their scheduling system / people / processes did not want to have redeye flights.
- Waterluvian 4y ago
  I don’t fly much but I noticed things like “the same flight number takes off at the same time each day and is always the same plane as a different flight number coming the other direction.” It must really help all the employees with routine and consistency even if it’s not optimal.
  - Bonooru 4y ago
    Optimal isn't the right way to think about it. It's a tradeoff. Hub-and-spoke is usually better at getting you to your destination in less absolute time given the same number of total flights since you can have more frequent "shuttle flights" that travel to the hub, exchange goods/passengers, and shuttle back. Point-to-point on the other hand, is better for minimizing travel time since you go directly to the destination.
    - thrashh 4y ago
elijaht 4y ago
My datapoint: Had a flight this Thursday from RIC->ATL cancelled this morning. Interesting that they are cancelling flights this far out, where both cities are relatively unaffected by the storm beyond some cold temps. Was a blessing in disguise for me as I could find a similar flight on a different airline. Hope anyone else flying Southwest can find a way to their destination
- hrunt 4y ago
  The agent this morning told me that they are using the next three days to move staff and planes where they need to be, and the result will be a lot of cancellations when the routes don't serve that. That's why nothing is getting rebooked until Saturday.
boomboomsubban 4y ago
There was a news story a few months ago about some airlines changing the way their crew software works to stop a third party app employees were using to better track their hours Was Southwest one of those airlines?
- makestuff 4y ago
  Looks like it was American https://news.ycombinator.com/item?id=33233975 https://news.ycombinator.com/item?id=33233975
0x445442 4y ago
So in my attempts to get my family home I’ve discovered that rental car company web sites are atrocious. The direct company sites (not Expedia for example) do not give you the ability to search for vehicles at multiple locations within a mileage radius. You are forced to enter a single location for pickup. This is also true for drop off. From what I can tell, rental car companies are not setup to easily service a one way rentals, which is what most people need right now. I’ve managed to reserve a car for pickup in KC and drop off in Phoenix for tomorrow. The quote for the two days, over $1200. This isn’t really a web site issue but it gives some context. Here’s the baffling bit. I make a reservation this morning at Enterprise in a KC suburb and get back a confirmation number, cool. Five minutes later I get a call back that they don’t have any cars available. I ask him why, because I got the confirmation number. He says the website isn’t accurate at the moment. WTF? So when a car is rented and driven off the lot the database doesn’t update? Or worse, the local dealer data is not synced to corporate in real time? This is a cascading mess. They don’t even have availability in Joplin, MO who’s is 3 hours from KC.
- Anechoic 4y ago
  From what I can tell, rental car companies are not setup to easily service a one way rentals, Which isn't surprising because at some point, someone needs to bring that car back to it's home base.
  - trollied 4y ago
    Indeed. Lots of them are franchises that own their own stock.
    - deleted 4y ago
      [deleted]
    - lotsofpulp 4y ago
      I thought none of the 3 US car rental brand holding companies franchised. But it seems that Avis Budget and Hertz do franchise. Enterprise does not. I wonder if that is why I seem to prefer Enterprise. Not that I would touch Hertz with a 10ft pole anyway.
bodhiandphysics 4y ago
One thing to note is that airline crew scheduling is np-hard (trivially you can reduce Hamiltonian cycle to it). Ak interesting semi-empiricle fact about np-complete problems. Take a decision problem to determine if there is a schedule with k crews. Now there is some k’ that is the minimum such k. For many Np problem problems the difficulty of the decision problem is related to the ratio of k to k’. (Not all decision problems are like this… those that are are on the complexity class APX. As a quick rule of thumb however this works) In terms of airline scheduling this means that if you more crews than you strictly need, your algorithm is probably going to successfully schedule them. When the number of crews drops below some critical value… all hell breaks loose. Thinking about this in complexity terms is useful I think. He issue isn’t just bad software… with a small enough number of crews, all software is bad, unless P = NP
- maxerickson 4y ago
  I wonder if the difficulty at this point is algorithmic, or if they even have the data to make "reasonable" decisions, never mind optimizing them. Like what do they do with the bags that are sitting on the plane for cancelled flights? Store them where they are? Figure out their origin and start queuing them to be sent there? Try to get information from the passenger about where to send it? And then with planes and crews, do they even have "potential" flights where they would be able to reliably seat a worthwhile number of passengers?
  - chasd00 4y ago
    yeah it sounds like recovering from this requires standing up a brand new organization in real time. I hope someone is watching the trouble ticket system, imagine that going down...
    - deleted 4y ago
      [deleted]
  - bodhiandphysics 4y ago
    I'm sure they know exactly where every plane and every flight crew is at every moment. My point about algorithmic complexity is that even with perfect knowledge, fixing this is a computationally difficult problem. The idea was to show how the properties of algorithms that perform combinatorial optimization actually matter!
deleted 4y ago
[deleted]
hrunt 4y ago
I am affected by this. We got halfway into our flight only to find our next leg was cancelled. SWA will not (cannot?) rebook anyone until the 31st. Our return flight was going to be Sunday, so we rebooked from our halfway stop back home on Sunday. Here are some crazy things I have encountered. Rental cars in our city are sold out. Same for cities within two hours drive. The websites will accept reservations, but when you show up, they tell you they have no cars. Because it's the holidays, busses and trains are booked and any flights on other airlines are crazy expensive ($1500 one way). When you are stuck in a city, you are probably truly stuck there. Your only hope of dealing with SWA is waiting in line at check-in or a gate. The phones don't work. Online chat doesn't work. The lines are long and slow. Luggage is hit or miss. If your bag was pulled off a plane, you might find it in baggage claim, but most bags are on a plane or on the tarmac. SWA told us it may be 30 days until we get our luggage. They won't pull bags for people, and the agents that we spoke with acknowledged and felt for people who may have had medicine in them. The workers are as befuddled as the passengers. They have been very nice and as helpful as they can be, but their phones haven't been working and their computer systems have been slow. On Twitter, someone posted a video of the announcement at Houston Hobby about no flights until the 31st and people keeping their receipts for hotels, etc. They said the same thing at our airport. People in the airport are so mad. It's unfortunate because it's not constructive. But tempers are flaring, and frustrated passengers who finally get to talk to an agent end up slowing things down because they spend a lot of time trying to hear something they're not going to hear. Ultimately, this is an operations failure. Companies talk a lot about accountability, but the typical way you hold people accountable is by replacing them with more capable people. It will be very interesting to see if any executives leave SWA over this. If not, I would say that no one was held accountable. To close, my family and I are fine. This is but a minor speed bump in life. No one is dying, and we will see how SWA takes care of the extra expenditures. Some people aren't so lucky. They have meds in bags, or finances that don't allow them to spend multiple nights in a hotel and get Uber trips for a few days. Hopefully, SWA takes care of them, too.
- imchillyb 4y ago
  Every airline, pre-boarding, informs passengers to place any and all necessary medication in their carry on bag. This has been procedure for the last 40 years. There is no excuse that is feasable or plausible for 'forgetting' important medications in the checked baggage. This is a reading comprehension problem and not an airline issue.
eiiot 4y ago
My family and I are stuck in Long Beach after our flight was cancelled. We called the airline and the next available flight is Saturday. No hotel compensation, no partnerships with other airlines for rebooking on another flight. What a nightmare.
- samename 4y ago
  I just found out they have to provide some compensation or reimbursement, eventually: https://www.transportation.gov/airconsumer/airline-customer-service-dashboard?carrier_target_id=29851 https://www.transportation.gov/airconsumer/airline-customer-... Save your receipts, and submit a reimbursement request. Escalate if needed.
  - mdavidn 4y ago
    That page describes Southwest's obligations after "controllable cancellations." When a cancellation is outside their control, Southwest only offers to "seek to arrange a discount" on accommodations. They list "weather" and "FAA-required crew duty limitations" as elements outside their control.
    - hrunt 4y ago
      SWA has been telling affected passengers to keep their receipts for reimbursement later, so hopefully once things clear up, they will go beyond the letter of their policy to help make things right with their customers. No guarantees, but also not many options otherwise for those of us affected. Their brand is going to need it, so whatever that cost is will probably be worth it for them.
    - 650REDHAIR 4y ago
      This hasn’t been weather related in days.
  - geekrax 4y ago
    Or use credit card charge back and let them fight for you.
- 4y ago
jeffrallen 4y ago
You get what you pay for. I suspect, weather permitting, NetJets is still flying. I fly very rarely. One of the reasons is that I cannot afford to fly on carriers who charge the correct fare for sufficient quality that actually reliably gets you to your destination. For destinations on the local rail network, I can afford it.
vhiremath4 4y ago
I received a notification that my flight had been cancelled from North Carolina to New York at 12am for a 12pm flight today. My “software incident” senses started tingling and I immediately redeemed credit and rented a car instead (currently on the road). So happy I did that.
- kylehotchkiss 4y ago
  You should have taken the train and then you would have gotten more HN time in!
  - vhiremath4 4y ago
    Damn this is a much better idea. Next time!
uptownfunk 4y ago
They need a new CTO, there is no reason any of this could not have been prevented by a solid engineering design. If we can have autonomous vehicles, rockets that can be reused, social media like tiktok, insta, snap, Twitter then no reason we can’t have mission critical systems to support southwest (or any airline for that matter) solved by technology
- bob1029 4y ago
  > there is no reason any of this could not have been prevented by a solid engineering design Not unless this new CTO's roadmap includes development of a time machine. Engineering design alone will not prevent this kind of outcome.
  - uptownfunk 4y ago
    There can be a lot of improvements made to the current experience. Why couldn’t I rebook my cancelled flight through the app? Why couldn’t I be told which baggage claim to reclaim the bags from my cancelled flight? Why did I have to wait two hours to talk to a southwest agent to get a refund? Why did I have to ask for southwest to reimburse me for the damages? Why couldn’t all of these things be handled easily through the app? Maybe it won’t prevent my flight from being cancelled, but when they do cancel my flight I’m not left scrambling to figure a plan B so I don’t miss meeting my family for the holidays.
    - deleted 4y ago
      [deleted]
- chasd00 4y ago
  I've made this same comment in other replies but none of those examples present the kind of labor issues an airline has. If your essential workers don't show up / quit and operations grind to a halt then no amount of technology is going to fix that
- eightysixfour 4y ago
  The comparisons you chose are terrible. We don't have widespread autonomous vehicles. Rockets are complicated but primarily constrained by knowable physics problems. Social media sites are entirely digital systems that do not need to deal with the complexity of the real world and, where they do (like networking connections), they're built on abstractions to manage that complexity. Logistics problems, especially the logistics of moving humans, are both complicated and complex. You are talking about systems which must interact with manual, human-in-the-loop processes every step of the way. The feedback loops between the digital systems and physical systems are often loose and very costly to tighten (regulations, union rules, passenger behaviors) and the underlying systems that all participants must interact with were some of the very first widespread digital systems. Today's problem was probably preventable, but don't underestimate how hard the problems become as soon as a system begins to depend on humans and the real world.
smm11 4y ago
I don't understand this. I just flew from West to East Coast, then back, on another airline. Left on time from origin and transfer airport both ways, and arrived early on all flights. The "bomb cyclone" has been gone days, and SW passengers are still stuck, sleeping on floors, or have their bags hundreds of miles away.
- pc86 4y ago
  Southwest is just not a good airline.
  - silisili 4y ago
    In my 20 or so years of travel, Southwest has never been a good airline. But what more boggles my mind is that they've never been the cheapest, either(ignoring spirit and the like). Usually American or United was. Was that just where I lived (primarily in the east), or am I missing something?
    - mdavidn 4y ago
      I live on the west cost. When I compare United and Southwest, United is cheaper on face value but more expensive with bags. They hide some of total cost in fees.
      - silisili 4y ago
        That's a good point. I rarely travel domestically with checked luggage.
    - bmitc 4y ago
      There’s no way Southwest hasn’t been the cheapest. I just cannot see that being possible given that they don’t nickel and dime you for everything. United is by far the worst airline imaginable. I actively will not fly with them after being stuck on an airplane with them for 19 hours straight due to their fuckup, only to have to spend a night in New Jersey at a terrible hotel because we then missed our connecting flight (obviously). In the end, all they offered was a measly certificate that required you to use it at United (and was the equivalent of a mere fraction of the total flight cost).
dang 4y ago
Recent and related: Massive Southwest Airlines disruption leaves customers stranded - https://news.ycombinator.com/item?id=34145286 https://news.ycombinator.com/item?id=34145286 - Dec 2022 (102 comments)
paultopia 4y ago
So, like, what do we think... I have a flight on Jan 3. Book a different airline now or pray they've gotten their heads out of their asses by then?
- elijaht 4y ago
  I would hedge with a refundable ticket to a different airline. If Southwest is still having issues you might be able to get the other ticket comped to some degree and still have a flight, if they aren't having issues you can refund the other fare
  - paultopia 4y ago
    Ooh, good idea. Thank you.
11235813213455 4y ago
This saves millions of tonnes of CO2 and pollution
- Havoc 4y ago
  Temporarily. In longer run it’s likely worse. The vast majority of people will fly later/different airlines
- horns4lyfe 4y ago
  Yes, because these people will permanently settle where they’re stuck, no doubt.
  - 11235813213455 4y ago
    Or they may understand that these extreme cold events are caused by climate change (just one seat in one plane round trip pollutes as much as my whole in 1 year (400kg CO2))
vincent___ 4y ago
[flagged]
vincent___ 4y ago
[flagged]
vincent___ 4y ago
[flagged]
lancesells 4y ago
That's an incredible amount of people traveling. Even at 100 people a flight that's 540,000. Half a million people.
bwooceli 4y ago
And the communication is a nightmare. My mom learned that her flight was cancelled Monday via text message, but there is NOTHING on the website indicating the cancellation. I spent 3 hours on hold to get to a rep who rebooked her for a Thursday flight which was also cancelled today. It says to go to a specific southwest.com page, which just takes you to the standard reservation page which, again, has nothing about the cancellation (looks all green). And unless you go to a "travel hardships" page and enter your reservation number AGAIN, you're funneled into their standard cancellation for flight credit (vs. a refund on the original form of payment). It's a total cluster.
crmd 4y ago
Maybe the fact that one airline was allowed to be large enough to cancel 2,750 flights per day is part of the problem.
deleted 4y ago
[deleted]
anm89 4y ago
There should be MAJOR fines and penalties here. The reasons stuff like this happens all the time, is because it's just a business expense to the company, while they strand real people in incredibly difficult and often dangerous and occasionally life threatening situations. But for the airline, it's just good business. Would 100% support an executive getting a long prison sentence on this as well. I know it will never happen but serious consequences are the only way stuff like this stops happening.
pifm_guy 4y ago
I suspect the root cause will turn out to be 'disgruntled employee ran rm -rf and deleted the backups'... And they did it over Christmas and during big storms to maximize the impact to the company.
maherbeg 4y ago
Oh boy we’re affected by this too. This is infinitely more frustrating with a toddler. Luckily we found a flight out at an airport 2.5hrs away and have a friend who can drive us. Every rental car in Nebraska is unavailable for the next few days which is crazy.
hindsightbias 4y ago
After several years of this stuff, this is the shape of things to come. Get used to it. There aren’t enough pilots in the pipeline and your neighborhood is doing everything it can to shut down your local GA airport where people actually learn to fly.
dekhn 4y ago
Airlines seem to run their systems at very close to 100% capacity at nearly all times which is sort of the worst approach for resilience in the face of large disasters.
VectorLock 4y ago
The thought of a business as big as Southwest having to do a hard reboot is wild to me.
zahma 4y ago
Buttigieg has gotten involved, and based on that, I sense a Congressional hearing is in the works. This debacle has affected way too many people at the worst time of year for it to escape public vivisection.
cat_plus_plus 4y ago
Isn't this hacker news? Weather happened, now lots of planes and flight crews are not at airports they are supposed to be at for the next flight. Southwest probably can only fly specific routes at specific time due to air traffic congestion and regulatory approvals, so can't always fly on a straight line to correct location. By the time they get to the correct place, planes may need maintenance and crews may need sleep. Solving this in close to optimum time is probably an NP-hard problem and company's existing software and employee training is not up to handling this edge case. So there are probably a lot of people pencilling in their best guess for who should go where and calling crews on the phone to let them know. Might take a while to get back to normal.
nunez 4y ago
SWA averaged about 4100 flights/day in 2021. 5400 flight cancellations is about half of this load. irrops and cancellations during holidays has unfortunately been normal especially since COVID but I'm guessing fewer employees and the winter storm would cause this to happen.

584 points by edward 4y ago

Astonishing. Based on reports from employees on Reddit (https://www.reddit.com/r/SouthwestAirlines/comments/zw32yt/psa_from_a_swa_employee_since_the_company_wont/ https://www.reddit.com/r/SouthwestAirlines/comments/zw32yt/p...), the actual story here is that Southwest has had a complete operational failure and is simply incapable of determining where their crews are, what flights they are eligible for, and associating them with flights on the schedule. It sounds like there is a semi-automated system which broke somehow, or more likely failed to support the load of cascading changes that resulted from weather disruptions, and they simply don't have the capacity or flexibility to deal with this other than by cancelling flights for three days until they can sort everything out manually. Reading between the lines a bit, one possible root cause is that their semi-automated system required crew members to update their status by phone when something goes wrong (perhaps with another human in the loop), and the sheer volume of disruption overloaded their phone system and resulted in the automated system becoming completely decoupled from the actual state of reality without sufficient bandwidth in the phone channel to get it back in sync in a reasonable amount of time.

0x445442 4y ago
Yeah the storm angle is a lie.
- BitwiseFool 4y ago
  This is hearsay, but I believe any cancellations due to weather are not subject to fines and fees from the FAA and the airline is not obligated to provide more generous compensation/accommodation for customers. So if there's any reasonable way to blame a failure on inclement weather airline companies will do so without hesitation.
  - 0x445442 4y ago
    You’re wrong dude. https://www.reddit.com/r/SouthwestAirlines/comments/zw6upo/hey_guys_swa_pilot_with_a_little_information/?utm_source=share&utm_medium=ios_app&utm_name=iossmf https://www.reddit.com/r/SouthwestAirlines/comments/zw6upo/h...
    - ipython 4y ago
      How does that post in any way refute the parents point? I agree with the parent that management will try like hell to blame this on the weather- heck they already are. Just check out their press releases.
      - 0x445442 4y ago
        Yeah I’m a bit confused how the comment even relates to mine. I took it to mean that my assertion that laying blame on the weather for this mess is bogus and a lie. But that assumption may be wrong.
    - ThunderSizzle 4y ago
      He's not wrong. Every airline will attempt to blame operational issues on non-finable causes, such as weather.
      - tedunangst 4y ago
        So what are the finable causes for flight cancellation?
  - bumby 4y ago
    >cancellations due to weather are not subject to fines and fees from the FAA I thought this too, but apparently there are no federal regulations other than reimbursement for the direct costs. "Airlines are not required to provide passengers with money or other compensation for costs that fall outside of the cancelled airline ticket and fees tied directly to the airline ticket (such as baggage fees, seat upgrades, etc.) when flights are cancelled." [1] "Each airline has its own policies about what it will do for customers on bumped or cancelled flights. There are no federal requirements." [2] However, there are rules around "bumping" passengers on a flight. "An airline is required to compensate you after involuntarily bumping you from an oversold flight in certain situations. However, there are many situations where you are not entitled to compensation." [3] [1] https://www.transportation.gov/individuals/aviation-consumer-protection/flight-delays-cancellations https://www.transportation.gov/individuals/aviation-consumer... [2] https://www.faa.gov/faq/what-are-policies-bumped-or-cancelled-flights-0 https://www.faa.gov/faq/what-are-policies-bumped-or-cancelle... [3] https://www.transportation.gov/individuals/aviation-consumer-protection/bumping-oversales https://www.transportation.gov/individuals/aviation-consumer...
- kube-system 4y ago
  It is likely both. Scheduling is quite easy when it goes normally. Scheduling becomes a more difficult task when more exceptions happen. That reddit post is by is some boots-on-the-ground employee, not someone who understands the algorithmic details of SWAs somewhat unique routing challenges.
  - phpisthebest 4y ago
    It seems they do their scheduling like they do their seating... it is all Free for All
    - bombcar 4y ago
      Should just text all pilots to bum-rush any plane they’re equipped to fly. Why not add the two front seats into the free for all?
      - kube-system 4y ago
        You joke, but being able to swap any pilot to any plane is part of the reason their fleet is 100% 737s.
  - EFreethought 4y ago
    > Scheduling is quite easy when it goes normally. Scheduling becomes a more difficult task when more exceptions happen. I think some slack/spare capacity or redundancy somewhere would make a LOT of problems less likely to happen. Running lean only works out when nothing goes wrong. I know I should not be shocked by the extent of human stupidity, but why the "business" types do not get this is bizarre to me. And paying people more would help too.
    - HDThoreaun 4y ago
      The airline business is too competitive for labor slack. People put up with spirit’s bullshit because it’s $5 cheaper. Any airline that incorporates slack will immediately find itself without customers. If they make it to a catastrophe it would work out, but it’s extremely unlikely that they will be on business that long.
CharlesW 4y ago
According to that post they also have no idea where bags are: "Checked bags are currently a disaster. Plan to not see your checked luggage for at least a month. In the interest of 100% transparency, some bags will be 30+ days lost in the system."
deleted 4y ago
[deleted]
shmatt 4y ago
im kind of doubting that thread, not necessarily fraud but not every employee has the full picture * this memo[1] from the SWA VP thats circulating dated December 21st calling a staffing emergency saying all hell is about to break loose. I'm pretty sure that guy didn't know of the software crash before it happened * Every single piece of checked in luggage is going to take 30+ days to be found? Weird none of the extensive press has mentioned people not getting their bag. Also its pretty much an airport thing and not an airline thing. Airports are in charge of deplaning your luggage and bringing it to you * They're running 40% of flights completely manually with 0 scheduling software? That's extremely impressive [1] https://mobile.twitter.com/ruthschmidt/status/1607460985861988352 https://mobile.twitter.com/ruthschmidt/status/16074609858619...
- phpisthebest 4y ago
  There have been plently of reporting about the bag issue, including a Chicago local reporter posting a video of 100's or maybe even 1000's of bags in chicago most of southwest tags on them It appears there was a MASSIVE disconnect between baggage and people, bags are ending up all over the place even though the people never left.
  - tiahura 4y ago
    Sounds the the perfect setup for an AirTag commercial.
    - bombcar 4y ago
      iPhone alert: there are ten thousand airtags near you
      - giantrobot 4y ago
        Well that at least narrows it down to near you. /s
      - joezydeco 4y ago
        I'm kind of curious about that. If AirTags rely on nearby iPhones to capture tag IDs as they pass by and relay them to Apple, will a ramp worker with an iPhone in his pocket start seeing shorter battery life?
caycep 4y ago
Granted my usually routine flight in 3 days, from BUR to SJC got canceled. And no way to get thru and the usual handy dandy "full refund!" button has been replaced w just an email form with "we'll get back to you!" I sort of believe it
hinkley 4y ago
Somewhere there’s a retired guy sipping his morning coffee and saying, “Told ya.”
- CyberDildonics 4y ago
  There are people actively doing interviews saying that.
pembrook 4y ago
Incidents like this go a long way toward teaching corporate America that software shouldn't be some forgotten "IT" vertical on your org chart that you staff with random contractors. Software is core to your business. If you don't invest in it, you're going to pay the price.
- bmitc 4y ago
  People mistaking software as something that just does something is the problem. Software is stored and executable knowledge, so if some component of that breaks down, you need to be able to pull it from somewhere else. Otherwise, you get situations like this.
- lezojeda 4y ago
  They are already pretty resentful at us because we have such high salaries compared to the rest of the population... I bet that nobody is willing to give any penny more to their IT departments
hintymad 4y ago
> the actual story here is that Southwest has had a complete operational failure and is simply incapable of determining where their crews are, what flights they are eligible for, and associating them with flights on the schedule. This is amazing. Southwest was famous for its operational efficiency and quality. Companies eagerly learned from them. I wonder what has changed.
- iamtheworstdev 4y ago
  it's definitely epic. I know this is anecdotal but a friend of mine and pilot for Southwest had to pay for his own hotel room after he captained a flight a couple of days ago. SW apparently thought he was in another state (and it wasn't a neighboring one) even though he just flew the flight that they scheduled him on. Not long (hours) after that he was told to drive to the nearest Southwest physical location and check in. Rumor is that all Southwest pilots were told that a physical presence was required to verify location.
- GavinMcG 4y ago
  Nothing necessarily changed. Efficiency can become fragility, and quality is rarely measured in a way that includes rare events—partly by definition, and partly because we tend to ignore externalities and unlikely scenarios when we make evaluations.
achow 4y ago
The Reddit post is now deleted. Saved screenshot: https://i.imgur.com/bchgmm7.jpeg https://i.imgur.com/bchgmm7.jpeg
pwillia7 4y ago
Unremoved reddit post: On behalf of all employees: WE ARE SORRY! I will give it to you straight- this meltdown was caused entirely by Southwest. It was triggered by the storm, but the failure to recover quickly is on Southwest 100%. If you are still hearing “weather” almost a week after the storm, it’s not true. Couple main points: 1. Please be patient with us. We desperately want to do everything we can to get you where you’re going. 2. This shitstorm is because the crew scheduling software went belly up and it almost all has to be unraveled over the phone with crew members calling scheduling. If we had better technology which eliminated the need for phone calls, this would have been fixed by now. 3. If you are able to find alternative transportation to your final destination- DO IT. Another airline, bus, train, Lyft, rental car, ANYTHING. Southwest WILL NOT be able to get you to your destination anytime in the next few days. 4. Like I said, it’s gonna take at least a week to get back to normal operations for Southwest. If anyone has questions, I will try to answer them. I work ground ops at one of SWA’s hubs. EDIT FOR FAQs—— Checked bags are currently a disaster. Plan to not see your checked luggage for at least a month. In the interest of 100% transparency, some bags will be 30+ days lost in the system. Will my flight for X date go out? Next 3 days- plan on a cancellation. 4-7 days- likely to go as scheduled. 7+ days- should see operational recovery.
wwweston 4y ago
Other Reddit discussion (with the occasional insider chiming in): https://www.reddit.com/r/bestof/comments/zwd1fq/u4sammich_explains_why_southwest_airlines_is/ https://www.reddit.com/r/bestof/comments/zwd1fq/u4sammich_ex... https://www.reddit.com/r/flying/comments/zw5lsl/southwest_pilots_hows_it_going/ https://www.reddit.com/r/flying/comments/zw5lsl/southwest_pi...
indymike 4y ago
> Southwest has had a complete operational failure and is simply incapable of determining where their crews are, what flights they are eligible for, and associating them with flights on the schedule. This has to be really hard as most systems were probably built assuming something that smelled like full staffing. Pilots can only fly X hours, and flight attendants usually can't be forced to take overtime ... there have to be similar constraints on ground crew, maintenance and gate workers, too. So when you are planning for an understaffed airline, it has to be extremely difficult to optimize where the failures will occur.
mysterydip 4y ago
Somewhere buried in a million lines of code is the comment "// this should never happen" that is in the block currently being executed.
- shapefrog 4y ago
  // add fix for snowstorm here

0x445442 4y ago

This is BS. The storm was last week.

pimlottc 4y ago
Effects from the storm linger. Buffalo’s airport shut down due to weather issues and remains closed through at least Wednesday.
- josephcsible 4y ago
  There's also the issue that a storm, no matter how bad, shouldn't make them forget where they left their crew members and your bags.
- 0x445442 4y ago
  In the case of Southwest the scope of the issues is not because of the weather.
  - EMM_386 4y ago
    It actually is because of the weather. The storms left planes and crew scattered around at all the wrong airports, so they essentially have to reboot the system. Other airlines are not affected by this as much as they fly out of hubs.
- JustLurking2022 4y ago
  Seems they only linger for Southwest, as they made up 90% of all flight cancellations today.

deleted 4y ago

[deleted]

sys32768 4y ago

Our local airport closed yesterday due to icy runways. Planes would circle a while then refuel 100s of miles away, then return to try again. We don't often see freezing rain here. It creates a hard, bubbly layer of ice that cannot be scraped off with the usual tool.

Kon-Peki 4y ago
Get out the curling rocks, eh!

mensetmanusman 4y ago

A software bug can destroy a company these days.

derwiki 4y ago
Check out the Knight Capital software bug in 2012 that ate a cool 440m and almost made them go bankrupt

awillen 4y ago

I can't for the life of me understand how they're allowed to say this is due to weather (and thus not provide the same kind of compensation they'd have to if it were "Southwest's fault"). Sure, weather is the precipitating (pun intended, sorry) cause, but given that other airlines are almost all at <10% cancellations, clearly Southwest's total lack of operational robustness and competence are the real issue. The CEO himself emailed Southwest staff and explained that one big problem is their antiquated computer systems - that is unequivocally their fault (and very personally the CEO's fault, since it's very much his job to direct the sort of large scale financial investment to do something like overhaul their system). I hope that DOT follows through on their earlier statement that they're going to investigate this. The CEO of Southwest has ruined more Christmases than the Grinch. Failures like this should trigger incredibly severe consequences (tickets refunded at 4x cost, $20/day paid to any passenger whose luggage is lost until they get it, lost slots in airports) that make it absolutely irrational to operate in the manner Southwest is.

deleted 4y ago
[deleted]
fairity 4y ago
> Failures like this should trigger incredibly severe consequences (tickets refunded at 4x cost, $20/day paid to any passenger whose luggage is lost until they get it, lost slots in airports) that make it absolutely irrational to operate in the manner Southwest is No need to implement the penalties you described if you’re simply trying to align incentives. Their market cap has fallen by $1b, primarily due to this incident.
- awillen 4y ago
  Their market cap today isn't what matters - if people forget about this and continue to fly Southwest (which, let's be honest, many will), their share price will recover when they announce earnings. The reality is they save a lot of money by keeping virtually no slack in their system and not doing things like scaling up customer service during the holidays. If they lose some money in flight cancellations, it's still rational for them to operate like this. Plus the reality is they probably won't actually lose much - most people whose flights were cancelled will still end up taking Southwest because it's too expensive (or not possible) to book last minute tickets on other airlines. That means they just push them into the very end of Dec/early Jan when they have open capacity anyway.

ProjectArcturis 4y ago

I have always really wondered how airline scheduling software works. Despite being pretty good with algorithms, I just have no idea how you'd make a system that's robust to weather and mechanical delays.

castratikron 4y ago
Sounds like they don't have much of an idea either.
twawaaay 4y ago
I don't have experience with airline scheduling but I have experience with software in large financial corporations like banks. The situation with banks (at least the ones I worked with) is that there is a huge amount of software maintained mostly by mediocre to bad teams. These teams fail a lot, the software fails a lot, and yet everything seems to keep going. It is not about code or algorithm quality, it is about procedural side of things -- how the organisation is "programmed" to respond to failures. I use the word "programmed" in a very broad sense -- for me setting up a paper checklist and being able to rely on people to follow it is the same as programming. I suspect the main difference between airlines and banks is that banks can afford to throw money on the problem and just get things done regardless of how inefficiently. Airlines are in the much worse position -- they were able to afford being inefficient and throwing money at the problem in the past but can't do it anymore. They work with old, outdated software that wasn't built with efficiency in mind but now don't have funds to change it and are forced to maintain what they have. This may also be the answer to why sometimes they just don't have capacity to react to problem and let it cascade to bring everything to a halt.
- blevin 4y ago
  Instead of framing this as saying they "don't have" the funds and were "forced" into inaction, another framing that the board of directors must consider is that current SWA leadership failed in their responsibility to recognize, prioritize, and manage the actual needs of their decades old business.

htag 4y ago

Is this a problem with the weather, with the scheduling software, or with the recent shrink then expand of airline demand? Either way, I don't think I'll book a flight with SW for a long, long time.

alistairSH 4y ago
Are there any airlines that haven’t had similar problems over the last 5-10 years? Seems like they’ve all had issues. And if they haven’t, it’s just a matter of time.

LeifCarrotson 4y ago

Same thing happened on July 20, 2016: https://www.dallasnews.com/business/local-companies/2016/08/10/cost-of-southwest-s-tech-outage-climbs-to-at-least-54-million/ https://www.dallasnews.com/business/local-companies/2016/08/... at the time, then-CEO now-chairman Gary Kelly said: > "What's unique is the partial failure, it's never happened," he said. "This isn't a drill you can run." https://web.archive.org/web/20161112192103/http://www.dallasnews.com/business/southwest-airlines/2016/07/30/southwest-ceo-router-failure-grounded-flights-equated-thousand-year-flood https://web.archive.org/web/20161112192103/http://www.dallas... Delta had a similar outage due to a datacenter fire, grounding all domestic flights. Southwest was uniquely slow in taking days to start up again. And if the way my American Airlines ticket switched my birthdate to January 1st, 2000 is any indication, many airlines still need to modernize.

vrc 4y ago
Google runs Disaster Recovery Training annually (DiRT) where security teams are tasked with simulating these “black swan” events. Seems like this practice needs to expand to more industries.
- bobkazamakis 4y ago
  >Seems like this practice needs to expand to more industries. I think you've mistaken this for something immediately increases quarterly gains with no regard to long-term strategy.
  - Analemma_ 4y ago
    In that case the government should smash Southwest with a billion dollar fine so the cost of not doing this drilling exceeds the cost of doing it.
    - hallway_monitor 4y ago
      Asking the government to step in for additional regulation is rarely helpful. For this type of failure, the free market will determine whether processes and tools improve, or whether the status quo is good enough.

AlbertCory 4y ago

From personal experience: I used to preferentially use SWA because they were great. Over the years, it seems to have become that every single flight is significantly delayed. Now I preferentially don't use them.

el_benhameen 4y ago
I love not being nickel and dimed by them, and the companion pass makes them significantly cheaper for family travel, but I absolutely avoid holiday travel with them, especially winter holidays.

asah 4y ago

tl;dr: distributed systems are hard.

chasd00 4y ago
if i remember my lore correctly it's not a distributed system. It was originally either a mainframe or like AS400 application. I want to say it was related to Saber somehow... maybe they leased it. When i interviewed there years ago they were porting it to some kind of Java stack or maybe they had already ported it and were building it out further. On a tangent, I have to admit, their interview was the best one i've ever had. Very competent people, well rounded process, it was actually more fun than stressful but super challenging too. / this was a lonng time ago, like 10+ years, it could be a very very different animal now
- bronson 4y ago
  I don't think Southwest was ever on Sabre. Their scheduling used to be in-house, now on Amadeus. Computerized operations started with Braniff's software and have ... ah, grown since then. Name almost any technology and it's probably running somewhere in their stack.
francisofascii 4y ago
or maybe you meant to say logistics with little slack is hard
horns4lyfe 4y ago
Especially when you outsource it to the lowest bidding WITCH shop you can find. Incredible that the US is still pumping out MBAs that see IT as a cost center. If you are of a sufficient size and depend on software to function, you are a software company and need to allocate your resources accordingly.

DoingSomeThings 4y ago

Incredible to see that 14 months ago there was an almost duplicate outage. Clearly the operations teams are running too lean and don't have enough slack to handle predictable weather/sickness events. Seems like something you'd plan for. Even at the 98%+ uptime range https://news.ycombinator.com/item?id=28823774 https://news.ycombinator.com/item?id=28823774

chasd00 4y ago
There's no slack at an airline, it's basically a real time system. Imagine a bottling plant, if one part of the line malfunctions the rest of the line just keeps throwing bottles at the malfunctioning unit until the stop button is hit. There's no way to build in artificial delay to those kinds of systems. Even if you could do that at an airline the margins are so razor thin. I bet there's a dollar value assigned to every second a plane is not in the air.
- kibwen 4y ago
  > Even if you could do that at an airline the margins are so razor thin. I bet there's a dollar value assigned to every second a plane is not in the air. You either pay the predictable, ongoing cost or you pay the massive, unpredictable cost like they are currently. There's no free lunch. That is, unless you're an executive looking to boost short-term profits by eliminating "redundancy" and then peacing out before the whole thing collapses from your short-sighted, greedy ineptitude.
  - dehrmann 4y ago
    People don't like to hear this now, but airlines chose the right thing to optimize for. Air travel is rarely that important to have these redundancies, and they vote with their wallets every time they buy economy tickets. Given that safe is a baseline, most people value price over reliability.
    - ThunderSizzle 4y ago
      In my experience, every airline will screw you if your not their frequent flyer. Fly once a year? No one is going to waive a fee or refund you anything. Might as well go for cheapest price. My wife's experience shows SWA as an exception in that.

chasd00 4y ago

years ago i interviewed for, and was given an offer, to work on this very system. glad i didn't take it. I feel bad for the devs trying to figure out what's going on.

bronson 4y ago
It's an absolute nightmare of fixed-field records and XML (Southwest uses Amadeus for scheduling but Sabre is just as bad; and AFAIK operations at all airlines are a chaotic hodge podge). Southwest performs all operations in Central Time because nobody's figured out how to reliably add time zones to all systems all at once. This means they have something like a 34 hour operational day, and every flight must take off and land in the same operational day. It's a satisfying job for the right personality type!
- encoderer 4y ago
  My god. This is terrible but also riveting. I wish there was a website devoted to collecting stories of how these internal systems really work. Airlines. IRS. credit agencies. Etc.
  - jeffrallen 4y ago
    It's called "The Daily WTF".
  - hnu0847 4y ago
    Agreed, I hope some article, book, documentary, etc. is eventually published about this.
- cratermoon 4y ago
  In some recent work I've been doing for an airline, I've worked with 5 different and incompatible ways to represent a passenger, depending on what part of the system you're working in.
  - bronson 4y ago

jrochkind1 4y ago

> Other issues that have exacerbated the airline's struggle to accommodate the holiday rush include problems with "connecting flight crews to their schedules," Perry said. That issue has made it difficult for employees to access crew scheduling services and get reassignments. Wait, what issue? Those words didn't actually say anything. Is this implying that it was a software failure with the scheduling software?

code_runner 4y ago
Sounds like it’s doing more than implying it. The scheduling software seems to be pretty hosed at the moment
- jrochkind1 4y ago
  To me, "problems connecting flight crews to their schedules" does not clearly say it's a software issue, but maybe implies it. But you have other information that it is? Any info on the details? Like, I'd think this is what scheduling software is for...
  - acedTrex 4y ago
    Appears their scheduling software is just woefully inadequate and requires significant human intervention and it just gave up. https://www.reddit.com/r/flying/comments/zw5lsl/comment/j1tne9z/?utm_source=share&utm_medium=web2x&context=3 https://www.reddit.com/r/flying/comments/zw5lsl/comment/j1tn...
    - jrochkind1 4y ago
      I feel like this is a good illustration of the problem with "eh, why does technical debt matter, everything is still working isn't it?" It is, until it's not. And then it's too late.
  - epylar 4y ago
    Pilots literally having to call an 800 number to get their schedule, not enough capacity on the 800 number.

DrWumbo 4y ago

[flagged]

josephcsible 4y ago
Because trains don't need scheduling software or crews?
- sokoloff 4y ago
  One of the problems with SWA’s case is that cabin crews can work a longer duty day than flight crews and airplanes can’t leave on passenger service without a full complement of both, so the airplane scheduling problem might be more complex.
- lostinroutine 4y ago
  I'm guessing the GP means that there will hopefully be an increased interest for alternatives to planes that one can take when there are air travel crises like this.
- ciphol 4y ago
  Trains have major weather issues much less frequently than planes.
  - deleted 4y ago
    [deleted]
  - josephcsible 4y ago
    This storm affected trains too <https://www.trains.com/trn/news-reviews/news-wire/amtrak-cancellations-long-departure-delays-continue/ https://www.trains.com/trn/news-reviews/news-wire/amtrak-can...> <https://chi.streetsblog.org/2022/12/28/the-polar-vortex-express-what-it-was-like-riding-amtrak-during-last-weeks-storm-disruptions/ https://chi.streetsblog.org/2022/12/28/the-polar-vortex-expr...>. What was the last weather event that resulted in significant disruption to air travel but didn't disrupt rail travel?
- overtonwhy 4y ago

RedShift1 4y ago

Is this also one of the companies that completely outsourced their IT?

qwertyuiop_ 4y ago
Yup sweep it under the rug. SW outsourced all of their development and support to WITCH. The VPs who get condos in Bahamas in return will not ever mention that this is the result of a failure of crew scheduling software because they signed off on the outsourcing.
water554 4y ago
Data point delta airlines IT is almost entirely offshore

faangiq 4y ago

Let’s see the C suite asset clawbacks.

supernova87a 4y ago

As I understand it, Southwest is particularly badly affected by such issues because they run a lot of flights that are not in a hub-spoke model, but rather serial flights one non-hub city to the next and next (like eventually coming back in a loop). You can see this by going to Flightaware.com for example, and following back a flight's previous destinations. See for example https://flightaware.com/live/flight/SWA1092/history/20221223/2210Z/KSJC/KSAN https://flightaware.com/live/flight/SWA1092/history/20221223... and "track inbound plane" a couple times. They jump around the country, much less frequently going back to a hub as other airlines do. That means that the planes and crews have a relatively harder time recovering from system-wide disasters because they don't have as part of normal operations as much ability to centralize or pool resources and get people/planes reorganized. (everyone go back to base, consolidate passengers, crew, planes and redeploy them and sort things out in one place) Unfortunate, but that's their model. Good for some purposes, not so good for others. Maybe it's them being quirky and an active choice. I mean, up until a few years ago they did not fly to Hawaii because their scheduling system / people / processes did not want to have redeye flights.

Waterluvian 4y ago
I don’t fly much but I noticed things like “the same flight number takes off at the same time each day and is always the same plane as a different flight number coming the other direction.” It must really help all the employees with routine and consistency even if it’s not optimal.
- Bonooru 4y ago
  Optimal isn't the right way to think about it. It's a tradeoff. Hub-and-spoke is usually better at getting you to your destination in less absolute time given the same number of total flights since you can have more frequent "shuttle flights" that travel to the hub, exchange goods/passengers, and shuttle back. Point-to-point on the other hand, is better for minimizing travel time since you go directly to the destination.
  - thrashh 4y ago

elijaht 4y ago

My datapoint: Had a flight this Thursday from RIC->ATL cancelled this morning. Interesting that they are cancelling flights this far out, where both cities are relatively unaffected by the storm beyond some cold temps. Was a blessing in disguise for me as I could find a similar flight on a different airline. Hope anyone else flying Southwest can find a way to their destination

hrunt 4y ago
The agent this morning told me that they are using the next three days to move staff and planes where they need to be, and the result will be a lot of cancellations when the routes don't serve that. That's why nothing is getting rebooked until Saturday.

boomboomsubban 4y ago

There was a news story a few months ago about some airlines changing the way their crew software works to stop a third party app employees were using to better track their hours Was Southwest one of those airlines?

makestuff 4y ago
Looks like it was American https://news.ycombinator.com/item?id=33233975 https://news.ycombinator.com/item?id=33233975

0x445442 4y ago

So in my attempts to get my family home I’ve discovered that rental car company web sites are atrocious. The direct company sites (not Expedia for example) do not give you the ability to search for vehicles at multiple locations within a mileage radius. You are forced to enter a single location for pickup. This is also true for drop off. From what I can tell, rental car companies are not setup to easily service a one way rentals, which is what most people need right now. I’ve managed to reserve a car for pickup in KC and drop off in Phoenix for tomorrow. The quote for the two days, over $1200. This isn’t really a web site issue but it gives some context. Here’s the baffling bit. I make a reservation this morning at Enterprise in a KC suburb and get back a confirmation number, cool. Five minutes later I get a call back that they don’t have any cars available. I ask him why, because I got the confirmation number. He says the website isn’t accurate at the moment. WTF? So when a car is rented and driven off the lot the database doesn’t update? Or worse, the local dealer data is not synced to corporate in real time? This is a cascading mess. They don’t even have availability in Joplin, MO who’s is 3 hours from KC.

Anechoic 4y ago
From what I can tell, rental car companies are not setup to easily service a one way rentals, Which isn't surprising because at some point, someone needs to bring that car back to it's home base.
- trollied 4y ago
  Indeed. Lots of them are franchises that own their own stock.
  - deleted 4y ago
    [deleted]
  - lotsofpulp 4y ago
    I thought none of the 3 US car rental brand holding companies franchised. But it seems that Avis Budget and Hertz do franchise. Enterprise does not. I wonder if that is why I seem to prefer Enterprise. Not that I would touch Hertz with a 10ft pole anyway.

bodhiandphysics 4y ago

One thing to note is that airline crew scheduling is np-hard (trivially you can reduce Hamiltonian cycle to it). Ak interesting semi-empiricle fact about np-complete problems. Take a decision problem to determine if there is a schedule with k crews. Now there is some k’ that is the minimum such k. For many Np problem problems the difficulty of the decision problem is related to the ratio of k to k’. (Not all decision problems are like this… those that are are on the complexity class APX. As a quick rule of thumb however this works) In terms of airline scheduling this means that if you more crews than you strictly need, your algorithm is probably going to successfully schedule them. When the number of crews drops below some critical value… all hell breaks loose. Thinking about this in complexity terms is useful I think. He issue isn’t just bad software… with a small enough number of crews, all software is bad, unless P = NP

maxerickson 4y ago
I wonder if the difficulty at this point is algorithmic, or if they even have the data to make "reasonable" decisions, never mind optimizing them. Like what do they do with the bags that are sitting on the plane for cancelled flights? Store them where they are? Figure out their origin and start queuing them to be sent there? Try to get information from the passenger about where to send it? And then with planes and crews, do they even have "potential" flights where they would be able to reliably seat a worthwhile number of passengers?
- chasd00 4y ago
  yeah it sounds like recovering from this requires standing up a brand new organization in real time. I hope someone is watching the trouble ticket system, imagine that going down...
  - deleted 4y ago
    [deleted]
- bodhiandphysics 4y ago
  I'm sure they know exactly where every plane and every flight crew is at every moment. My point about algorithmic complexity is that even with perfect knowledge, fixing this is a computationally difficult problem. The idea was to show how the properties of algorithms that perform combinatorial optimization actually matter!

deleted 4y ago

[deleted]

hrunt 4y ago

I am affected by this. We got halfway into our flight only to find our next leg was cancelled. SWA will not (cannot?) rebook anyone until the 31st. Our return flight was going to be Sunday, so we rebooked from our halfway stop back home on Sunday. Here are some crazy things I have encountered. Rental cars in our city are sold out. Same for cities within two hours drive. The websites will accept reservations, but when you show up, they tell you they have no cars. Because it's the holidays, busses and trains are booked and any flights on other airlines are crazy expensive ($1500 one way). When you are stuck in a city, you are probably truly stuck there. Your only hope of dealing with SWA is waiting in line at check-in or a gate. The phones don't work. Online chat doesn't work. The lines are long and slow. Luggage is hit or miss. If your bag was pulled off a plane, you might find it in baggage claim, but most bags are on a plane or on the tarmac. SWA told us it may be 30 days until we get our luggage. They won't pull bags for people, and the agents that we spoke with acknowledged and felt for people who may have had medicine in them. The workers are as befuddled as the passengers. They have been very nice and as helpful as they can be, but their phones haven't been working and their computer systems have been slow. On Twitter, someone posted a video of the announcement at Houston Hobby about no flights until the 31st and people keeping their receipts for hotels, etc. They said the same thing at our airport. People in the airport are so mad. It's unfortunate because it's not constructive. But tempers are flaring, and frustrated passengers who finally get to talk to an agent end up slowing things down because they spend a lot of time trying to hear something they're not going to hear. Ultimately, this is an operations failure. Companies talk a lot about accountability, but the typical way you hold people accountable is by replacing them with more capable people. It will be very interesting to see if any executives leave SWA over this. If not, I would say that no one was held accountable. To close, my family and I are fine. This is but a minor speed bump in life. No one is dying, and we will see how SWA takes care of the extra expenditures. Some people aren't so lucky. They have meds in bags, or finances that don't allow them to spend multiple nights in a hotel and get Uber trips for a few days. Hopefully, SWA takes care of them, too.

imchillyb 4y ago
Every airline, pre-boarding, informs passengers to place any and all necessary medication in their carry on bag. This has been procedure for the last 40 years. There is no excuse that is feasable or plausible for 'forgetting' important medications in the checked baggage. This is a reading comprehension problem and not an airline issue.

eiiot 4y ago

My family and I are stuck in Long Beach after our flight was cancelled. We called the airline and the next available flight is Saturday. No hotel compensation, no partnerships with other airlines for rebooking on another flight. What a nightmare.

samename 4y ago
I just found out they have to provide some compensation or reimbursement, eventually: https://www.transportation.gov/airconsumer/airline-customer-service-dashboard?carrier_target_id=29851 https://www.transportation.gov/airconsumer/airline-customer-... Save your receipts, and submit a reimbursement request. Escalate if needed.
- mdavidn 4y ago
  That page describes Southwest's obligations after "controllable cancellations." When a cancellation is outside their control, Southwest only offers to "seek to arrange a discount" on accommodations. They list "weather" and "FAA-required crew duty limitations" as elements outside their control.
  - hrunt 4y ago
    SWA has been telling affected passengers to keep their receipts for reimbursement later, so hopefully once things clear up, they will go beyond the letter of their policy to help make things right with their customers. No guarantees, but also not many options otherwise for those of us affected. Their brand is going to need it, so whatever that cost is will probably be worth it for them.
  - 650REDHAIR 4y ago
    This hasn’t been weather related in days.
- geekrax 4y ago
  Or use credit card charge back and let them fight for you.
4y ago

jeffrallen 4y ago

You get what you pay for. I suspect, weather permitting, NetJets is still flying. I fly very rarely. One of the reasons is that I cannot afford to fly on carriers who charge the correct fare for sufficient quality that actually reliably gets you to your destination. For destinations on the local rail network, I can afford it.

vhiremath4 4y ago

I received a notification that my flight had been cancelled from North Carolina to New York at 12am for a 12pm flight today. My “software incident” senses started tingling and I immediately redeemed credit and rented a car instead (currently on the road). So happy I did that.

kylehotchkiss 4y ago
You should have taken the train and then you would have gotten more HN time in!
- vhiremath4 4y ago
  Damn this is a much better idea. Next time!

uptownfunk 4y ago

They need a new CTO, there is no reason any of this could not have been prevented by a solid engineering design. If we can have autonomous vehicles, rockets that can be reused, social media like tiktok, insta, snap, Twitter then no reason we can’t have mission critical systems to support southwest (or any airline for that matter) solved by technology

bob1029 4y ago
> there is no reason any of this could not have been prevented by a solid engineering design Not unless this new CTO's roadmap includes development of a time machine. Engineering design alone will not prevent this kind of outcome.
- uptownfunk 4y ago
  There can be a lot of improvements made to the current experience. Why couldn’t I rebook my cancelled flight through the app? Why couldn’t I be told which baggage claim to reclaim the bags from my cancelled flight? Why did I have to wait two hours to talk to a southwest agent to get a refund? Why did I have to ask for southwest to reimburse me for the damages? Why couldn’t all of these things be handled easily through the app? Maybe it won’t prevent my flight from being cancelled, but when they do cancel my flight I’m not left scrambling to figure a plan B so I don’t miss meeting my family for the holidays.
  - deleted 4y ago
    [deleted]
chasd00 4y ago
I've made this same comment in other replies but none of those examples present the kind of labor issues an airline has. If your essential workers don't show up / quit and operations grind to a halt then no amount of technology is going to fix that
eightysixfour 4y ago
The comparisons you chose are terrible. We don't have widespread autonomous vehicles. Rockets are complicated but primarily constrained by knowable physics problems. Social media sites are entirely digital systems that do not need to deal with the complexity of the real world and, where they do (like networking connections), they're built on abstractions to manage that complexity. Logistics problems, especially the logistics of moving humans, are both complicated and complex. You are talking about systems which must interact with manual, human-in-the-loop processes every step of the way. The feedback loops between the digital systems and physical systems are often loose and very costly to tighten (regulations, union rules, passenger behaviors) and the underlying systems that all participants must interact with were some of the very first widespread digital systems. Today's problem was probably preventable, but don't underestimate how hard the problems become as soon as a system begins to depend on humans and the real world.

smm11 4y ago

I don't understand this. I just flew from West to East Coast, then back, on another airline. Left on time from origin and transfer airport both ways, and arrived early on all flights. The "bomb cyclone" has been gone days, and SW passengers are still stuck, sleeping on floors, or have their bags hundreds of miles away.

pc86 4y ago
Southwest is just not a good airline.
- silisili 4y ago
  In my 20 or so years of travel, Southwest has never been a good airline. But what more boggles my mind is that they've never been the cheapest, either(ignoring spirit and the like). Usually American or United was. Was that just where I lived (primarily in the east), or am I missing something?
  - mdavidn 4y ago
    I live on the west cost. When I compare United and Southwest, United is cheaper on face value but more expensive with bags. They hide some of total cost in fees.
    - silisili 4y ago
      That's a good point. I rarely travel domestically with checked luggage.
  - bmitc 4y ago
    There’s no way Southwest hasn’t been the cheapest. I just cannot see that being possible given that they don’t nickel and dime you for everything. United is by far the worst airline imaginable. I actively will not fly with them after being stuck on an airplane with them for 19 hours straight due to their fuckup, only to have to spend a night in New Jersey at a terrible hotel because we then missed our connecting flight (obviously). In the end, all they offered was a measly certificate that required you to use it at United (and was the equivalent of a mere fraction of the total flight cost).

dang 4y ago

Recent and related: Massive Southwest Airlines disruption leaves customers stranded - https://news.ycombinator.com/item?id=34145286 https://news.ycombinator.com/item?id=34145286 - Dec 2022 (102 comments)

paultopia 4y ago

So, like, what do we think... I have a flight on Jan 3. Book a different airline now or pray they've gotten their heads out of their asses by then?

elijaht 4y ago
I would hedge with a refundable ticket to a different airline. If Southwest is still having issues you might be able to get the other ticket comped to some degree and still have a flight, if they aren't having issues you can refund the other fare
- paultopia 4y ago
  Ooh, good idea. Thank you.

11235813213455 4y ago

This saves millions of tonnes of CO2 and pollution

Havoc 4y ago
Temporarily. In longer run it’s likely worse. The vast majority of people will fly later/different airlines
horns4lyfe 4y ago
Yes, because these people will permanently settle where they’re stuck, no doubt.
- 11235813213455 4y ago
  Or they may understand that these extreme cold events are caused by climate change (just one seat in one plane round trip pollutes as much as my whole in 1 year (400kg CO2))

vincent___ 4y ago

[flagged]

vincent___ 4y ago

[flagged]

vincent___ 4y ago

[flagged]

lancesells 4y ago

That's an incredible amount of people traveling. Even at 100 people a flight that's 540,000. Half a million people.

bwooceli 4y ago

And the communication is a nightmare. My mom learned that her flight was cancelled Monday via text message, but there is NOTHING on the website indicating the cancellation. I spent 3 hours on hold to get to a rep who rebooked her for a Thursday flight which was also cancelled today. It says to go to a specific southwest.com page, which just takes you to the standard reservation page which, again, has nothing about the cancellation (looks all green). And unless you go to a "travel hardships" page and enter your reservation number AGAIN, you're funneled into their standard cancellation for flight credit (vs. a refund on the original form of payment). It's a total cluster.

crmd 4y ago

Maybe the fact that one airline was allowed to be large enough to cancel 2,750 flights per day is part of the problem.

deleted 4y ago

[deleted]

anm89 4y ago

There should be MAJOR fines and penalties here. The reasons stuff like this happens all the time, is because it's just a business expense to the company, while they strand real people in incredibly difficult and often dangerous and occasionally life threatening situations. But for the airline, it's just good business. Would 100% support an executive getting a long prison sentence on this as well. I know it will never happen but serious consequences are the only way stuff like this stops happening.

pifm_guy 4y ago

I suspect the root cause will turn out to be 'disgruntled employee ran rm -rf and deleted the backups'... And they did it over Christmas and during big storms to maximize the impact to the company.

maherbeg 4y ago

Oh boy we’re affected by this too. This is infinitely more frustrating with a toddler. Luckily we found a flight out at an airport 2.5hrs away and have a friend who can drive us. Every rental car in Nebraska is unavailable for the next few days which is crazy.

hindsightbias 4y ago

After several years of this stuff, this is the shape of things to come. Get used to it. There aren’t enough pilots in the pipeline and your neighborhood is doing everything it can to shut down your local GA airport where people actually learn to fly.

dekhn 4y ago

Airlines seem to run their systems at very close to 100% capacity at nearly all times which is sort of the worst approach for resilience in the face of large disasters.

VectorLock 4y ago

The thought of a business as big as Southwest having to do a hard reboot is wild to me.

zahma 4y ago

Buttigieg has gotten involved, and based on that, I sense a Congressional hearing is in the works. This debacle has affected way too many people at the worst time of year for it to escape public vivisection.

cat_plus_plus 4y ago

Isn't this hacker news? Weather happened, now lots of planes and flight crews are not at airports they are supposed to be at for the next flight. Southwest probably can only fly specific routes at specific time due to air traffic congestion and regulatory approvals, so can't always fly on a straight line to correct location. By the time they get to the correct place, planes may need maintenance and crews may need sleep. Solving this in close to optimum time is probably an NP-hard problem and company's existing software and employee training is not up to handling this edge case. So there are probably a lot of people pencilling in their best guess for who should go where and calling crews on the phone to let them know. Might take a while to get back to normal.

nunez 4y ago

SWA averaged about 4100 flights/day in 2021. 5400 flight cancellations is about half of this load. irrops and cancellations during holidays has unfortunately been normal especially since COVID but I'm guessing fewer employees and the winter storm would cause this to happen.