8 ms·

Airbus A380 – $6.1B in additional costs due to project delays

78 points by jakub_g 9y ago

bsdpython 9y ago
"Internal reviews identified that the heart of the problem was the fact that the different design groups working on the project had used different Computer Aided Design (CAD) software to create the engineering drawings. The development of the aircraft was a collaboration between 16 sites spread across 4 different countries. German and Spanish designers had used one version of the software (CATIA version 4), while British and French teams had upgraded to version 5." Really interesting! While I've hardly worked on anything as complex as an A380, I take it as a high priority for all software packages, from dev to test to prod, all be on the exact same version. Python 3.4 on one box and 3.6 on another? Let's not.
- sixdimensional 9y ago
  I work for an aerospace-related org. You would be shocked at how even within a single org nobody enforces standards across different groups. Even within this one org they use a handful of different CAD systems whether by choice, or as a result of time passing and people and technology changing. Nobody spends the money to bring old drawings forward into newer tech when it changes either, and the CAD files tend to often be proprietary binary formats that are difficult to work with. Really quite annoying and wasteful, but it is the way it is because people don't think about these problems and plan/ spend in advance to avoid them.
  - rjsw 9y ago
    The aerospace companies have spent a lot of money on open CAD formats [1], though it wouldn't have helped in this particular case. [1] https://en.wikipedia.org/wiki/ISO_10303 https://en.wikipedia.org/wiki/ISO_10303
- obstinate 9y ago
  Yep. At my employer, all programs that get deployed to production must be built with either the current version of the toolchain, or the previous version. All builds are hermetic, and it's a requirement that all programs get rebuilt and redeployed at least once every N months, so that the programs out in the wild aren't built with toolchains and libraries that are too wildly different from head. But there's a whole team dedicated to the task of maintaining that state here. I can imagine that at a company where this is not the core competency, there would be some slippage.
- davidf18 9y ago
  In a way it is ironic because in aeronautical engineering everything is well tested with backup systems when it comes to making the airframes, engines, and the entire airplane. Yet, they did not engineer their software system according to the same standards of residency which is a part of engineering often forgotten about by many. One of the great value-added of firms using open source software is indeed the integration testing of various software packages. Lightbend is one example, they do a full Scala stack https://www.lightbend.com/platform https://www.lightbend.com/platform Databricks for Spark, Cloudera and MapR for Spark/Hadoop, etc.
  - davidf18 9y ago
    EDIT: Autocorrect correction: Yet, they did not engineer their software system according to the same standards of ++resiliency++ which is a part of engineering often forgotten about by many.
- hliyan 9y ago
  What's your opinion on docker in production where every component runs on its own container with its own version of dependencies?
- mattmanser 9y ago
  Several years ago I worked with a large multinational company where every site had their own IT teams. A lot of mergers, etc. Probably much like BAE. A firm who's major product is info. Some of those teams were extremely incompetent. My client had made a video of the CEO speaking to the whole company which turned out wouldn't fit on their internal sharepoint site because there was a 50 meg limit for videos. The CEO was not at all impressed with the quality of a 20 min video at 50 meg so in desperation, they ended up asking us to host it for them. But then vimeo didn't work for some sites, then wistia for others, in the end we ended up having to set it up on Amazon's cloudfront. So each site had their own firewall rules, own white-lists, bad IT connections, etc. Throughout this debacle, we got claims of our incompetence by IT personnel at their various sites. Really opened my eyes at how disorganised a large organisation can become, as well as how people can hold positions in large orgs that they have no ability to perform by politicking. And that's how you end up with loads of different versions of software at different sites in the same company. EDIT: Actually, I recall incorrectly, we originally used cloudfront, which didn't work, then vimeo was blocked on some sites, but no-one had blocked wistia yet as they were still a fairly new entrant. Random endorsement: Wistia are great, love their API, love their support, great video hosting company.
  - maxxxxx 9y ago
    I see the same at my company. I think the reason is that the top guys in non-IT industries don't see IT as core competence but as a cost center that ideally gets outsourced to the lowest bidder. Unless there is a very strong CIO different divisions won't talk to each other and no standardization happens. Even if one division does a good job they usually can't sustain that for long because their budgets will get cut. It's a very unhappy situation for everybody and the best people leave after a while.
  - tajen 9y ago
    Not sure politicking is the root reason for Airbus: Catia is just hugely expensive, so upgrading is really a considerable cost. But probably not worth causing $6.1bn delay – Technical debt can bite you in weird ways.
  - dakial1 9y ago
- gumby 9y ago
  > Really interesting! ... I take it as a high priority for all software packages, from dev to test to prod, all be on the exact same version. Python 3.4 on one box and 3.6 on another? Let's not. The canonical way to get around this is unit test and CI. This article shows that airbus did that...and still got screwed. Which shows that your approach is a good one!
stoolpigeon 9y ago
Most comfortable passenger aircraft I've ever flown on. I've only flown them with Emirates - and it may be a combination of their choices for how it is equipped inside along with their service, but either way if I ever have a chance to be on an A-380 I jump at it.
- shawkinaw 9y ago
  I was screwed out of my opportunity to fly in an A380 by JetStar, a horrible discount airline in Australia whose random flight delay made us miss our flight out of Sydney. Ended up on a 747.
- deleted 9y ago
  [deleted]
- caminante 9y ago
  How does it compare to the Dreamliner/Boeing 787? I like the Dreamliner. The ride is noticeably smoother and the improvements to cabin air filtration, humidity management, cabin pressurization at lower altitudes, larger windows, and "mood lighting" make for a more enjoyable ride [0]. [0]https://backpackerlee.wordpress.com/2014/08/28/the-airbus-a380-vs-the-boeing-787-dreamliner/ https://backpackerlee.wordpress.com/2014/08/28/the-airbus-a3...
  - davidmr 9y ago
    They're both magnificent planes, but given the option I will usually take the 380. At least on the upper deck, it's far quieter. Over a 10-15 hour flight, that goes a long way with me.
  - sitepodmatt 9y ago
    All nice bonuses when you've got your knees in your nostrils and elbow locked with the person next to you. Almost all 787s joining fleets have smaller width or legroom than their replacements (whatever they are a380, 777, a330 etc..). There are exceptions Air India is one and I forget the other..
danjc 9y ago
The article states that calculation of bend radii (of wiring/cabling) was the problem. If that was the case, surely one of the two versions of their design software was calculating incorrectly? If so, one would think that the whole team being on the same version wouldn't have helped (if they were on a version that calculated lengths incorrectly). But perhaps there's more subtlety to the calculation discrepancy than is evident in the article.
InTheArena 9y ago
There is a bigger problem. The growth of ETOPS planes (planes with two engines that can fly over the Atlantic or Pacific) with fewer seats that need to be filled for a route to be profitable have basically made this plane irrelevant. This includes the 787, the 777, and Airbus's own A350. Airbus greenlit the project based on original sales projections of 1,200 A380s being sold. So far around three hundred have been sold. In addition Airbus projected a brand new market called "VLA" (very large aircraft >500 seats) would come into existence and demand sixty to seventy airframes a year. They therefore compromised the original design of the A380 to target a re-winged and re-engined stretch version down the line. So far most carriers run under 500 seats, with no-one even coming close the theoretical 850 seats that the current Airbus could do, to say nothing about a next generation A380. Airbus explicitly bet that point to point routes would go away in favor of fortress hubs and hub to hub traffic. The reverse has instead happened. United is making a small fortune now with hub to previously unserved cities in Asia. Norwegian is disrupting the entire Atlantic market by flying 787s into other carriers hubs, destroying any economic advantage that the hubs have. Airbus wanted plane that was "the biggest and the best" for prestige purposes. That led them to green-light the A380. The A308s engineering problems are problems, but what ultimately dooms it is a basic conceptual problem with the plane itself. It's basically a prestige project, and is going to leave Airbus and European tax-payers on the hook. All that said, the A380 is a great plane to fly on. I fly it regularly to Bangkok via Seoul. More often then not, the entire upper deck (which is business only on Korean) is 75% empty.
- rpmcmurphy 9y ago
  I call the A380 the Flytanic. It's not even a very atrractive plane. The 747 is retro now, but it is a beautiful plane. Whenever I can, I fly the 787 on long haul flights. Super comfortable, love it.
  - Joeri 9y ago
    I recently flew economy in a 787 for the first time and it was the most uncomfortable flight I've ever had. Incredibly narrow seats, and way too much cabin noise. I have heard the same thing from other people. I wish airlines would charge a bit more per seat instead of trying to keep shrinking the seats. As it stands you have to pay a lot extra in many cases to get a comfortable seat.
gumby 9y ago
Talk about burying the lede: > In today’s complex integrated supply chains, stories of failed configuration management during the design process are a sadly all too common. This is a good way to kill a lean and virtual startup, and surely has. Also fascinating: the manager who raised the problem in the first place was blocked by upper management and then fired because the problem occurred.
mkh 9y ago
I admit I know virtually nothing about the aircraft industry, but I always wonder why Airbus doesn't do the initially planned cargo version of the A380. The only reason I can imagine is that there aren't enough customers for the cargo version, but why is this so? Isn't air cargo growing? Would an cargo A380 be too expensive? Some comments suggest the A380 has bad fuel efficiency per passenger mile, but some others suggest this is because airlines put less passenger seats in the plane than possible. Surely this can't be a problem for freight planes? What is it?
- jakub_g 9y ago
  Apparently it's about the weight/space ratio which makes is not economical to fly cargo with A380: "An A380-F would be too fat to fly at a profit: The plane would hit the maximum payload (a constraint of weight) before its maximum cubic space (a constraint of volume). Its design can’t support the maximum payload required to generate a profit. (...) The A380-F would be able to carry 60% more volume than the 747, but only 28% more weight. It wouldn’t be fully loaded at typical levels of air cargo density, or at least nothing close to what can be supported by the thrust capacity of the 747." https://www.flexport.com/blog/airbus-a380-no-cargo-equivalent/ https://www.flexport.com/blog/airbus-a380-no-cargo-equivalen...
- rogerbinns 9y ago
  The dedicated cargo freighter market has been in decline for quite a while, but the volume of cargo has been increasing. The reason is cargo goes in the holds of passenger aircraft. This is nice extra money for the airline: the plane was going to go anyway, the crew is paid for, the fuel difference is minimal etc. Remember that planes are already double decker (triple for A380). They look like this: https://qph.ec.quoracdn.net/main-qimg-4c732e2865f73d86b72c5a3a402fa974-c https://qph.ec.quoracdn.net/main-qimg-4c732e2865f73d86b72c5a... The bottom deck has fuel tanks, passenger bags, and space for cargo. The A380 (passenger version) is problematic for cargo because there are two passenger decks which means up to twice the lower deck taken up with baggage leaving less for the cargo.
everybodyknows 9y ago
> ... when the A380 Program Manager attempted to move the German designers onto the same CAD system as the French, he met a wall of resistance. Personal rivalries and national pride are reported to have been issues ... And yet there are other, more objective motives for resistance. As all big-org managers know: 1. Reputational: Belatedly switching to your peer organization's tools suggests you made a poor planning decision earlier. 2. Work shifting: The switch itself will inevitably consume many weeks of calender time, though non-technical upper management will insist on pretending otherwise. While you're struggling to switch, the rival peer org will press its advantage, collecting milestone checkoffs and praise from higher-ups. Alternative is to block the change, so as to bury some considerable share of the tool mismatch pain and delay in the rival org.
tiatia 9y ago
Don't write it off. 1. They say the plane may have been "10 years too early" 2. It will get more fuel efficient engines. https://translate.google.com/translate?sl=it&tl=en&js=y&prev=_t&hl=en&ie=UTF-8&u=https%3A%2F%2Fwww.welt.de%2Fwirtschaft%2Farticle140020669%2FDer-A380-wurde-wohl-zehn-Jahre-zu-frueh-eingefuehrt.html&edit-text= https://translate.google.com/translate?sl=it&tl=en&js=y&prev...
- valuearb 9y ago
  All planes will get more fuel efficient engines. Direct routes are likely to remain dominant.
  - tiatia 9y ago
    Where? In the US? I am not sure this is the case in markets with hight growth rates.
secfirstmd 9y ago
It also important to point out that a lot of airports (for example, Dublin) are not equipped to actually be able to manage the A380. Where as a lot are capable of the 747.
- Tsiklon 9y ago
  An addendum to your point: Aer Lingus (the Irish Flag Carrier) used to fly the 747 out of Dublin.

78 points by jakub_g 9y ago

"Internal reviews identified that the heart of the problem was the fact that the different design groups working on the project had used different Computer Aided Design (CAD) software to create the engineering drawings. The development of the aircraft was a collaboration between 16 sites spread across 4 different countries. German and Spanish designers had used one version of the software (CATIA version 4), while British and French teams had upgraded to version 5." Really interesting! While I've hardly worked on anything as complex as an A380, I take it as a high priority for all software packages, from dev to test to prod, all be on the exact same version. Python 3.4 on one box and 3.6 on another? Let's not.

sixdimensional 9y ago
I work for an aerospace-related org. You would be shocked at how even within a single org nobody enforces standards across different groups. Even within this one org they use a handful of different CAD systems whether by choice, or as a result of time passing and people and technology changing. Nobody spends the money to bring old drawings forward into newer tech when it changes either, and the CAD files tend to often be proprietary binary formats that are difficult to work with. Really quite annoying and wasteful, but it is the way it is because people don't think about these problems and plan/ spend in advance to avoid them.
- rjsw 9y ago
  The aerospace companies have spent a lot of money on open CAD formats [1], though it wouldn't have helped in this particular case. [1] https://en.wikipedia.org/wiki/ISO_10303 https://en.wikipedia.org/wiki/ISO_10303
obstinate 9y ago
Yep. At my employer, all programs that get deployed to production must be built with either the current version of the toolchain, or the previous version. All builds are hermetic, and it's a requirement that all programs get rebuilt and redeployed at least once every N months, so that the programs out in the wild aren't built with toolchains and libraries that are too wildly different from head. But there's a whole team dedicated to the task of maintaining that state here. I can imagine that at a company where this is not the core competency, there would be some slippage.
davidf18 9y ago
In a way it is ironic because in aeronautical engineering everything is well tested with backup systems when it comes to making the airframes, engines, and the entire airplane. Yet, they did not engineer their software system according to the same standards of residency which is a part of engineering often forgotten about by many. One of the great value-added of firms using open source software is indeed the integration testing of various software packages. Lightbend is one example, they do a full Scala stack https://www.lightbend.com/platform https://www.lightbend.com/platform Databricks for Spark, Cloudera and MapR for Spark/Hadoop, etc.
- davidf18 9y ago
  EDIT: Autocorrect correction: Yet, they did not engineer their software system according to the same standards of ++resiliency++ which is a part of engineering often forgotten about by many.
hliyan 9y ago
What's your opinion on docker in production where every component runs on its own container with its own version of dependencies?
mattmanser 9y ago
Several years ago I worked with a large multinational company where every site had their own IT teams. A lot of mergers, etc. Probably much like BAE. A firm who's major product is info. Some of those teams were extremely incompetent. My client had made a video of the CEO speaking to the whole company which turned out wouldn't fit on their internal sharepoint site because there was a 50 meg limit for videos. The CEO was not at all impressed with the quality of a 20 min video at 50 meg so in desperation, they ended up asking us to host it for them. But then vimeo didn't work for some sites, then wistia for others, in the end we ended up having to set it up on Amazon's cloudfront. So each site had their own firewall rules, own white-lists, bad IT connections, etc. Throughout this debacle, we got claims of our incompetence by IT personnel at their various sites. Really opened my eyes at how disorganised a large organisation can become, as well as how people can hold positions in large orgs that they have no ability to perform by politicking. And that's how you end up with loads of different versions of software at different sites in the same company. EDIT: Actually, I recall incorrectly, we originally used cloudfront, which didn't work, then vimeo was blocked on some sites, but no-one had blocked wistia yet as they were still a fairly new entrant. Random endorsement: Wistia are great, love their API, love their support, great video hosting company.
- maxxxxx 9y ago
  I see the same at my company. I think the reason is that the top guys in non-IT industries don't see IT as core competence but as a cost center that ideally gets outsourced to the lowest bidder. Unless there is a very strong CIO different divisions won't talk to each other and no standardization happens. Even if one division does a good job they usually can't sustain that for long because their budgets will get cut. It's a very unhappy situation for everybody and the best people leave after a while.
- tajen 9y ago
  Not sure politicking is the root reason for Airbus: Catia is just hugely expensive, so upgrading is really a considerable cost. But probably not worth causing $6.1bn delay – Technical debt can bite you in weird ways.
- dakial1 9y ago
gumby 9y ago
> Really interesting! ... I take it as a high priority for all software packages, from dev to test to prod, all be on the exact same version. Python 3.4 on one box and 3.6 on another? Let's not. The canonical way to get around this is unit test and CI. This article shows that airbus did that...and still got screwed. Which shows that your approach is a good one!

stoolpigeon 9y ago

Most comfortable passenger aircraft I've ever flown on. I've only flown them with Emirates - and it may be a combination of their choices for how it is equipped inside along with their service, but either way if I ever have a chance to be on an A-380 I jump at it.

shawkinaw 9y ago
I was screwed out of my opportunity to fly in an A380 by JetStar, a horrible discount airline in Australia whose random flight delay made us miss our flight out of Sydney. Ended up on a 747.
deleted 9y ago
[deleted]
caminante 9y ago
How does it compare to the Dreamliner/Boeing 787? I like the Dreamliner. The ride is noticeably smoother and the improvements to cabin air filtration, humidity management, cabin pressurization at lower altitudes, larger windows, and "mood lighting" make for a more enjoyable ride [0]. [0]https://backpackerlee.wordpress.com/2014/08/28/the-airbus-a380-vs-the-boeing-787-dreamliner/ https://backpackerlee.wordpress.com/2014/08/28/the-airbus-a3...
- davidmr 9y ago
  They're both magnificent planes, but given the option I will usually take the 380. At least on the upper deck, it's far quieter. Over a 10-15 hour flight, that goes a long way with me.
- sitepodmatt 9y ago
  All nice bonuses when you've got your knees in your nostrils and elbow locked with the person next to you. Almost all 787s joining fleets have smaller width or legroom than their replacements (whatever they are a380, 777, a330 etc..). There are exceptions Air India is one and I forget the other..

danjc 9y ago

The article states that calculation of bend radii (of wiring/cabling) was the problem. If that was the case, surely one of the two versions of their design software was calculating incorrectly? If so, one would think that the whole team being on the same version wouldn't have helped (if they were on a version that calculated lengths incorrectly). But perhaps there's more subtlety to the calculation discrepancy than is evident in the article.

InTheArena 9y ago

There is a bigger problem. The growth of ETOPS planes (planes with two engines that can fly over the Atlantic or Pacific) with fewer seats that need to be filled for a route to be profitable have basically made this plane irrelevant. This includes the 787, the 777, and Airbus's own A350. Airbus greenlit the project based on original sales projections of 1,200 A380s being sold. So far around three hundred have been sold. In addition Airbus projected a brand new market called "VLA" (very large aircraft >500 seats) would come into existence and demand sixty to seventy airframes a year. They therefore compromised the original design of the A380 to target a re-winged and re-engined stretch version down the line. So far most carriers run under 500 seats, with no-one even coming close the theoretical 850 seats that the current Airbus could do, to say nothing about a next generation A380. Airbus explicitly bet that point to point routes would go away in favor of fortress hubs and hub to hub traffic. The reverse has instead happened. United is making a small fortune now with hub to previously unserved cities in Asia. Norwegian is disrupting the entire Atlantic market by flying 787s into other carriers hubs, destroying any economic advantage that the hubs have. Airbus wanted plane that was "the biggest and the best" for prestige purposes. That led them to green-light the A380. The A308s engineering problems are problems, but what ultimately dooms it is a basic conceptual problem with the plane itself. It's basically a prestige project, and is going to leave Airbus and European tax-payers on the hook. All that said, the A380 is a great plane to fly on. I fly it regularly to Bangkok via Seoul. More often then not, the entire upper deck (which is business only on Korean) is 75% empty.

rpmcmurphy 9y ago
I call the A380 the Flytanic. It's not even a very atrractive plane. The 747 is retro now, but it is a beautiful plane. Whenever I can, I fly the 787 on long haul flights. Super comfortable, love it.
- Joeri 9y ago
  I recently flew economy in a 787 for the first time and it was the most uncomfortable flight I've ever had. Incredibly narrow seats, and way too much cabin noise. I have heard the same thing from other people. I wish airlines would charge a bit more per seat instead of trying to keep shrinking the seats. As it stands you have to pay a lot extra in many cases to get a comfortable seat.

gumby 9y ago

Talk about burying the lede: > In today’s complex integrated supply chains, stories of failed configuration management during the design process are a sadly all too common. This is a good way to kill a lean and virtual startup, and surely has. Also fascinating: the manager who raised the problem in the first place was blocked by upper management and then fired because the problem occurred.

mkh 9y ago

I admit I know virtually nothing about the aircraft industry, but I always wonder why Airbus doesn't do the initially planned cargo version of the A380. The only reason I can imagine is that there aren't enough customers for the cargo version, but why is this so? Isn't air cargo growing? Would an cargo A380 be too expensive? Some comments suggest the A380 has bad fuel efficiency per passenger mile, but some others suggest this is because airlines put less passenger seats in the plane than possible. Surely this can't be a problem for freight planes? What is it?

jakub_g 9y ago
Apparently it's about the weight/space ratio which makes is not economical to fly cargo with A380: "An A380-F would be too fat to fly at a profit: The plane would hit the maximum payload (a constraint of weight) before its maximum cubic space (a constraint of volume). Its design can’t support the maximum payload required to generate a profit. (...) The A380-F would be able to carry 60% more volume than the 747, but only 28% more weight. It wouldn’t be fully loaded at typical levels of air cargo density, or at least nothing close to what can be supported by the thrust capacity of the 747." https://www.flexport.com/blog/airbus-a380-no-cargo-equivalent/ https://www.flexport.com/blog/airbus-a380-no-cargo-equivalen...
rogerbinns 9y ago
The dedicated cargo freighter market has been in decline for quite a while, but the volume of cargo has been increasing. The reason is cargo goes in the holds of passenger aircraft. This is nice extra money for the airline: the plane was going to go anyway, the crew is paid for, the fuel difference is minimal etc. Remember that planes are already double decker (triple for A380). They look like this: https://qph.ec.quoracdn.net/main-qimg-4c732e2865f73d86b72c5a3a402fa974-c https://qph.ec.quoracdn.net/main-qimg-4c732e2865f73d86b72c5a... The bottom deck has fuel tanks, passenger bags, and space for cargo. The A380 (passenger version) is problematic for cargo because there are two passenger decks which means up to twice the lower deck taken up with baggage leaving less for the cargo.

everybodyknows 9y ago

> ... when the A380 Program Manager attempted to move the German designers onto the same CAD system as the French, he met a wall of resistance. Personal rivalries and national pride are reported to have been issues ... And yet there are other, more objective motives for resistance. As all big-org managers know: 1. Reputational: Belatedly switching to your peer organization's tools suggests you made a poor planning decision earlier. 2. Work shifting: The switch itself will inevitably consume many weeks of calender time, though non-technical upper management will insist on pretending otherwise. While you're struggling to switch, the rival peer org will press its advantage, collecting milestone checkoffs and praise from higher-ups. Alternative is to block the change, so as to bury some considerable share of the tool mismatch pain and delay in the rival org.

tiatia 9y ago

Don't write it off. 1. They say the plane may have been "10 years too early" 2. It will get more fuel efficient engines. https://translate.google.com/translate?sl=it&tl=en&js=y&prev=_t&hl=en&ie=UTF-8&u=https%3A%2F%2Fwww.welt.de%2Fwirtschaft%2Farticle140020669%2FDer-A380-wurde-wohl-zehn-Jahre-zu-frueh-eingefuehrt.html&edit-text= https://translate.google.com/translate?sl=it&tl=en&js=y&prev...

valuearb 9y ago
All planes will get more fuel efficient engines. Direct routes are likely to remain dominant.
- tiatia 9y ago
  Where? In the US? I am not sure this is the case in markets with hight growth rates.

secfirstmd 9y ago

It also important to point out that a lot of airports (for example, Dublin) are not equipped to actually be able to manage the A380. Where as a lot are capable of the 747.

Tsiklon 9y ago
An addendum to your point: Aer Lingus (the Irish Flag Carrier) used to fly the 747 out of Dublin.