13 ms·

What you talk about the FAA doing thier own testing and validation is (unfortunately) almost impossible. Back in the 90s, a senior FAA official (I think a Direc

What you talk about the FAA doing thier own testing and validation is (unfortunately) almost impossible. Back in the 90s, a senior FAA official (I think a Director) had said something along the lines of “The FAA does not and cannot check everything, we just see that companies are doing their tests.” The FAA would require several times more manpower to be able to actually audit or test everything, and that is assuming they have the technical skills (which they haven’t had for highly complex systems and electronics for decades now). I’m afraid I’m saying a lot of things from memory from the time I had written a report on the Max 8 accidents and an actor analysis. (I’d anyone is interested I could perhaps share it.) Most of this stuff came from the DoT report on the accident, the rest from reputable news articles.

yason 5y ago
FAA could facilitate smoketesting and focus on parts they deem the most suspicious/experimental/new in a new airplane. And then outsource the actual tests to a third party.
- hef19898 5y ago
  By law, the certifiction activities done by the EASA (I assume the FAA is the same) cannot be done by third parties. It also worth noting, that the aircraft certification procedures worked pretty well for decades, up until Boeing started to lie to the FAA. Not sure how Boeing kept its design org approval after that. EDIT: Some required activities by the organizations to be certified can be outsourced to third parties, the org itself is still accountable.
  - VBprogrammer 5y ago
    > Not sure how Boeing kept its design org approval after that That's an easy one. It's too big to fail. One way or another Boeing had to make it out the other end of the 737 Max disaster intact as an organisation. Anything else would have been unpalatable from the point of view of the American military industrial complex. I know that phrase is usually applied in a derogatory way but here I don't even disagree with the thinking.
    - hef19898 5y ago
      That seems to be the answer. And I kind of get the reasoning. Under EASE rules, there are accountable people at the head of approved design and production orgs. I hope those people at least got their licenses revoked, if something similar exists under FAA rules.
- FridayoLeary 5y ago
  What could possibly go wrong?
picsao 5y ago
They could sample statements sporadically and thus have some checks instead of none?
f6v 5y ago
Like, how many new major passenger planes do they have to test every year? 0.1 on average?
- TheSpiceIsLife 5y ago
  Or, more precisely: How many new major passenger planes a year that are trying to imitate the same type rating as an older aircraft do they have to test a year? In that specific case probably the important parts to review are the bits that try to make the new airframe handle the same as the old one.
- kqr 5y ago
  Let's say it is 0.1 on average, and the FAA would need 5 years just to test. That gives you an expected wait time of 10 years to get something tested, and 20 years if you include development time. I'm sure the industry benefits from lower turnaround times than that.
  - antihero 5y ago
    I would imagine they could test multiple planes at the same time.
    - kqr 5y ago
      Sure. So you take half the staff and put them on testing a second plane. Then each individual plane takes 10 years to test instead. (Remember that you don't gain person-hours for free just by shuffling people around.) What would this accomplish? Instead of a total system time average of 10 years, you now get an average testing time of 10 years to which you still need to add the queuing time, so you're even worse off than before. (The queueing time won't be five years with two servers in parallel, and I can't do the exact approximations in my head, but it'll be at least two years. In other words, by testing in parallel you worsen the cycle time from 20 years to at least 22 years.) This is a good general rule: by taking on more work in parallel, you'll make the turnaround time worse. This is why lean consultants go on about limiting work-in-progress. Also a call to learn some basic queuing theory! It comes in handy often.
lazide 5y ago
That sounds like they are knowingly incompetent? If a utility regulator had no one on staff who knew how utility scale power transmission/generation/whatever worked, that’s what we’d call it for sure. And aerospace engineers are dirt cheap. What sort of clown show is the FAA running?
- tw04 5y ago
  > That sounds like they are knowingly incompetent? That sounds like they are knowingly underfunded, which they are. > And aerospace engineers are dirt cheap. They are? Where are all the airline startups? I assume they measure in the hundreds with all this dirt cheap talent running around.
  - tester34 5y ago
    maybe this industry isn't like software where all equipment you need for almost all jobs is 16gb ram, nvme m2 disk, decent cpu laptop
  - lazide 5y ago
    Aerospace engineers have as much to do with airlines as software engineers have to do with data entry companies. Which is to say pretty much nothing. Aerospace requires lots of capital expense and is very ‘large customer’ driven and aerospace engineers play second (or third) fiddle to that, unlike in software.
- Waterluvian 5y ago
  I think you’re deeply misunderstanding the purpose of the FAA and the magnitude of the problem here. Like almost every other agency (the FDA didn’t do the vaccine trials; are they incompetent too?) they don’t do the tests. They just police the industry. Theoretically if you set laws and regulations and dole out severe punishment for bad behaviour, you don’t need to be the one running the tests. People should want their government to run on trust (if their culture is compatible with trust) because it’s far cheaper and more efficient.
  - 5y ago
dataflow 5y ago
It wouldn't require several times more manpower for the FAA to figure out there's something called MCAS that didn't exist before, would it? (Note that I'm talking in the current world we live in, not in a hypothetical world where Boeing would be a mortal enemy of the FAA doing everything within the stretch of human imagination to hide MCAS.)
- r3trohack3r 5y ago
  Out of curiosity, how confident are you that there aren’t any other novel systems that didn’t exist before on a similar level as MCAS? Having not had a public debacle around them, how much effort would it take for you to personally certify that MCAS is the only novel system of its caliber on these aircraft?
  - dataflow 5y ago
    > Out of curiosity, how confident are you that there aren’t any other novel systems that didn’t exist before on a similar level as MCAS? Me? What makes you think I would have information on this? If I had to hazard a random guess, I would think that, if you're talking about the MAX 8, there have been enough leaks and testimonies and whistleblowers that any comparable system would have probably been mentioned somewhere. I have no idea either way, I haven't read everything that's gotten out. But I don't see why a regulator couldn't use various means to figure stuff like this out with reasonably high confidence.
  - deleted 5y ago
    [deleted]
  - ethbr0 5y ago
    The airplane couldn't have been built without many system diagrams showing MCAS, or a similarly complex component. The FAA should have pulled schematics directly from Boeing engineering during certification. And the FAA should have someone with enough technical expertise and experience look at them. And that person should have said "The submitted information by Boeing doesn't include full details on this subsystem." Whether or not the Boeing test pilot highlighted the system for the FAA is a red herring. It's the FAA's job to find this, regardless of whether someone points them at it. If the FAA doesn't have the technical staffing or expertise to do this, then that's the problem. Charging the test pilot is necessary, but not sufficient. If the FAA infrequently performs this work (certification of a new aircraft), then flex in retired expertise! You can't tell me there aren't qualified, retired candidates (ex-industry or ex-FAA) who would have signed up for a year or two review. And all the better that they don't have career incentives! Essentially, this is the FAA charging Boeing for not doing the FAA's job correctly.
smnrchrds 5y ago
I don't think this style of regulation is limited to FAA. FDA does not independently test all drugs either. They just review and ensure the drug companies' tests are acceptable.
- mvc 5y ago
  I'm afraid this is the logical consequence of a majority of people falling for the lie that "government can't do things as efficiently as the private sector". When people vote for politicians who say "government is wasteful", "there's too much red tape" what did they think was going to happen?
  - pgeorgi 5y ago
    When politicians say "government is wasteful" or "government can't do things as efficiently as the private sector", what they're saying is that their management style is wasteful. Would somebody asserting that about _their_ _own_ _job_ even make it past the interview in the private sector?
    - TheOtherHobbes 5y ago
      What they're actually saying is "If you sponsor us we will make sure government money goes your way." Efficiency - or the lack of it - is absolutely not the point.
    - dash2 5y ago
      Or they could be saying that centralized monopolies are rarely efficient. That is certainly my experience. For example, SpaceX undercut NASA by a factor of about 10, according to this FT report: https://www.ft.com/content/25e2292b-a910-41c8-9c55-09096895f673 https://www.ft.com/content/25e2292b-a910-41c8-9c55-09096895f...
      - galangalalgol 5y ago
        plane-pal aircraft coming soon to a gate near you!
roofwellhams 5y ago
How can you be a regulator when you have no clue what are you doing?
ErikVandeWater 5y ago
> would require several times more manpower Or they could be several times more efficient. $17.5 billion/yr can do a lot.
pithon 5y ago
I've heard people break the design/implement/test phases down to essentially equal parts in terms of cost. It's not a small task if you think about a government agency's testing capabilities potentially needing to be roughly 1/3 of what the private industry is throwing at it. I can see where they need to be pragmatic and adopt a stance where they're really just overseeing the vendor's testing and doing some spot checks. Not ideal, though.
rdiddly 5y ago
None of those are unfixable problems at the FAA though. It all just boils down to a hiring problem. (If there were a will to actually have a functioning government.)
KennyBlanken 5y ago
Your comment boils down to a false dilemma fallacy between "FAA blindly trusts manufacturers" and "FAA conducts their own testing and validation." Clearly they could have been more actively reviewing stuff coming from Boeing, or engaging in some level of auditing. If the FAA had paid attention they would have seen a company desperate to compete modifying numerous basic characteristics of an airplane to the point of making it aerodynamically unstable, using a flight control system as a bandaid to fix this. Not to mention being such massively cheap assholes that they literally didn't install warning lights in the cockpit to tell the pilots when the sensor their flight computer would use to override control inputs, had failed.
- asah 5y ago
  Actually, this is a sauce effect of the complexity of modern systems: regulators have no choice but to trust companies and regulate the results of tests, with heavy penalties for cheating. Consider pharmaceutical trials: it's impractical for the FDA to check on each patient. Financial audits often assume the client isn't directly lying to the auditor, with criminal penalties as the disincentive. The alternative requires raising costs and slowing progress and innovation, which is a nonstart in a competitive environment such as Boeing vs Airbus.
  - OneTimePetes 5y ago
    It can be done, it just costs money. It can not be done within the liberal frame of mind, were everything that costs money is tax and thus theft. So why not reformulate it correctly. In my ideology, it is not possible to solve this problem. If that ideology would be out of the way, it could actually be tested pretty good via unittest running a simulation. So once developed those tests would actually be pretty cheap to run. Its just this ideologic blindspot that prevents good safety.
    - kiklion 5y ago
      > It can be done, it just costs money. Well money and time. You speak as if you think that if only the great billionaires accepted a little less money then we could have safety. How many lives are you willing to lose for a multi year delay to accommodate the FDA recreating every required test that the pharmaceutical company did? How many lives are currently lost because the current system doesn’t work? Your suggestion is akin to voter id’s as a requirement to prevent voter fraud. How many legitimate voters are going to be prevented from voting due to the new rules to stop how many prior confirmed cases of voter fraud? This isn’t a lack of resources. This is society deciding that the resources are better used elsewhere.
HPsquared 5y ago
A pair of eyeballs, a brain and a set of teeth are all a regulator needs, in other words.
2OEH8eoCRo0 5y ago
What manpower would be needed? How often are new aircraft validated? That's pretty scary that at the end of the day it's just a corporate shill pilot validating. What is the point of all the regulations if that's all that it takes?
- FlyMoreRockets 5y ago
  Every single component in every single aircraft certificated in the USA is tracked all the way to the mine. A quick web search reveals around 11.3 million people work in the aviation industry directly. A significant portion of this would have to be duplicated if the FAA were to verify everything.
axus 5y ago
Perhaps the FAA could get Airbus to contribute a review of the test plan when a new plane is close to release. Then you'd certainly see a more lively technical debate.
pdonis 5y ago
> What you talk about the FAA doing thier own testing and validation is (unfortunately) almost impossible. No, it isn't. It's just more work than the government feels like doing, involving more technical skill than the government feels like hiring. The problem is that the government can't have it both ways. It can't both claim that it is regulating airlines and airplane manufacturers to protect public safety, and also claim that it can't independently check what the regulated entities are telling it. It has to be one or the other: either we get the actual independent regulation that the government claims to be doing, with whatever resources it takes, or we all admit that we are not going to get that because the government is incapable of doing it, and we figure out some other way of ensuring safety.

yason 5y ago

FAA could facilitate smoketesting and focus on parts they deem the most suspicious/experimental/new in a new airplane. And then outsource the actual tests to a third party.

hef19898 5y ago
By law, the certifiction activities done by the EASA (I assume the FAA is the same) cannot be done by third parties. It also worth noting, that the aircraft certification procedures worked pretty well for decades, up until Boeing started to lie to the FAA. Not sure how Boeing kept its design org approval after that. EDIT: Some required activities by the organizations to be certified can be outsourced to third parties, the org itself is still accountable.
- VBprogrammer 5y ago
  > Not sure how Boeing kept its design org approval after that That's an easy one. It's too big to fail. One way or another Boeing had to make it out the other end of the 737 Max disaster intact as an organisation. Anything else would have been unpalatable from the point of view of the American military industrial complex. I know that phrase is usually applied in a derogatory way but here I don't even disagree with the thinking.
  - hef19898 5y ago
    That seems to be the answer. And I kind of get the reasoning. Under EASE rules, there are accountable people at the head of approved design and production orgs. I hope those people at least got their licenses revoked, if something similar exists under FAA rules.
FridayoLeary 5y ago
What could possibly go wrong?

picsao 5y ago

They could sample statements sporadically and thus have some checks instead of none?

f6v 5y ago

Like, how many new major passenger planes do they have to test every year? 0.1 on average?

TheSpiceIsLife 5y ago
Or, more precisely: How many new major passenger planes a year that are trying to imitate the same type rating as an older aircraft do they have to test a year? In that specific case probably the important parts to review are the bits that try to make the new airframe handle the same as the old one.
kqr 5y ago
Let's say it is 0.1 on average, and the FAA would need 5 years just to test. That gives you an expected wait time of 10 years to get something tested, and 20 years if you include development time. I'm sure the industry benefits from lower turnaround times than that.
- antihero 5y ago
  I would imagine they could test multiple planes at the same time.
  - kqr 5y ago
    Sure. So you take half the staff and put them on testing a second plane. Then each individual plane takes 10 years to test instead. (Remember that you don't gain person-hours for free just by shuffling people around.) What would this accomplish? Instead of a total system time average of 10 years, you now get an average testing time of 10 years to which you still need to add the queuing time, so you're even worse off than before. (The queueing time won't be five years with two servers in parallel, and I can't do the exact approximations in my head, but it'll be at least two years. In other words, by testing in parallel you worsen the cycle time from 20 years to at least 22 years.) This is a good general rule: by taking on more work in parallel, you'll make the turnaround time worse. This is why lean consultants go on about limiting work-in-progress. Also a call to learn some basic queuing theory! It comes in handy often.

lazide 5y ago

That sounds like they are knowingly incompetent? If a utility regulator had no one on staff who knew how utility scale power transmission/generation/whatever worked, that’s what we’d call it for sure. And aerospace engineers are dirt cheap. What sort of clown show is the FAA running?

tw04 5y ago
> That sounds like they are knowingly incompetent? That sounds like they are knowingly underfunded, which they are. > And aerospace engineers are dirt cheap. They are? Where are all the airline startups? I assume they measure in the hundreds with all this dirt cheap talent running around.
- tester34 5y ago
  maybe this industry isn't like software where all equipment you need for almost all jobs is 16gb ram, nvme m2 disk, decent cpu laptop
- lazide 5y ago
  Aerospace engineers have as much to do with airlines as software engineers have to do with data entry companies. Which is to say pretty much nothing. Aerospace requires lots of capital expense and is very ‘large customer’ driven and aerospace engineers play second (or third) fiddle to that, unlike in software.
Waterluvian 5y ago
I think you’re deeply misunderstanding the purpose of the FAA and the magnitude of the problem here. Like almost every other agency (the FDA didn’t do the vaccine trials; are they incompetent too?) they don’t do the tests. They just police the industry. Theoretically if you set laws and regulations and dole out severe punishment for bad behaviour, you don’t need to be the one running the tests. People should want their government to run on trust (if their culture is compatible with trust) because it’s far cheaper and more efficient.
- 5y ago

dataflow 5y ago

It wouldn't require several times more manpower for the FAA to figure out there's something called MCAS that didn't exist before, would it? (Note that I'm talking in the current world we live in, not in a hypothetical world where Boeing would be a mortal enemy of the FAA doing everything within the stretch of human imagination to hide MCAS.)

r3trohack3r 5y ago
Out of curiosity, how confident are you that there aren’t any other novel systems that didn’t exist before on a similar level as MCAS? Having not had a public debacle around them, how much effort would it take for you to personally certify that MCAS is the only novel system of its caliber on these aircraft?
- dataflow 5y ago
  > Out of curiosity, how confident are you that there aren’t any other novel systems that didn’t exist before on a similar level as MCAS? Me? What makes you think I would have information on this? If I had to hazard a random guess, I would think that, if you're talking about the MAX 8, there have been enough leaks and testimonies and whistleblowers that any comparable system would have probably been mentioned somewhere. I have no idea either way, I haven't read everything that's gotten out. But I don't see why a regulator couldn't use various means to figure stuff like this out with reasonably high confidence.
- deleted 5y ago
  [deleted]
- ethbr0 5y ago
  The airplane couldn't have been built without many system diagrams showing MCAS, or a similarly complex component. The FAA should have pulled schematics directly from Boeing engineering during certification. And the FAA should have someone with enough technical expertise and experience look at them. And that person should have said "The submitted information by Boeing doesn't include full details on this subsystem." Whether or not the Boeing test pilot highlighted the system for the FAA is a red herring. It's the FAA's job to find this, regardless of whether someone points them at it. If the FAA doesn't have the technical staffing or expertise to do this, then that's the problem. Charging the test pilot is necessary, but not sufficient. If the FAA infrequently performs this work (certification of a new aircraft), then flex in retired expertise! You can't tell me there aren't qualified, retired candidates (ex-industry or ex-FAA) who would have signed up for a year or two review. And all the better that they don't have career incentives! Essentially, this is the FAA charging Boeing for not doing the FAA's job correctly.

smnrchrds 5y ago

I don't think this style of regulation is limited to FAA. FDA does not independently test all drugs either. They just review and ensure the drug companies' tests are acceptable.

mvc 5y ago
I'm afraid this is the logical consequence of a majority of people falling for the lie that "government can't do things as efficiently as the private sector". When people vote for politicians who say "government is wasteful", "there's too much red tape" what did they think was going to happen?
- pgeorgi 5y ago
  When politicians say "government is wasteful" or "government can't do things as efficiently as the private sector", what they're saying is that their management style is wasteful. Would somebody asserting that about _their_ _own_ _job_ even make it past the interview in the private sector?
  - TheOtherHobbes 5y ago
    What they're actually saying is "If you sponsor us we will make sure government money goes your way." Efficiency - or the lack of it - is absolutely not the point.
  - dash2 5y ago
    Or they could be saying that centralized monopolies are rarely efficient. That is certainly my experience. For example, SpaceX undercut NASA by a factor of about 10, according to this FT report: https://www.ft.com/content/25e2292b-a910-41c8-9c55-09096895f673 https://www.ft.com/content/25e2292b-a910-41c8-9c55-09096895f...
    - galangalalgol 5y ago
      plane-pal aircraft coming soon to a gate near you!

roofwellhams 5y ago

How can you be a regulator when you have no clue what are you doing?

ErikVandeWater 5y ago

> would require several times more manpower Or they could be several times more efficient. $17.5 billion/yr can do a lot.

pithon 5y ago

I've heard people break the design/implement/test phases down to essentially equal parts in terms of cost. It's not a small task if you think about a government agency's testing capabilities potentially needing to be roughly 1/3 of what the private industry is throwing at it. I can see where they need to be pragmatic and adopt a stance where they're really just overseeing the vendor's testing and doing some spot checks. Not ideal, though.

rdiddly 5y ago

None of those are unfixable problems at the FAA though. It all just boils down to a hiring problem. (If there were a will to actually have a functioning government.)

KennyBlanken 5y ago

Your comment boils down to a false dilemma fallacy between "FAA blindly trusts manufacturers" and "FAA conducts their own testing and validation." Clearly they could have been more actively reviewing stuff coming from Boeing, or engaging in some level of auditing. If the FAA had paid attention they would have seen a company desperate to compete modifying numerous basic characteristics of an airplane to the point of making it aerodynamically unstable, using a flight control system as a bandaid to fix this. Not to mention being such massively cheap assholes that they literally didn't install warning lights in the cockpit to tell the pilots when the sensor their flight computer would use to override control inputs, had failed.

asah 5y ago
Actually, this is a sauce effect of the complexity of modern systems: regulators have no choice but to trust companies and regulate the results of tests, with heavy penalties for cheating. Consider pharmaceutical trials: it's impractical for the FDA to check on each patient. Financial audits often assume the client isn't directly lying to the auditor, with criminal penalties as the disincentive. The alternative requires raising costs and slowing progress and innovation, which is a nonstart in a competitive environment such as Boeing vs Airbus.
- OneTimePetes 5y ago
  It can be done, it just costs money. It can not be done within the liberal frame of mind, were everything that costs money is tax and thus theft. So why not reformulate it correctly. In my ideology, it is not possible to solve this problem. If that ideology would be out of the way, it could actually be tested pretty good via unittest running a simulation. So once developed those tests would actually be pretty cheap to run. Its just this ideologic blindspot that prevents good safety.
  - kiklion 5y ago
    > It can be done, it just costs money. Well money and time. You speak as if you think that if only the great billionaires accepted a little less money then we could have safety. How many lives are you willing to lose for a multi year delay to accommodate the FDA recreating every required test that the pharmaceutical company did? How many lives are currently lost because the current system doesn’t work? Your suggestion is akin to voter id’s as a requirement to prevent voter fraud. How many legitimate voters are going to be prevented from voting due to the new rules to stop how many prior confirmed cases of voter fraud? This isn’t a lack of resources. This is society deciding that the resources are better used elsewhere.

HPsquared 5y ago

A pair of eyeballs, a brain and a set of teeth are all a regulator needs, in other words.

2OEH8eoCRo0 5y ago

What manpower would be needed? How often are new aircraft validated? That's pretty scary that at the end of the day it's just a corporate shill pilot validating. What is the point of all the regulations if that's all that it takes?

FlyMoreRockets 5y ago
Every single component in every single aircraft certificated in the USA is tracked all the way to the mine. A quick web search reveals around 11.3 million people work in the aviation industry directly. A significant portion of this would have to be duplicated if the FAA were to verify everything.

axus 5y ago

Perhaps the FAA could get Airbus to contribute a review of the test plan when a new plane is close to release. Then you'd certainly see a more lively technical debate.

pdonis 5y ago

> What you talk about the FAA doing thier own testing and validation is (unfortunately) almost impossible. No, it isn't. It's just more work than the government feels like doing, involving more technical skill than the government feels like hiring. The problem is that the government can't have it both ways. It can't both claim that it is regulating airlines and airplane manufacturers to protect public safety, and also claim that it can't independently check what the regulated entities are telling it. It has to be one or the other: either we get the actual independent regulation that the government claims to be doing, with whatever resources it takes, or we all admit that we are not going to get that because the government is incapable of doing it, and we figure out some other way of ensuring safety.