4 ms·

tl; dr -- "Pinning Last Night’s Disaster On Diversity Hiring Is Unsupportable"

suby 2y ago
I think it's unfortunate if this is your only take away. Two things can be true at once: A) Diversity hiring is unrelated to the crash B) There is a real scandal with hiring and the FAA What I have read so far about this situation is outrageous. My understanding is that in 2014, Obama appointed a head of the FAA who wanted to diversify the workforce. They introduced a biographical questionnaire which had rather arbitrary criteria for passing. It appears to have been explicitly designed to fail people that do not know the magical answers. The failure rate was 90%. This is not the only example, but perhaps the most egregious questions: you are asked what your best subject was in highschool. If you answered science, you get 15 points (a substantial amount). No points for any other subject. Next, you're asked your best subject in college. If you answer history, you get 15 points. No points for any other subject. Here is an alleged recreation of the test: https://kaisoapbox.com/projects/faa_biographical_assessment/ https://kaisoapbox.com/projects/faa_biographical_assessment/ There are supposedly voicemails in which they helped select candidates pass this biographical questionnaire by providing their preferred race/gendered candidates with the answers. I haven't been able to find a voice mail online, but Fox Business reported that these voicemails exist. Per a 2016 Yahoo Finance article, an internal FAA report falsely cleared the employee of wrongdoing. It's my understanding that there is currently a lawsuit making its way through the courts regarding this. It's also my understanding that some allege that the problems are not all resolved yet with hiring, despite the questionnaire being withdrawn in 2018. I'm not sure of the specifics of how there may still be problems with hiring and the FAA. https://x.com/tracewoodgrains/status/1752091831095939471 https://x.com/tracewoodgrains/status/1752091831095939471
- rayiner 2y ago
  What allegedly happened at the FAA wasn’t just “let’s try to have a more diverse workforce.” It was “let’s collide with racial advocacy groups to make a test that would allow us to deny white applicants because they won’t have the secret pass words.” Anybody who was involved in this obviously has severely compromised judgment. Who can trust anything else they did?
  - apical_dendrite 2y ago
    From what I can tell from reading the exhibits in the court case, that's not what happened. This is a case of overfitting, rather than a conspiracy to create a process with "secret pass words". They wanted to create a process that would not lead to disparate impacts, but would also give them competent controllers. So they took the biographical data that they had collected in the 80s and 90s and constructed a test such that the output of the test would give them the result that they wanted. This is clearly dumb, and they ended up with a model that was overfit to the old data and didn't generalize (or even make much logical sense). Likely there were a tiny number of black controllers when they gathered the biographical data, so those few data points, and their idiosyncrasies, had a large impact on the answer weights that they generated, making the model weird and pretty nonsensical. It's not a conspiracy, it's just bad methodology in service of a dubious goal.
    - xyzzyz 2y ago
      This is not just "bad methodology in service of dubious goal". It's blatantly illegal, and not just by weak "disparate impact" standard. According to disparate impact standard, procedures that result in disparate outcomes between protected classes are illegal, even if causing these outcomes was not the intent of the policy. Here, the disparate impact was not just unintended side effect, it was the deliberate goal. They deliberately constructed the test to discriminate based on race. This is even ignoring the alleged cases of insiders leaking answer key to some applicants.
- Eisenstein 2y ago
  Both of those links come from that same person, tracewoodgrains, who links the documents on their google drive. Looking through the documents, it appears there was a case that happened in 2014, which was investigated in 2016. They found an FAA employee in a New York state office held a teleconference about how to apply to the open positions. People asked him questions and he answered them. The investigate was closed noting the following: "OIG reviewed official government emails for the period January 2013 to July 2015 and did not identify any additional matters or information of specific interest to this investigation. AEO-500’s ROI Case Number AHQ20150170 did not develop information that demonstrated HR employees gave improper advantages in processing the applications of ATCS candidates affiliated with the NBCFAE. A review of the ROI did not identify any issues that warranted additional review in this investigation. The findings in this investigation did not warrant a referral to a federal prosecutor." What exactly makes 'diverse' candidates more likely to answer 'science' and then 'history' on those questions? Are you saying there was a conspiracy to create hard to answer questions on the test and then give out the answers to diversity candidates? Occam's razor says this was a bullshit 'behavioral' test that meant nothing but disqualified a lot of candidates, and someone who knew that gave others a way to bypass it to get their foot in the door in one instance.
- lakis 2y ago
  From the website "The FAA has discontinued use of this Behavioural Assessment since 2018." That was 7 years ago.
comeonbro 2y ago
I'm sure this is multifactorial and the other factors are probably as or more important, but especially if you are familiar with the very egregious specifics of the FAA diversity hiring scandal (https://www.tracingwoodgrains.com/p/the-faas-hiring-scandal-a-quick-overview https://www.tracingwoodgrains.com/p/the-faas-hiring-scandal-...), "is unsupportable" here is a strong claim that is playing the classic and well-worn role that "there's no evidence" often does in bad journalism: https://www.astralcodexten.com/p/the-phrase-no-evidence-is-a-red-flag https://www.astralcodexten.com/p/the-phrase-no-evidence-is-a...
- apical_dendrite 2y ago
  I dug into one of the claims in the blog, and as far as I can tell, he's completely misinterpreting the evidence. The blog says: > An FAA employee and then-president of the NBCFAE's Washington Suburban chapter, provided NBCFAE members with "buzz words" in January 2014 that would automatically push their resumes to the tops of HR files. It's true that this person said that in the email, but if you actually look at the list of buzzwords, it's clear that this person was bullshitting and inflating his own importance (or maybe just fundamentally misunderstanding something). The list is on the last page of this document: https://drive.google.com/drive/folders/17Vi9dDtZvbwHDafrygRGYcG888f-6PDs https://drive.google.com/drive/folders/17Vi9dDtZvbwHDafrygRG... It looks like it's a page that was photocopied from a book about how to write a resume. It's a list of dozens of incredibly generic verbs like "manage", "analyze", "administer", "make", "improve", "design", etc. pretty much any resume will have at least some of these verbs. There's just no way you could build a system that would use these verbs to secretly screen in resumes of people who are in the know because everyone uses these verbs. A far more plausible explanation is that this guy was trying to make himself sound more powerful than eh really was. The email this person sent is awful, but it seems to be one person's very misguided attempt to push an agenda, not some sort of secret plot by FAA.
  - rayiner 2y ago
    Look at the exam and scoring rubric: https://kaisoapbox.com/projects/faa_biographical_assessment/ https://kaisoapbox.com/projects/faa_biographical_assessment/ Look at questions 29 and 33. The first (about whether negative feedback bothers you) is a plausible question but the grading is completely nonsensical. The second question, about art/dance classes you took in college, is nonsensical both as to the question and the answers. These seem obviously designed to be gamed with secret information. This was used as a mandatory screen for several years. The FAA didn’t fix it, Congress found out and banned it. How many people at the FAA saw this and green lighted it?

suby 2y ago

I think it's unfortunate if this is your only take away. Two things can be true at once: A) Diversity hiring is unrelated to the crash B) There is a real scandal with hiring and the FAA What I have read so far about this situation is outrageous. My understanding is that in 2014, Obama appointed a head of the FAA who wanted to diversify the workforce. They introduced a biographical questionnaire which had rather arbitrary criteria for passing. It appears to have been explicitly designed to fail people that do not know the magical answers. The failure rate was 90%. This is not the only example, but perhaps the most egregious questions: you are asked what your best subject was in highschool. If you answered science, you get 15 points (a substantial amount). No points for any other subject. Next, you're asked your best subject in college. If you answer history, you get 15 points. No points for any other subject. Here is an alleged recreation of the test: https://kaisoapbox.com/projects/faa_biographical_assessment/ https://kaisoapbox.com/projects/faa_biographical_assessment/ There are supposedly voicemails in which they helped select candidates pass this biographical questionnaire by providing their preferred race/gendered candidates with the answers. I haven't been able to find a voice mail online, but Fox Business reported that these voicemails exist. Per a 2016 Yahoo Finance article, an internal FAA report falsely cleared the employee of wrongdoing. It's my understanding that there is currently a lawsuit making its way through the courts regarding this. It's also my understanding that some allege that the problems are not all resolved yet with hiring, despite the questionnaire being withdrawn in 2018. I'm not sure of the specifics of how there may still be problems with hiring and the FAA. https://x.com/tracewoodgrains/status/1752091831095939471 https://x.com/tracewoodgrains/status/1752091831095939471

rayiner 2y ago
What allegedly happened at the FAA wasn’t just “let’s try to have a more diverse workforce.” It was “let’s collide with racial advocacy groups to make a test that would allow us to deny white applicants because they won’t have the secret pass words.” Anybody who was involved in this obviously has severely compromised judgment. Who can trust anything else they did?
- apical_dendrite 2y ago
  From what I can tell from reading the exhibits in the court case, that's not what happened. This is a case of overfitting, rather than a conspiracy to create a process with "secret pass words". They wanted to create a process that would not lead to disparate impacts, but would also give them competent controllers. So they took the biographical data that they had collected in the 80s and 90s and constructed a test such that the output of the test would give them the result that they wanted. This is clearly dumb, and they ended up with a model that was overfit to the old data and didn't generalize (or even make much logical sense). Likely there were a tiny number of black controllers when they gathered the biographical data, so those few data points, and their idiosyncrasies, had a large impact on the answer weights that they generated, making the model weird and pretty nonsensical. It's not a conspiracy, it's just bad methodology in service of a dubious goal.
  - xyzzyz 2y ago
    This is not just "bad methodology in service of dubious goal". It's blatantly illegal, and not just by weak "disparate impact" standard. According to disparate impact standard, procedures that result in disparate outcomes between protected classes are illegal, even if causing these outcomes was not the intent of the policy. Here, the disparate impact was not just unintended side effect, it was the deliberate goal. They deliberately constructed the test to discriminate based on race. This is even ignoring the alleged cases of insiders leaking answer key to some applicants.
Eisenstein 2y ago
Both of those links come from that same person, tracewoodgrains, who links the documents on their google drive. Looking through the documents, it appears there was a case that happened in 2014, which was investigated in 2016. They found an FAA employee in a New York state office held a teleconference about how to apply to the open positions. People asked him questions and he answered them. The investigate was closed noting the following: "OIG reviewed official government emails for the period January 2013 to July 2015 and did not identify any additional matters or information of specific interest to this investigation. AEO-500’s ROI Case Number AHQ20150170 did not develop information that demonstrated HR employees gave improper advantages in processing the applications of ATCS candidates affiliated with the NBCFAE. A review of the ROI did not identify any issues that warranted additional review in this investigation. The findings in this investigation did not warrant a referral to a federal prosecutor." What exactly makes 'diverse' candidates more likely to answer 'science' and then 'history' on those questions? Are you saying there was a conspiracy to create hard to answer questions on the test and then give out the answers to diversity candidates? Occam's razor says this was a bullshit 'behavioral' test that meant nothing but disqualified a lot of candidates, and someone who knew that gave others a way to bypass it to get their foot in the door in one instance.
lakis 2y ago
From the website "The FAA has discontinued use of this Behavioural Assessment since 2018." That was 7 years ago.

comeonbro 2y ago

I'm sure this is multifactorial and the other factors are probably as or more important, but especially if you are familiar with the very egregious specifics of the FAA diversity hiring scandal (https://www.tracingwoodgrains.com/p/the-faas-hiring-scandal-a-quick-overview https://www.tracingwoodgrains.com/p/the-faas-hiring-scandal-...), "is unsupportable" here is a strong claim that is playing the classic and well-worn role that "there's no evidence" often does in bad journalism: https://www.astralcodexten.com/p/the-phrase-no-evidence-is-a-red-flag https://www.astralcodexten.com/p/the-phrase-no-evidence-is-a...

apical_dendrite 2y ago
I dug into one of the claims in the blog, and as far as I can tell, he's completely misinterpreting the evidence. The blog says: > An FAA employee and then-president of the NBCFAE's Washington Suburban chapter, provided NBCFAE members with "buzz words" in January 2014 that would automatically push their resumes to the tops of HR files. It's true that this person said that in the email, but if you actually look at the list of buzzwords, it's clear that this person was bullshitting and inflating his own importance (or maybe just fundamentally misunderstanding something). The list is on the last page of this document: https://drive.google.com/drive/folders/17Vi9dDtZvbwHDafrygRGYcG888f-6PDs https://drive.google.com/drive/folders/17Vi9dDtZvbwHDafrygRG... It looks like it's a page that was photocopied from a book about how to write a resume. It's a list of dozens of incredibly generic verbs like "manage", "analyze", "administer", "make", "improve", "design", etc. pretty much any resume will have at least some of these verbs. There's just no way you could build a system that would use these verbs to secretly screen in resumes of people who are in the know because everyone uses these verbs. A far more plausible explanation is that this guy was trying to make himself sound more powerful than eh really was. The email this person sent is awful, but it seems to be one person's very misguided attempt to push an agenda, not some sort of secret plot by FAA.
- rayiner 2y ago
  Look at the exam and scoring rubric: https://kaisoapbox.com/projects/faa_biographical_assessment/ https://kaisoapbox.com/projects/faa_biographical_assessment/ Look at questions 29 and 33. The first (about whether negative feedback bothers you) is a plausible question but the grading is completely nonsensical. The second question, about art/dance classes you took in college, is nonsensical both as to the question and the answers. These seem obviously designed to be gamed with secret information. This was used as a mandatory screen for several years. The FAA didn’t fix it, Congress found out and banned it. How many people at the FAA saw this and green lighted it?