4 ms·

"Automated audit" sounds to me like an oxymoron.

tialaramex 5y ago
Audit isn't automated, it's often done by local "professional services" firms (often from the Big Four) on site and is the least useful tool we have but is better than nothing. However Mozilla's processing of audit documents created by that audit is (somewhat) automated. Here's an example task we can give to a machine: We want audit letters to be for the correct period of time, and mention all the correct CAs. So, look through the letter and find the dates, and the identifiers for the CAs, and check they match what's expected. If they don't flag for the CA to get us a suitable letter. This catches the case where the CA sent us the 2019 letter again or a letter doesn't mention all their intermediates. We don't need to care why that happened at this stage, we just want to ensure it can't happen. Humans are mostly not evil, but they are lazy, incompetent, and sometimes greedy. Automated checking covers those problems pretty well. Auditors are often reluctant to lie for you, but they're usually willing to omit facts you'd prefer weren't mentioned. Audit of AB1235 shows big problems? Just write that audit of AA1234 and AB1234 were fine and omit all mention of AB1235 from the letter. The automated check will pull that up, why isn't AB1235 mentioned?
- denton-scratch 5y ago
  Thank you for the explanation. So an automated audit isn't really an automated audit, it's a manual audit for which the audit report is checked automatically.
  - tialaramex 5y ago
    I think the problem is you've parsed the sentence wrong. The phrase Mozilla uses is "Automated Audit Letter Validation" and you've said to yourself, oh, there's an "Automated Audit" and this is validation of letters connected with that. Nope. There are "Audit Letters" - letters reporting the outcome of each audit - and they need to be Validated, now Mozilla has Automated part of that work. Judging from other reactions I think the vast majority of people parsed this correctly in context, though it's possible I'm wrong. Occasionally poor editing will result in text that most native speakers parse wrongly, these are named "Crash Blossoms" from the naming example headline, "Violinist linked to JAL crash blossoms". A violinist (metaphorically) blossomed, and the headline is explaining which one by telling us it's the one "linked to JAL crash". This bug (statements which have ambiguous parsing) is probably unavoidable in natural languages, although similar bugs in programming languages are arguably a design mistake.

Audit isn't automated, it's often done by local "professional services" firms (often from the Big Four) on site and is the least useful tool we have but is better than nothing. However Mozilla's processing of audit documents created by that audit is (somewhat) automated. Here's an example task we can give to a machine: We want audit letters to be for the correct period of time, and mention all the correct CAs. So, look through the letter and find the dates, and the identifiers for the CAs, and check they match what's expected. If they don't flag for the CA to get us a suitable letter. This catches the case where the CA sent us the 2019 letter again or a letter doesn't mention all their intermediates. We don't need to care why that happened at this stage, we just want to ensure it can't happen. Humans are mostly not evil, but they are lazy, incompetent, and sometimes greedy. Automated checking covers those problems pretty well. Auditors are often reluctant to lie for you, but they're usually willing to omit facts you'd prefer weren't mentioned. Audit of AB1235 shows big problems? Just write that audit of AA1234 and AB1234 were fine and omit all mention of AB1235 from the letter. The automated check will pull that up, why isn't AB1235 mentioned?

denton-scratch 5y ago
Thank you for the explanation. So an automated audit isn't really an automated audit, it's a manual audit for which the audit report is checked automatically.
- tialaramex 5y ago
  I think the problem is you've parsed the sentence wrong. The phrase Mozilla uses is "Automated Audit Letter Validation" and you've said to yourself, oh, there's an "Automated Audit" and this is validation of letters connected with that. Nope. There are "Audit Letters" - letters reporting the outcome of each audit - and they need to be Validated, now Mozilla has Automated part of that work. Judging from other reactions I think the vast majority of people parsed this correctly in context, though it's possible I'm wrong. Occasionally poor editing will result in text that most native speakers parse wrongly, these are named "Crash Blossoms" from the naming example headline, "Violinist linked to JAL crash blossoms". A violinist (metaphorically) blossomed, and the headline is explaining which one by telling us it's the one "linked to JAL crash". This bug (statements which have ambiguous parsing) is probably unavoidable in natural languages, although similar bugs in programming languages are arguably a design mistake.