3 ms·

> And if that automated system can make a decision to flag the extension, it could also include in the email specifically what caused that flagging to happen U

by umvi 6y ago

> And if that automated system can make a decision to flag the extension, it could also include in the email specifically what caused that flagging to happen Unless it's a black box machine learning model that just says yes/no but can't tell you why

adrianN 6y ago
I think it's more likely that their flagging is heuristic based and could easily defeated by a malicious extension author and giving detailed feedback makes reverse engineering the rules trivial.
- fauigerzigerk 6y ago
  Law enforcement has a solution for that: Do tell people what the rules are and what they are doing wrong. Don‘t tell them how they got caught and what methods were used to catch them. Telling people that they are going to be severely punished within days by a very powerful global entity, without telling them which specific rules they are in breach of or what they have to change is a dystopian nightmare.
  - ISL 6y ago
    Law enforcement also has to tell people how they were caught, perhaps not the entire story, but at least most of it. That, too, is a good thing.
    - fauigerzigerk 6y ago
      Yes, to a degree. But for instance, tax authorities don't have to reveal the algorithm they use to determine who gets audited.
      - metaphor 6y ago
        Being audited isn't an explicit accusation that you've violated anything either; at best, it's an implicit suspicion to which the tax authority is fully entitled to probe and we as taxpayers are given reasonable opportunity to corroborate claims.
        fauigerzigerk 6y ago
        Absolutely. My point is simply that you can tell people what the rules are and which ones they may have broken without revealing absolutely everything about how you detect any malicious behaviour.
- srg0 6y ago
  > flagging is heuristic based and could easily defeated by a malicious extension author Well, if the reason is the "https://*/*" https://*/*" permission or something like that, and it is flagged and reported automatically, and everyone learns not to use it, it's a win-win. Benign extensions will have smaller targets on their backs, and malicious extensions will become less malicious OR will have to jump through hoops to achieve what they're trying to achieve (and will become easier to detect).
behnamoh 6y ago
> Unless it's a black box machine learning model that just says yes/no but can't tell you why Recently there's been some interest in Explainable Artificial Intelligence (XAI) which addresses the problem of "black box" machines. There are techniques such as SHAP and LIME that can help the ML designers provide explanations for why any specific observation has been "rejected". That said, I'm not sure if that's what Google uses here.
tripzilch 6y ago
This isn't something vague or hard like OCR, voice-recognition or image classification. It's a finite list of app permissions. Unless it's a random number generator that just says yes/no but can't tell you why -- is just as valid of a non-excuse for using the entirely wrong solution to a problem, and surprise the solution doesn't work very well.

by umvi 6y ago

adrianN 6y ago

I think it's more likely that their flagging is heuristic based and could easily defeated by a malicious extension author and giving detailed feedback makes reverse engineering the rules trivial.

fauigerzigerk 6y ago
Law enforcement has a solution for that: Do tell people what the rules are and what they are doing wrong. Don‘t tell them how they got caught and what methods were used to catch them. Telling people that they are going to be severely punished within days by a very powerful global entity, without telling them which specific rules they are in breach of or what they have to change is a dystopian nightmare.
- ISL 6y ago
  Law enforcement also has to tell people how they were caught, perhaps not the entire story, but at least most of it. That, too, is a good thing.
  - fauigerzigerk 6y ago
    Yes, to a degree. But for instance, tax authorities don't have to reveal the algorithm they use to determine who gets audited.
    - metaphor 6y ago
      Being audited isn't an explicit accusation that you've violated anything either; at best, it's an implicit suspicion to which the tax authority is fully entitled to probe and we as taxpayers are given reasonable opportunity to corroborate claims.
      - fauigerzigerk 6y ago
        Absolutely. My point is simply that you can tell people what the rules are and which ones they may have broken without revealing absolutely everything about how you detect any malicious behaviour.
srg0 6y ago
> flagging is heuristic based and could easily defeated by a malicious extension author Well, if the reason is the "https://*/*" https://*/*" permission or something like that, and it is flagged and reported automatically, and everyone learns not to use it, it's a win-win. Benign extensions will have smaller targets on their backs, and malicious extensions will become less malicious OR will have to jump through hoops to achieve what they're trying to achieve (and will become easier to detect).

behnamoh 6y ago

> Unless it's a black box machine learning model that just says yes/no but can't tell you why Recently there's been some interest in Explainable Artificial Intelligence (XAI) which addresses the problem of "black box" machines. There are techniques such as SHAP and LIME that can help the ML designers provide explanations for why any specific observation has been "rejected". That said, I'm not sure if that's what Google uses here.

tripzilch 6y ago

This isn't something vague or hard like OCR, voice-recognition or image classification. It's a finite list of app permissions. Unless it's a random number generator that just says yes/no but can't tell you why -- is just as valid of a non-excuse for using the entirely wrong solution to a problem, and surprise the solution doesn't work very well.