9 ms·

I used to work on an anti malware team. There were two controversial issues that related to that work that I want to bring up here. First, what is malware? I

by efitz 3y ago

I used to work on an anti malware team. There were two controversial issues that related to that work that I want to bring up here. First, what is malware? It’s actually very hard to define in such a way that makes everyone happy. The line gets really blurry on the border of nuisance vs malicious, and on security tools, which might do things in the name of privacy that malicious software might do to hide its tracks. The other issue is false positives. We tried really hard to avoid false positives; we ran tests with known good binaries to see if they were inappropriately detected, etc. We feared false positives more than false negatives. I am sure that the MS antimalware team is having a bad day and not acting in bad faith.

stronglikedan 3y ago
I don't believe they're acting in bad faith either, and certainly nothing is being removed it from the system as the clickbait title implies. The user has the option to stop if from being removed, and the team recommends "adding Tor to Microsoft’s protection software exclusion list and restoring “tor.exe” from quarantine if Defender affected Tor’s operation". Seems perfectly reasonable to me.
jj999 3y ago
Can there be legal liability with anti malware? By example, could flagging some software as malware expose you to being sued for libel?
- horsawlarway 3y ago
  This is getting down voted and I'm not sure why. Google has flagged some of my personal domains as "deceptive login" (why: I could not fucking tell you... they give no info, and my logs show absolutely nothing going on. My best guess is I used chrome auto fill in them while testing letsencrypt staging certs after switching cert providers) But... they are very clearly publishing info claiming my site is malicious, and it's not. It seems like a super clear case of libel, which I won't pursue because it's personal sites and I'm not losing revenue or anything, but it seems open and shut. It's not even exempt due to general section 230 clauses. It's not user content. It's just them.
  - yummypaint 3y ago
    If you're up for it, have a lawyer send them a nastygram/C&D and post on HN about what happens. Many people would probably be interested.
  - rainsford 3y ago
    I'm not a lawyer, but my understanding of libel is that at least in the US you need an element of intent to be guilty of libel. Simply being wrong about something isn't libel unless you can prove they know it was wrong or should have known it was wrong. Unless you can show Google knows your site isn't malicious and is labeling it that way anyways, it seems more likely it would be an open and shut case in their favor. I'm also fairly confident Google's lawyers have put a fair amount of thought into this and are already prepared to demonstrate how their labeling is done in good faith, etc. I didn't do any downvoting, but I suspect it could be a reaction to the fact that treating this as libel would make operating any security company or service effectively impossible by requiring zero false positives. That seems like an obviously unworkable standard even if getting incorrectly flagged as malicious admittedly does suck.
    - horsawlarway 3y ago
      I've submitted more than a dozen reports to their service. To their credit - Mozilla pulled it from their copy of the safebrowsing list and no longer reports it as malicious. Google does though. Google also has no contact in place to appeal, and I've provided written contact information if they have any actual evidence of abuse that I can prevent. They have had these reports for more than 10 months now, and I resubmit roughly once every week or so.
- efitz 3y ago
  Yes [1]. In this case, the issue was that the AM software itself contained security bugs, but depriving you of use of your computer would probably be actionable. IANAL and have no idea if you could win that. The software does have disclaimer of warranties and limits on liability to the purchase price as part of the terms of use [2] [1] https://www.prnewswire.com/news-releases/consumers-file-class-action-lawsuit-against-symantec-for-defective-antivirus-software-300746568.html https://www.prnewswire.com/news-releases/consumers-file-clas... [2] https://www.microsoft.com/en-us/legal/terms-of-use https://www.microsoft.com/en-us/legal/terms-of-use [note: I am not 100% certain that this is the TOU specifically applicable to MS AM but most TOU look similar]
- quags 3y ago
  This would be similar to spam and blocklists. I know people have tried to sue blocklists (maps and spamhaus) but from memory they were not successful in the end. That included blocking email , sometimes with larger blocks with ip ranges not directly involved but in a similar range. Malware being flagged on an anti virus would be similar but in those case the user has an option to exclude the block. Google and Firefox block domains for phishing regularly as well.
KennyBlanken 3y ago
"Staggering incompetence" is indistinguishable from "bad faith." Someone didn't do the most basic of QA: > Microsoft Defender is detecting the latest version of Tor Browser as malware because it is using a new heuristic detection method that is designed to identify Trojans that use Tor to hide their activity. However, the heuristic method is too broad and also flags the Tor Browser itself as malware. The lack of consideration for the impact is what makes it bad faith; they clearly didn't care to properly test it, or did and didn't care that they were also flagging the browser itself. I'm sure quite a few people thing the Tor browser doesn't have a legitimate use.
- jahsome 3y ago
  Do you personally write utterly perfect code with absolutely no defects or unexpected/unintended consequences on the first try, every single time?
- efitz 3y ago
  How are they supposed to discover every new binary for every application as soon as it is released, correctly classify them as known good, and add them to the QA testing, in a time frame that you would not classify as “incompetence”? If it’s so easy that only incompetence or malice could explain it, why don’t you go work for them and fix it? I’m sure that people who have spent decades in the field would appreciate your wisdom.
  - Karunamon 3y ago
    It is the Tor Browser. From the official project. It lives at https://www.torproject.org/download/ https://www.torproject.org/download/ - Literally the first thing you get directed to if you go to a search engine and enter something like "download tor". This fruit could only be lower-hanging if it shipped with Windows. If they can't be bothered to look for the single most obvious non-malicious use of the thing they are trying to detect, it says very concerning things. Things like "we beta test our file removal tool in production", except "production" is "millions of unwitting people's PCs" rather than some website.
GuB-42 3y ago
I find it especially funny when anti-malware tools flag things like 4k intros or even smaller demoscene production. The people doing that are shaving bytes, fitting music and 3D graphics in self-contained executables smaller than many HN posts. Having enough space for malware after that would be insane. The reason these are often flagged as malware is that they are typically using runtime compression and a variety of tricks to save space, these techniques are unusual in legitimate programs and common in malware. Malware scanners could add support for the likes of crinkler and kkrunchy, but these are probably too niche to care.
- Terr_ 3y ago
  While it is true those executables may be small files, that doesn't mean they can't be part of a larger malware suite, one where the vestibulk of innocent looking code is elsewhere, and the specially-packed executable contains the damning secret sauce.
  - GuB-42 3y ago
    They definitely could be part of malware, that's why they are flagged as such. I know the context, I know what's in the archive with it, and I am pretty sure the executable are safe, but the scanner doesn't know, and doesn't get to see the nice effects for conformation. Most people simply turn off the scanner when dealing with these files. Scanners could discriminate. These productions are often compressed with well known packers. It would be a "simple" matter of unpacking the file and scanning the result executable, which is often just a bunch of graphics and sound calls. But that's significant work for such a niche application. These tiny packers are also way more resource hungry than their size would suggest.
    - Terr_ 3y ago
      > These tiny packers are also way more resource hungry than their size would suggest. I wonder if that might also expose some attack-surface, like with "zip bombs".
- kadoban 3y ago
hparadiz 3y ago
Bad faith? Windows defender flags KMS Tools as a malware trojan.
userbinator 3y ago
It’s actually very hard to define in such a way that makes everyone happy. There's an easy definition: does it give value to the user, or act against the user's interests? The former isn't, the latter is. Of course, by that definition, Windows itself would be considered malware. I suspect the other comment here referencing libel is why "potentially unwanted" is another category that they often use.
- efitz 3y ago
  That's way too subjective. Here's Microsoft's definition: https://learn.microsoft.com/en-us/microsoft-365/security/intelligence/criteria?view=o365-worldwide https://learn.microsoft.com/en-us/microsoft-365/security/int...
- zulban 3y ago
  By your definition, any software with a bad bug is malware. Next time you propose a definition, try to break it as an adversary as a test.
  - andrepew 3y ago
    I think intent is probably implied. All software has bugs so with your assumption all software is malware.
- tmpX7dMeXU 3y ago
  This comment is very much an extension of the “I could build this in a weekend” mindset.
lost_tourist 3y ago
I certainly hope this is false positive rather than Microsoft considering it malware, since it obviously is not unless you live in some place like North Korean and have the NK secret police's perspective.
hulitu 3y ago
> I am sure that the MS antimalware team is having a bad day and not acting in bad faith. It is not the first time they do this.

by efitz 3y ago

stronglikedan 3y ago

I don't believe they're acting in bad faith either, and certainly nothing is being removed it from the system as the clickbait title implies. The user has the option to stop if from being removed, and the team recommends "adding Tor to Microsoft’s protection software exclusion list and restoring “tor.exe” from quarantine if Defender affected Tor’s operation". Seems perfectly reasonable to me.

jj999 3y ago

Can there be legal liability with anti malware? By example, could flagging some software as malware expose you to being sued for libel?

horsawlarway 3y ago
This is getting down voted and I'm not sure why. Google has flagged some of my personal domains as "deceptive login" (why: I could not fucking tell you... they give no info, and my logs show absolutely nothing going on. My best guess is I used chrome auto fill in them while testing letsencrypt staging certs after switching cert providers) But... they are very clearly publishing info claiming my site is malicious, and it's not. It seems like a super clear case of libel, which I won't pursue because it's personal sites and I'm not losing revenue or anything, but it seems open and shut. It's not even exempt due to general section 230 clauses. It's not user content. It's just them.
- yummypaint 3y ago
  If you're up for it, have a lawyer send them a nastygram/C&D and post on HN about what happens. Many people would probably be interested.
- rainsford 3y ago
  I'm not a lawyer, but my understanding of libel is that at least in the US you need an element of intent to be guilty of libel. Simply being wrong about something isn't libel unless you can prove they know it was wrong or should have known it was wrong. Unless you can show Google knows your site isn't malicious and is labeling it that way anyways, it seems more likely it would be an open and shut case in their favor. I'm also fairly confident Google's lawyers have put a fair amount of thought into this and are already prepared to demonstrate how their labeling is done in good faith, etc. I didn't do any downvoting, but I suspect it could be a reaction to the fact that treating this as libel would make operating any security company or service effectively impossible by requiring zero false positives. That seems like an obviously unworkable standard even if getting incorrectly flagged as malicious admittedly does suck.
  - horsawlarway 3y ago
    I've submitted more than a dozen reports to their service. To their credit - Mozilla pulled it from their copy of the safebrowsing list and no longer reports it as malicious. Google does though. Google also has no contact in place to appeal, and I've provided written contact information if they have any actual evidence of abuse that I can prevent. They have had these reports for more than 10 months now, and I resubmit roughly once every week or so.
efitz 3y ago
Yes [1]. In this case, the issue was that the AM software itself contained security bugs, but depriving you of use of your computer would probably be actionable. IANAL and have no idea if you could win that. The software does have disclaimer of warranties and limits on liability to the purchase price as part of the terms of use [2] [1] https://www.prnewswire.com/news-releases/consumers-file-class-action-lawsuit-against-symantec-for-defective-antivirus-software-300746568.html https://www.prnewswire.com/news-releases/consumers-file-clas... [2] https://www.microsoft.com/en-us/legal/terms-of-use https://www.microsoft.com/en-us/legal/terms-of-use [note: I am not 100% certain that this is the TOU specifically applicable to MS AM but most TOU look similar]
quags 3y ago
This would be similar to spam and blocklists. I know people have tried to sue blocklists (maps and spamhaus) but from memory they were not successful in the end. That included blocking email , sometimes with larger blocks with ip ranges not directly involved but in a similar range. Malware being flagged on an anti virus would be similar but in those case the user has an option to exclude the block. Google and Firefox block domains for phishing regularly as well.

KennyBlanken 3y ago

"Staggering incompetence" is indistinguishable from "bad faith." Someone didn't do the most basic of QA: > Microsoft Defender is detecting the latest version of Tor Browser as malware because it is using a new heuristic detection method that is designed to identify Trojans that use Tor to hide their activity. However, the heuristic method is too broad and also flags the Tor Browser itself as malware. The lack of consideration for the impact is what makes it bad faith; they clearly didn't care to properly test it, or did and didn't care that they were also flagging the browser itself. I'm sure quite a few people thing the Tor browser doesn't have a legitimate use.

jahsome 3y ago
Do you personally write utterly perfect code with absolutely no defects or unexpected/unintended consequences on the first try, every single time?
efitz 3y ago
How are they supposed to discover every new binary for every application as soon as it is released, correctly classify them as known good, and add them to the QA testing, in a time frame that you would not classify as “incompetence”? If it’s so easy that only incompetence or malice could explain it, why don’t you go work for them and fix it? I’m sure that people who have spent decades in the field would appreciate your wisdom.
- Karunamon 3y ago
  It is the Tor Browser. From the official project. It lives at https://www.torproject.org/download/ https://www.torproject.org/download/ - Literally the first thing you get directed to if you go to a search engine and enter something like "download tor". This fruit could only be lower-hanging if it shipped with Windows. If they can't be bothered to look for the single most obvious non-malicious use of the thing they are trying to detect, it says very concerning things. Things like "we beta test our file removal tool in production", except "production" is "millions of unwitting people's PCs" rather than some website.

GuB-42 3y ago

I find it especially funny when anti-malware tools flag things like 4k intros or even smaller demoscene production. The people doing that are shaving bytes, fitting music and 3D graphics in self-contained executables smaller than many HN posts. Having enough space for malware after that would be insane. The reason these are often flagged as malware is that they are typically using runtime compression and a variety of tricks to save space, these techniques are unusual in legitimate programs and common in malware. Malware scanners could add support for the likes of crinkler and kkrunchy, but these are probably too niche to care.

Terr_ 3y ago
While it is true those executables may be small files, that doesn't mean they can't be part of a larger malware suite, one where the vestibulk of innocent looking code is elsewhere, and the specially-packed executable contains the damning secret sauce.
- GuB-42 3y ago
  They definitely could be part of malware, that's why they are flagged as such. I know the context, I know what's in the archive with it, and I am pretty sure the executable are safe, but the scanner doesn't know, and doesn't get to see the nice effects for conformation. Most people simply turn off the scanner when dealing with these files. Scanners could discriminate. These productions are often compressed with well known packers. It would be a "simple" matter of unpacking the file and scanning the result executable, which is often just a bunch of graphics and sound calls. But that's significant work for such a niche application. These tiny packers are also way more resource hungry than their size would suggest.
  - Terr_ 3y ago
    > These tiny packers are also way more resource hungry than their size would suggest. I wonder if that might also expose some attack-surface, like with "zip bombs".
kadoban 3y ago

hparadiz 3y ago

Bad faith? Windows defender flags KMS Tools as a malware trojan.

userbinator 3y ago

It’s actually very hard to define in such a way that makes everyone happy. There's an easy definition: does it give value to the user, or act against the user's interests? The former isn't, the latter is. Of course, by that definition, Windows itself would be considered malware. I suspect the other comment here referencing libel is why "potentially unwanted" is another category that they often use.

efitz 3y ago
That's way too subjective. Here's Microsoft's definition: https://learn.microsoft.com/en-us/microsoft-365/security/intelligence/criteria?view=o365-worldwide https://learn.microsoft.com/en-us/microsoft-365/security/int...
zulban 3y ago
By your definition, any software with a bad bug is malware. Next time you propose a definition, try to break it as an adversary as a test.
- andrepew 3y ago
  I think intent is probably implied. All software has bugs so with your assumption all software is malware.
tmpX7dMeXU 3y ago
This comment is very much an extension of the “I could build this in a weekend” mindset.

lost_tourist 3y ago

I certainly hope this is false positive rather than Microsoft considering it malware, since it obviously is not unless you live in some place like North Korean and have the NK secret police's perspective.

hulitu 3y ago

> I am sure that the MS antimalware team is having a bad day and not acting in bad faith. It is not the first time they do this.