8 ms·

Is this an oversight of the AV software companies? Did no one come up with this before? Could it be that if people did come up with this before that a lot of Wi

Is this an oversight of the AV software companies? Did no one come up with this before? Could it be that if people did come up with this before that a lot of Windows computers have viruses without them knowing? Is their virus detection scheme fundamentally flawed? Should I be shocked? Shouldn't I be? I'm currently shocked but I don't know if it's justified, not an expert in the field.

devit 11y ago
No, this is not surprising at all. It's impossible to determine whether software is malicious or not (Rice's theorem). Antivirus software only reliably detects code that is identical to known malicious software.
- catnaroek 11y ago
  > It's impossible to determine whether software is malicious or not (Rice's theorem). Which is why proof-carrying code [https://en.wikipedia.org/wiki/Proof-carrying_code https://en.wikipedia.org/wiki/Proof-carrying_code] is a good idea: the onus ought to be on the programmer to provide (machine-checkable) evidence that their program is safe to use, for whatever notion of “safe” might make sense in your system.
  - nickpsecurity 11y ago
    That's still in its infancy. Future results might counter what we figured out. The certified compilers are nice for subversion-resistant development but not running untrustworthy code. There's significant difference between their models and the software/hardware combo actually running. Better route was started in Burrough's where you pick a language good at correct programs and carefully design a safe machine around it. Same with System/38, SAFE (crash-safe.org) for functional, and Cambrige's CHERI for C language. The fundamentals work as advertised along with ability to enforce arbitrary security policies. Then design and security stuff are built on that. Only thing known to work consistently to any degree of success. On COTS hardware, separation kernels and compiler transforms on legacy code are about best that we can do.
- V-2 11y ago
  That's not true - there's heurystic analysis techniques, generic signature detection etc. Of course they may not meet your definition of "reliably"
  - aleeds 11y ago
    I think that when it comes to this, each individual gets to define what reliably is, and nothing else matters.
  - ryao 11y ago
    > That's not true - there's heurystic analysis techniques, generic signature detection etc. Of course they may not meet your definition of "reliably" The only reliable things in security are the things that an attacker cannot bypass even when knowing that they in use (e.g. RSA). The premise of the article being discussed is that heuristics are trivial to bypass.
- eru 11y ago
  If you are happy to get lots of false alarms, it's doable with a whitelist approach.
- ryao 11y ago
  Would you say that rice's theorem is a generalization of the halting problem? It looks like "halting problem being unsolvable" -> "rice's theorem" by a subset relationship. Consequently, if rice's theorem were false, you could solve the halting problem by modus tollens. That being said, I had using the halting problem as my way of saying that identification of malicious software is impossible because infinite loops can be malicious and I had been unaware of rice's theorem. I will use that in my explanations in the future.
wolfgke 11y ago
> Is this an oversight of the AV software companies? No, this is a principle limitation of any AV software that is based on blacklisting. > Did no one come up with this before? Of course other people came up with similar ideas before. > Should I be shocked? If and only if you had trust in your AV software before.
- hayao 11y ago
  In what ways can I protect my PC and my web browser? I sometimes access sensible things and I want proper security.
  - dublinben 11y ago
    Don't run Windows, and don't run Javascript.
  - moonshinefe 11y ago
    Use an ad blocker (uBlock origin), keep your OS/Browser up to date. If you really want to get paranoid you can use NoScript or something like that (but you'll give up some convenience). The main thing is to make sure you trust the things you're clicking on. If you have to visit websites or try programs you don't trust, some people have virtual machines specifically for those situations. They'll visit the site/open the program inside the VM, and if something sketchy does happen, it'll be contained within the VM and not infect the host OS (unless it's incredibly sophisticated malware that can break out of VMs--but very unlikely you'd be targeted by something like that).
  - kbenson 11y ago
    Well, if you really want to sandbox some internet behavior (for example, porn, which I imagine has the highest percentage of sites delivering a malicious payload), use a virtual machine manager (VirtualBox) and set up a virtual machine (some Linux variant may serve best) for that specific type of access. You can do this multiple times, once for each type of access, such as a dedicated VM for accessing your bank website. If you're really paranoid, you can save the state of the virtual machine before use, and restore the prior state every time you use, it, preventing any changes to the VM. You would occasionally want to start it up, install all the recommended updates, and then save the state again though.
ryao 11y ago
Antivirus software is as useful for security as monster HDMI cables are useful for improving digital picture quality, so this is not surprising. The only thing that antivirus software does semi-decently is identify known software binaries. Antivirus software cannot reliably identify unknown binaries through heuristics because writing software to understand unknown software binaries is impossible in general. There are potentially an infinite number of ways of proving that, but the easiest way that occurs to me is that one of the many things necessary for understanding unknown software in general is solving the halting problem, which was proven to be impossible in general by Alan Turing. Furthermore, the utility for a database of known malicious binaries is practically non-existent. Malicious software is always designed to exploit some vulnerability and once the vulnerability is fixed by the vendor, there is nothing for the antivirus software to do. If you could apply the definition update that the antivirus software needed to catch malicious software, you could have applied the vendor patch that fixed the vulnerability the malicious software used in the first place. That not only makes the definition update unnecessary, but handles the unknown things that the definition update would never have caught. In the cases of a vendor being slow to patch, refusing to patch (e.g. the exploits used by the hot potato proof of concept code for all current Windows versions) or the user not applying the patch in time (e.g. lack of scheduled downtime), the inability of antivirus software to catch unknown software using those vulnerabilities provides a false sense of security. If a system is specifically targeted by a malicious hacker, the hacker would use something that antivirus software would not catch, such as a script kiddie tool against which there are no known definitions or custom code. Being unfortunate enough to be attacked by a virus, trojan, etcetera before they get definitions also means there is no protection. Real security requires doing things like minimizing attack area and configuring things competently (e.g. not using your username as your password). That is something that you cannot get from an antivirus vendor.

devit 11y ago

No, this is not surprising at all. It's impossible to determine whether software is malicious or not (Rice's theorem). Antivirus software only reliably detects code that is identical to known malicious software.

catnaroek 11y ago
> It's impossible to determine whether software is malicious or not (Rice's theorem). Which is why proof-carrying code [https://en.wikipedia.org/wiki/Proof-carrying_code https://en.wikipedia.org/wiki/Proof-carrying_code] is a good idea: the onus ought to be on the programmer to provide (machine-checkable) evidence that their program is safe to use, for whatever notion of “safe” might make sense in your system.
- nickpsecurity 11y ago
  That's still in its infancy. Future results might counter what we figured out. The certified compilers are nice for subversion-resistant development but not running untrustworthy code. There's significant difference between their models and the software/hardware combo actually running. Better route was started in Burrough's where you pick a language good at correct programs and carefully design a safe machine around it. Same with System/38, SAFE (crash-safe.org) for functional, and Cambrige's CHERI for C language. The fundamentals work as advertised along with ability to enforce arbitrary security policies. Then design and security stuff are built on that. Only thing known to work consistently to any degree of success. On COTS hardware, separation kernels and compiler transforms on legacy code are about best that we can do.
V-2 11y ago
That's not true - there's heurystic analysis techniques, generic signature detection etc. Of course they may not meet your definition of "reliably"
- aleeds 11y ago
  I think that when it comes to this, each individual gets to define what reliably is, and nothing else matters.
- ryao 11y ago
  > That's not true - there's heurystic analysis techniques, generic signature detection etc. Of course they may not meet your definition of "reliably" The only reliable things in security are the things that an attacker cannot bypass even when knowing that they in use (e.g. RSA). The premise of the article being discussed is that heuristics are trivial to bypass.
eru 11y ago
If you are happy to get lots of false alarms, it's doable with a whitelist approach.
ryao 11y ago
Would you say that rice's theorem is a generalization of the halting problem? It looks like "halting problem being unsolvable" -> "rice's theorem" by a subset relationship. Consequently, if rice's theorem were false, you could solve the halting problem by modus tollens. That being said, I had using the halting problem as my way of saying that identification of malicious software is impossible because infinite loops can be malicious and I had been unaware of rice's theorem. I will use that in my explanations in the future.

wolfgke 11y ago

> Is this an oversight of the AV software companies? No, this is a principle limitation of any AV software that is based on blacklisting. > Did no one come up with this before? Of course other people came up with similar ideas before. > Should I be shocked? If and only if you had trust in your AV software before.

hayao 11y ago
In what ways can I protect my PC and my web browser? I sometimes access sensible things and I want proper security.
- dublinben 11y ago
  Don't run Windows, and don't run Javascript.
- moonshinefe 11y ago
  Use an ad blocker (uBlock origin), keep your OS/Browser up to date. If you really want to get paranoid you can use NoScript or something like that (but you'll give up some convenience). The main thing is to make sure you trust the things you're clicking on. If you have to visit websites or try programs you don't trust, some people have virtual machines specifically for those situations. They'll visit the site/open the program inside the VM, and if something sketchy does happen, it'll be contained within the VM and not infect the host OS (unless it's incredibly sophisticated malware that can break out of VMs--but very unlikely you'd be targeted by something like that).
- kbenson 11y ago
  Well, if you really want to sandbox some internet behavior (for example, porn, which I imagine has the highest percentage of sites delivering a malicious payload), use a virtual machine manager (VirtualBox) and set up a virtual machine (some Linux variant may serve best) for that specific type of access. You can do this multiple times, once for each type of access, such as a dedicated VM for accessing your bank website. If you're really paranoid, you can save the state of the virtual machine before use, and restore the prior state every time you use, it, preventing any changes to the VM. You would occasionally want to start it up, install all the recommended updates, and then save the state again though.

ryao 11y ago

Antivirus software is as useful for security as monster HDMI cables are useful for improving digital picture quality, so this is not surprising. The only thing that antivirus software does semi-decently is identify known software binaries. Antivirus software cannot reliably identify unknown binaries through heuristics because writing software to understand unknown software binaries is impossible in general. There are potentially an infinite number of ways of proving that, but the easiest way that occurs to me is that one of the many things necessary for understanding unknown software in general is solving the halting problem, which was proven to be impossible in general by Alan Turing. Furthermore, the utility for a database of known malicious binaries is practically non-existent. Malicious software is always designed to exploit some vulnerability and once the vulnerability is fixed by the vendor, there is nothing for the antivirus software to do. If you could apply the definition update that the antivirus software needed to catch malicious software, you could have applied the vendor patch that fixed the vulnerability the malicious software used in the first place. That not only makes the definition update unnecessary, but handles the unknown things that the definition update would never have caught. In the cases of a vendor being slow to patch, refusing to patch (e.g. the exploits used by the hot potato proof of concept code for all current Windows versions) or the user not applying the patch in time (e.g. lack of scheduled downtime), the inability of antivirus software to catch unknown software using those vulnerabilities provides a false sense of security. If a system is specifically targeted by a malicious hacker, the hacker would use something that antivirus software would not catch, such as a script kiddie tool against which there are no known definitions or custom code. Being unfortunate enough to be attacked by a virus, trojan, etcetera before they get definitions also means there is no protection. Real security requires doing things like minimizing attack area and configuring things competently (e.g. not using your username as your password). That is something that you cannot get from an antivirus vendor.