4 ms·

> Security scanners should adopt LESS heuristics and focus more on VALIDATING their perceived findings Aren't you saying the same thing? In the context of the

> Security scanners should adopt LESS heuristics and focus more on VALIDATING their perceived findings Aren't you saying the same thing? In the context of the blog post I understood it to mean that an automated scanner, on retrieving content on a /.bash_history request, should look at the content and attempt to determine if it's full of bash commands and a correct finding (e.g. plaintext, full of new lines, lines start with common commands such as cd/ls, etc, ...), similar to how AV software makes (very) educated guesses on whether software is malicious or not. [1] Interested to hear your distinction between the definitions of heuristics and validations. 1 - https://en.wikipedia.org/wiki/Heuristic_(computer_science)#Antivirus_software https://en.wikipedia.org/wiki/Heuristic_(computer_science)#A...

naikrovek 3y ago
heuristics are imperfect; validation checks to make sure that you've found what you're looking for. the rub there, though, is that validation is extremely rigid and heuristics aren't. an attacker could (ideally) change a single bit in their signature to falsely pass validation but a heuristic will probably still catch it.
bostik 3y ago
> Interested to hear your distinction between the definitions of heuristics and validations. Okay, here it comes. Heuristics are essentially pattern-matching, because (as sibling points out) rigorous validation is extremely expensive. That's either in time, compute cycles, or sheer complexity. Really good - and smart - heuristics are fairly close to good validation, but in the security world, false positives and cutting corners are the norm. Vulnerability scanners of all sorts take massive short cuts. Everyone does banner grabbing, but very few try to validate that the announced version actually matches the known behaviour of that version. All kinds of scanners and end point agents collect the detected versions of software present on the system, but then don't check whether a supposedly detected vulnerability is actually present. An entire category of stupid agents broke when the point-release number for distro kernels went to four digits, and their boneheaded logic either treated the last figure as zero (.0) or truncated it to three digits. False positives everywhere, with "detected" kernel vulnerabilities all of a sudden sprouting across the fleets. Not a single one of these agents actually tried to validate the vulnerabilities they claimed to have spotted. (Banner grabbing works both ways: my IRC client claims to run on C-64.) Network vulnerability scanners are not any better. A particularly advanced scanner may look at the returned Content-Type header to spot crawled endpoints with interesting payload potential .. and then doesn't bother to use magic(5) to check whether the served content matches. Another network scanner detects that a host is running an LTS Debian derivative, but raises a high-severity finding because it spots network facing software using OpenSSL 1.1.1, which is getting EOL'd. The fact that the vendor provides continuing support is known and registered in various places across detection and alerting logic, but not used in this case. The heuristic used is "is this software EOL?", when the correct validation would be "is this software supported, up to date, and still receiving security fixes?". I could go on and on, but I think I've made my point. Even the best-of-breed[tm] heuristics as used by security software industry are a scourge and the root cause of frustration of everyone even remotely competent. But since you asked... malware is a special case. Heuristics in malware detection are a crutch, but they are nonetheless the least bad option. That's because with malware, you run into the halting problem. The only way to actually validate that a piece of suspected malware is indeed malware, is to execute it, and closely observe what it does. But there is no safe and easy way to do that. The universe of malware is highly adversarial, with counter-counter-counter-counter technologies present at every level by all sides. The best technologies use sandboxing to explode suspected payloads, but that only opens up a new vector - now the vendor has to worry about sandbox escapes too. And anti-malware agents themselves are running extremely complex operations at elevated permission levels, trying to hook or otherwise hijack software operations at the lowest possible OS levels, to prevent them from doing what the agent deems objectionable. We used to have a different name for this type of software back in the 90's. We called them rootkits. These days the industry calls them EDR agents. And I simply call them increased attack surface.
- adamckay 3y ago
  > I could go on and on, but I think I've made my point. Indeed. Thank you.

naikrovek 3y ago

heuristics are imperfect; validation checks to make sure that you've found what you're looking for. the rub there, though, is that validation is extremely rigid and heuristics aren't. an attacker could (ideally) change a single bit in their signature to falsely pass validation but a heuristic will probably still catch it.

bostik 3y ago

> Interested to hear your distinction between the definitions of heuristics and validations. Okay, here it comes. Heuristics are essentially pattern-matching, because (as sibling points out) rigorous validation is extremely expensive. That's either in time, compute cycles, or sheer complexity. Really good - and smart - heuristics are fairly close to good validation, but in the security world, false positives and cutting corners are the norm. Vulnerability scanners of all sorts take massive short cuts. Everyone does banner grabbing, but very few try to validate that the announced version actually matches the known behaviour of that version. All kinds of scanners and end point agents collect the detected versions of software present on the system, but then don't check whether a supposedly detected vulnerability is actually present. An entire category of stupid agents broke when the point-release number for distro kernels went to four digits, and their boneheaded logic either treated the last figure as zero (.0) or truncated it to three digits. False positives everywhere, with "detected" kernel vulnerabilities all of a sudden sprouting across the fleets. Not a single one of these agents actually tried to validate the vulnerabilities they claimed to have spotted. (Banner grabbing works both ways: my IRC client claims to run on C-64.) Network vulnerability scanners are not any better. A particularly advanced scanner may look at the returned Content-Type header to spot crawled endpoints with interesting payload potential .. and then doesn't bother to use magic(5) to check whether the served content matches. Another network scanner detects that a host is running an LTS Debian derivative, but raises a high-severity finding because it spots network facing software using OpenSSL 1.1.1, which is getting EOL'd. The fact that the vendor provides continuing support is known and registered in various places across detection and alerting logic, but not used in this case. The heuristic used is "is this software EOL?", when the correct validation would be "is this software supported, up to date, and still receiving security fixes?". I could go on and on, but I think I've made my point. Even the best-of-breed[tm] heuristics as used by security software industry are a scourge and the root cause of frustration of everyone even remotely competent. But since you asked... malware is a special case. Heuristics in malware detection are a crutch, but they are nonetheless the least bad option. That's because with malware, you run into the halting problem. The only way to actually validate that a piece of suspected malware is indeed malware, is to execute it, and closely observe what it does. But there is no safe and easy way to do that. The universe of malware is highly adversarial, with counter-counter-counter-counter technologies present at every level by all sides. The best technologies use sandboxing to explode suspected payloads, but that only opens up a new vector - now the vendor has to worry about sandbox escapes too. And anti-malware agents themselves are running extremely complex operations at elevated permission levels, trying to hook or otherwise hijack software operations at the lowest possible OS levels, to prevent them from doing what the agent deems objectionable. We used to have a different name for this type of software back in the 90's. We called them rootkits. These days the industry calls them EDR agents. And I simply call them increased attack surface.

adamckay 3y ago
> I could go on and on, but I think I've made my point. Indeed. Thank you.