The previous system at OpenDNS was using TRE agrep with a simple regex matching names commonly used for phishing. The output had a high amount of false positiv

The previous system at OpenDNS was using TRE agrep with a simple regex matching names commonly used for phishing. The output had a high amount of false positives (SEO, actual payment pages), but still spotted a decent amount of true positives. There was then a script trying to load the web page, and a simple classifier making a prediction based on the content. Classic phishing pages tend to add some obfuscation to avoid Google Safe Browsing and other phishing detection systems. As a result, web pages have little HTML tags; the DOM is built using obfuscated JS. Which is easy to reliably distinguish from minified scripts on a benign page based on the presence of actual content besides JS in the same document. Even if it only worked with some specific yet common phishing pages, the amount of false positive with this system was very low. The main problem is that many phishing pages are not hosted at the root path. Sometimes you get the phishing page from /, or a redirection from hxxp://phish/ to hxxp://phish/the/actual/path but most of the time, knowing the actual path is needed. And the DNS logs don't have this information. Compromised web sites are also commonly used for phishing and DNS doesn't help much either.