3 ms·

Since deep learning is dominating the news these days, this might be the opportunity to point to the potential new ways of dealing with ad blocking. One is usin

by pilooch 11y ago

Since deep learning is dominating the news these days, this might be the opportunity to point to the potential new ways of dealing with ad blocking. One is using character-based convolutional (or recurrent) networks that read the URL characters and classify it as legit or ad. The interesting gain here is that there's no more conflicting regex nor optimization to work on huge lists of blocking rules. Instead, URLs can be passed to the network in batches. I have conducted my own set of experiments and a usable deep model is available (see http://www.deepdetect.com/applications/text_model/ http://www.deepdetect.com/applications/text_model/ ). This is kinda shameful plug, but I'm really interested in feedback on these new ways of ad blocking with high accuracy. For sure running a deep net service is not as easy as installing ublock but there are ways. The whole source code is Open Source, as is the model. I have more data at hand, and some larger models could be built. Performance assessment would be a good next step as well. EDIT: the linked page is rather long, lookup 'Novel task' to get to a quick classification example

Pharaoh2 11y ago
Part of the requirement of adblock software is to inject itself into the request pipeline in realtime and reject ad requests while at the same time causing as small a stall as possible and also using the least amount of resources. NN's running inside a browser don't satisfy any of those criterias.
- pilooch 11y ago
  Not inside a browser of course. If you consider enterprise-wide applications, NN would be better and possibly faster (with GPU) than a bluecoat proxy.
  - Pharaoh2 11y ago
    I am still unconvinced if NN provide any advantage over regular expressions in this domain since it's not a hard problem to solve. Also, the request needs to be rejected before the connection is made so the only data to work with is the HTTP request headers. On an enterprise level, a better challenge to solve with NN would be to create a asic accelerated neural net hardware to filter packets/connections of IPS/IDS and firewall purpose with an extra ability of also possibly blocking advert.
barrkel 11y ago
There are already commercial products that use NN and other ML techniques to classify and block malware, click fraud etc. - Cisco has one. It's certainly a valid and viable approach, even in the face of determined opposition.
argonaut 11y ago
A URL character based classifier would be very very easy to game. Perhaps you meant an image-based classifier, though in this case Pharoah2's objection about speed is an insurmountable issue even on an enterprise server, since classifying a website on a CPU would take many seconds; on a GPU it would probably be several hundred milliseconds, which is a few hundred milliseconds too many!
- pilooch 11y ago
  Performance is in ms, see http://caffe.berkeleyvision.org/performance_hardware.html http://caffe.berkeleyvision.org/performance_hardware.html Lookup the testing numbers, 500 images / sec. Character-based models are way simpler, you can use test very easily using the original link to the model. They are not easily gamed either: they learn features as convolutional filters over the characters. These filters are more powerful than ngrams, as complex as the filters you may have seen for images. Now, about gaming the nets, there's an issue, which is the same as for image-based CNNs: some combinations of letters (or pixel) do exist that do not make a difference to the eye, but push the classifier into the wrong class. However, in order to find these combinations, having a first-hand on the underlying neural net and its weights is mandatory.
  - colejohnson66 11y ago
    Would crowd sourcing help? Like, what if when you block an element on a page, it asks if what you're blocking is an ad (or something else "no one" wants to see). If they click yes, it sends that URL or whatever to the neural net servers. If enough people block that element, it'll get blocked.
    - pilooch 11y ago
      Possibly. Having millions of URLs to populate each class is a good thing to start with. Gathered through other means, our current dataset has around 10M URLs in the 'ads' category. The model we made available to the public was built from 2M of these URLs. EDIT: of possible interest is that these models output a probability and possibly a confidence of having a URL blocked. Base on these, an blocker could ask for confirmation.
a_imho 11y ago
Implementation aside, I agree with your first sentence. It could also create a real incentive to try gaming NNs, that might help to further their capabilities.
newman314 11y ago
You should get in touch with gorhill to see if he would be interested integrating this into ublock...

by pilooch 11y ago

Pharaoh2 11y ago

Part of the requirement of adblock software is to inject itself into the request pipeline in realtime and reject ad requests while at the same time causing as small a stall as possible and also using the least amount of resources. NN's running inside a browser don't satisfy any of those criterias.

pilooch 11y ago
Not inside a browser of course. If you consider enterprise-wide applications, NN would be better and possibly faster (with GPU) than a bluecoat proxy.
- Pharaoh2 11y ago
  I am still unconvinced if NN provide any advantage over regular expressions in this domain since it's not a hard problem to solve. Also, the request needs to be rejected before the connection is made so the only data to work with is the HTTP request headers. On an enterprise level, a better challenge to solve with NN would be to create a asic accelerated neural net hardware to filter packets/connections of IPS/IDS and firewall purpose with an extra ability of also possibly blocking advert.

barrkel 11y ago

There are already commercial products that use NN and other ML techniques to classify and block malware, click fraud etc. - Cisco has one. It's certainly a valid and viable approach, even in the face of determined opposition.

argonaut 11y ago

A URL character based classifier would be very very easy to game. Perhaps you meant an image-based classifier, though in this case Pharoah2's objection about speed is an insurmountable issue even on an enterprise server, since classifying a website on a CPU would take many seconds; on a GPU it would probably be several hundred milliseconds, which is a few hundred milliseconds too many!

pilooch 11y ago
Performance is in ms, see http://caffe.berkeleyvision.org/performance_hardware.html http://caffe.berkeleyvision.org/performance_hardware.html Lookup the testing numbers, 500 images / sec. Character-based models are way simpler, you can use test very easily using the original link to the model. They are not easily gamed either: they learn features as convolutional filters over the characters. These filters are more powerful than ngrams, as complex as the filters you may have seen for images. Now, about gaming the nets, there's an issue, which is the same as for image-based CNNs: some combinations of letters (or pixel) do exist that do not make a difference to the eye, but push the classifier into the wrong class. However, in order to find these combinations, having a first-hand on the underlying neural net and its weights is mandatory.
- colejohnson66 11y ago
  Would crowd sourcing help? Like, what if when you block an element on a page, it asks if what you're blocking is an ad (or something else "no one" wants to see). If they click yes, it sends that URL or whatever to the neural net servers. If enough people block that element, it'll get blocked.
  - pilooch 11y ago
    Possibly. Having millions of URLs to populate each class is a good thing to start with. Gathered through other means, our current dataset has around 10M URLs in the 'ads' category. The model we made available to the public was built from 2M of these URLs. EDIT: of possible interest is that these models output a probability and possibly a confidence of having a URL blocked. Base on these, an blocker could ask for confirmation.

a_imho 11y ago

Implementation aside, I agree with your first sentence. It could also create a real incentive to try gaming NNs, that might help to further their capabilities.

newman314 11y ago

You should get in touch with gorhill to see if he would be interested integrating this into ublock...