4 ms·
Ask HN: How to locate ads on websites programmatically?
Let's say if i want to screenshot the ad banners which appear on the website. How can i do this?
The challenge is the determine what is a banner programmatically.
What technology can be used for this?
- byoung2 8y agoWhat are you using to fetch the pages? Something like CasperJS/PhantomJS? If the banners are images or iframes from an external domain you should be able to traverse the DOM and locate them this way. You would have to have a list of ad providers domains to look for and you might miss some.
- InGodsName 8y agoUsing chrome headless
- detaro 8y agoModify an ad-blocker to instead of blocking highlight ads?
- InGodsName 8y agoNice idea! Is there any resource on how modern ad blockers detect and block ad banners?
- greenyoda 8y agoSome ad blockers, like uBlock Origin, are open source, so you can examine the code and see what they're doing: https://github.com/gorhill/uBlock https://github.com/gorhill/uBlock The basic algorithm is to match URLs of loadable resources (images, JavaScript files, iframes, etc.) against huge lists of ad/malware/tracking sites. If the URL matches, the resource isn't loaded. The lists that uBlock Origin uses are enumerated here: https://github.com/gorhill/uBlock/blob/master/README.md https://github.com/gorhill/uBlock/blob/master/README.md
- detaro 8y agoAs greenyoda suggested, I'd start with source and documentation of uBlock. From what I know, the major blockers share the format for their blocklists. There's two types of blocks: visual blocks, which contain a CSS selector to hide, and network blocks. Since the latter commonly block scripts that load ad elements, you'd probably need to load the page twice, to observe what elements are not created when the ad-scripts are blocked.