5 ms·

I fear that MITMing ads is a dead end: 1. IIUC, when SNI is encrypted (in TLS 1.3?) almost everything is out the window. 2. Local devices can do DNS over HTTP

by teddyh 4y ago

I fear that MITMing ads is a dead end: 1. IIUC, when SNI is encrypted (in TLS 1.3?) almost everything is out the window. 2. Local devices can do DNS over HTTPS (DoH) and DNS over QUIC (DoQ) to look up their stuff, so DNS-based blocking will soon be obsolete. 3. The browser itself is controlled by the biggest ad-vendor around (Google), so you’ll probably get no help there. The only solutions are: A. Use browsers not controlled by Google (i.e. not any Chrome fork either). B. Use only apps and devices locally which do not display ads. (This is, in a way, a generalization of A.) C. Legislate away the business models of ads and the media and “smart” devices which use ads. (A very similar argument can be made for user tracking and telemetry.)

baxuz 4y ago
Does AdGuard also use this approach? https://kb.adguard.com/en/general/how-malware-protection-works https://kb.adguard.com/en/general/how-malware-protection-wor...
fomine3 4y ago
4. Deliver ads from same host as content (like Twitter, YouTube)
deleted 4y ago
[deleted]
ThePhysicist 4y ago
In my understanding ECH/ESNI shouldn't be an issue in this setup as long as the browser issues a domain-specific CONNECT request (i.e. "CONNECT google.com" instead of "CONNECT 24.154.13.11"). I think even with ECH enabled you should be able to impersonate the web server if you have a valid root CA certificate in the browsers' trust store. Remember, you're not performing "hostile" MITM-ing, but explicitly configure a proxy and root certificate in your browser. DNS shouldn't be an issue either as the browser leaves domain resolution to the proxy.
- teddyh 4y ago
  This is, of course, assuming that you can trust the browser to obey its proxy settings. (And proxy setting do not apply at all to local “smart” devices.)
mhils 4y ago
I wouldn't write it off - one possible trick here is to also MITM the DoH/DoQ server and disable ECH by removing the relevant records from the DNS response. We've just added DNS support to mitmproxy and this is a natural follow-up. :)
- pdimitar 4y ago
  Oh? Do you guys have a blog writeup? I would LOVE to read more about this! I want to eliminate the small amount of ads that make it through my PiHole.
  - mhils 4y ago
    Not yet, but you might be lucky soon. We have an RSS feed on mitmproxy.org and a Twitter account. :-)
XorNot 4y ago
Anyone who cares about ad blocking should not be using any Chromium based browser at this point, but isn't this the sort of tool you'd use at a network or virtual network level? There's no reason to let applications on your device bypass your own network settings - and this is something we probably need to start accommodating in Linux distros to start with (specifically: disabling all the weaponized E2E encryption that vendors are using, and forcibly MITM'ing it with keys under the users control). Network-namespaces should make this eminently possible - launch the user's entire environment into a network namespace which can only speak to "user rights" networking stack.
randomhodler84 4y ago
Run your own DoH filtering DNS server, I set this up a few months ago. DNS blocking is not obsoleted by transport encryption.
- teddyh 4y ago
  How do your force applications to use this server? I mean, even if you MITM the connection to the application’s preferred DoH server, the application probably checks the certificate of the DoH server and refuses to work at all if it can’t get a verified connection.
  - randomhodler84 4y ago
    You don’t mitm the DoH, you substitute it with your own server. I have yet to see DNS/DoH “pinning”, and apps (browsers) will let your override it. Embedding DNS entries in apps is a bad idea (as opposed to cert pinning, which is about fixed trust, and a good idea). Given that sometimes this is going to be blocked, even if they did it would fall to the host resolver.
- atommclain 4y ago
  Very curious about how you went about this as I would like to do the same.
  - randomhodler84 4y ago
    Many options, take a look at https://wiki.archlinux.org/title/DNS_over_HTTPS_servers https://wiki.archlinux.org/title/DNS_over_HTTPS_servers Update the DHCP on your router, all done.
- aesh2Xa1 4y ago
  OP is stating that "apps and devices" may circumvent DNS blocking by resorting to DoH. You can run your own DoH server, and you can even advertise it via your DHCP server, but clients ("apps and devices") do not need to accept the supplied servers for their own configuration.
trasz 4y ago
D. Create a whole bunch of VMs with browsers and “fake users” to DoS the whole ad-based business model.
- trasz 4y ago
  (Note: DoS not literally, of course, but by feeding it with so much fake metrics that the signal - metrics for actual, live users - gets lost in the noise.) Or perhaps forget about VMs, and have some kind of browser plugin that performs "fake browsing" - to throw off analytics - with your real cookies, but hidden from view, so it's not annoying to the live user browsing the web in the usual manner.
- teddyh 4y ago
  Some people might do this, but it will never be enough people to even register on the scales of the ad-funded businesses.

by teddyh 4y ago

baxuz 4y ago

Does AdGuard also use this approach? https://kb.adguard.com/en/general/how-malware-protection-works https://kb.adguard.com/en/general/how-malware-protection-wor...

fomine3 4y ago

4. Deliver ads from same host as content (like Twitter, YouTube)

deleted 4y ago

[deleted]

ThePhysicist 4y ago

In my understanding ECH/ESNI shouldn't be an issue in this setup as long as the browser issues a domain-specific CONNECT request (i.e. "CONNECT google.com" instead of "CONNECT 24.154.13.11"). I think even with ECH enabled you should be able to impersonate the web server if you have a valid root CA certificate in the browsers' trust store. Remember, you're not performing "hostile" MITM-ing, but explicitly configure a proxy and root certificate in your browser. DNS shouldn't be an issue either as the browser leaves domain resolution to the proxy.

teddyh 4y ago
This is, of course, assuming that you can trust the browser to obey its proxy settings. (And proxy setting do not apply at all to local “smart” devices.)

mhils 4y ago

I wouldn't write it off - one possible trick here is to also MITM the DoH/DoQ server and disable ECH by removing the relevant records from the DNS response. We've just added DNS support to mitmproxy and this is a natural follow-up. :)

pdimitar 4y ago
Oh? Do you guys have a blog writeup? I would LOVE to read more about this! I want to eliminate the small amount of ads that make it through my PiHole.
- mhils 4y ago
  Not yet, but you might be lucky soon. We have an RSS feed on mitmproxy.org and a Twitter account. :-)

XorNot 4y ago

Anyone who cares about ad blocking should not be using any Chromium based browser at this point, but isn't this the sort of tool you'd use at a network or virtual network level? There's no reason to let applications on your device bypass your own network settings - and this is something we probably need to start accommodating in Linux distros to start with (specifically: disabling all the weaponized E2E encryption that vendors are using, and forcibly MITM'ing it with keys under the users control). Network-namespaces should make this eminently possible - launch the user's entire environment into a network namespace which can only speak to "user rights" networking stack.

randomhodler84 4y ago

Run your own DoH filtering DNS server, I set this up a few months ago. DNS blocking is not obsoleted by transport encryption.

teddyh 4y ago
How do your force applications to use this server? I mean, even if you MITM the connection to the application’s preferred DoH server, the application probably checks the certificate of the DoH server and refuses to work at all if it can’t get a verified connection.
- randomhodler84 4y ago
  You don’t mitm the DoH, you substitute it with your own server. I have yet to see DNS/DoH “pinning”, and apps (browsers) will let your override it. Embedding DNS entries in apps is a bad idea (as opposed to cert pinning, which is about fixed trust, and a good idea). Given that sometimes this is going to be blocked, even if they did it would fall to the host resolver.
atommclain 4y ago
Very curious about how you went about this as I would like to do the same.
- randomhodler84 4y ago
  Many options, take a look at https://wiki.archlinux.org/title/DNS_over_HTTPS_servers https://wiki.archlinux.org/title/DNS_over_HTTPS_servers Update the DHCP on your router, all done.
aesh2Xa1 4y ago
OP is stating that "apps and devices" may circumvent DNS blocking by resorting to DoH. You can run your own DoH server, and you can even advertise it via your DHCP server, but clients ("apps and devices") do not need to accept the supplied servers for their own configuration.

trasz 4y ago

D. Create a whole bunch of VMs with browsers and “fake users” to DoS the whole ad-based business model.

trasz 4y ago
(Note: DoS not literally, of course, but by feeding it with so much fake metrics that the signal - metrics for actual, live users - gets lost in the noise.) Or perhaps forget about VMs, and have some kind of browser plugin that performs "fake browsing" - to throw off analytics - with your real cookies, but hidden from view, so it's not annoying to the live user browsing the web in the usual manner.
teddyh 4y ago
Some people might do this, but it will never be enough people to even register on the scales of the ad-funded businesses.