3 ms·
Even with CDNs and ESNI, discovering the site is possible using various techniques: https://blog.apnic.net/2019/08/23/what-can-you-learn-from-an-ip-address/ ht
by pabs3 4y ago
Even with CDNs and ESNI, discovering the site is possible using various techniques:
https://blog.apnic.net/2019/08/23/what-can-you-learn-from-an-ip-address/ https://blog.apnic.net/2019/08/23/what-can-you-learn-from-an... https://news.ycombinator.com/item?id=28103770 https://news.ycombinator.com/item?id=28103770
- 1vuio0pswjnm7 4y ago"We also accounted for several resources that get loaded from different web servers due to the sub-queries performed when a website is requested. The set of all these IPs contacted is referred to as the Page Load Fingerprint (PLF) of the website." This method is restricted to clients that automatically load resources and/or run Javascript, e.g., so-called "modern" browsers. It will not work with non-browser clients. For example, I make HTTP requests using TCP clients and HTTP generators. There is no automatic loading of resources nor Javascript support. Another example is CCBot, the Common Crawl's Nutch crawler. It does not request CSS, images, Javascript, etc. Some researchers claim that sites can be fingerprinted by HTTP response size. People claim this is mitigated by HTTP/2 multiplexing and pipelining.^1 Because I use TCP clients not a browser to make HTTP requests, I can use HTTP/1.1 pipelining. Thus HTTP response size is not reliably predictive of any specific site. I can easily vary the size using pipelining. 1. https://arxiv.org/pdf/1707.00641 https://arxiv.org/pdf/1707.00641
- pabs3 4y agoI guess to mitigate this, you need something like a VPN, Apple Private Relay or the Tor network. Those don't solve the problem of a global passive adversary though.
- dfawcus 4y agoSo there is not a lot to be done about the ~50% of sites where the IP is sufficient to identify the domain. For the others, where they suggest a heuristic based on a PLF, it seems something like uMatrix (or a proxy based equivalent) could reduce, or change the fingerprint. Potentially at the cost of the page looking different (or ugly). I happen to use uMatrix simply to reduce b/w and make better use of my display (i.e. use the full width, not a silly narrow column), and most of the web is still usable. However such techniques would be beyond the likes of my parents and probably also my siblings.
- 1vuio0pswjnm7 4y agoNo one is using this theoretical method to support surveillance capitalism let alone censorship.^1 However many entities are using SNI for such purposes. I am not going to send SNI when it is not required and I am not going to send cleartext SNI if I can use ESNI. Why make surveillance and censorship easier. 1. "The real-world inference will be slightly different from our closed-world assumption because a wider dataset will be available to the adversary. It can happen that a PLF signature that might seem unique in our study can actually belong to two different websites; it's optimistic but we have identified IP addresses that have mappings to unique domains and these can potentially be used to uniquely profile websites." Sniffing SNI requires no such inferences nor assumptions. It works reliably in the real-world.
- pabs3 4y agoHow do you tell if a site requires SNI before connecting?
- 1vuio0pswjnm7 4y agoGenerally, you don't. With CDNs and other hosting providers, sometimes one can use heuristics. For example, all AWS sites require SNI and do not support TLS1.3. As such, before connecting, I can match the IP against a list of AWS IP ranges. If the site is using AWS then the page is retrieved from archive.org instead of Amazon. Internet Archive does not require SNI and supports TLS1.3. How do "tech" company-sponsored browsers tell if a site requires SNI before connecting. They don't. What so-called "modern" browsers do is they _assume_ all sites require SNI. That solution favours "tech" companies over user privacy. Same story with QUIC. I take a different approach. I assume no SNI is required and I train the proxy to recognise the sites I have accessed. I assume TLS TLS1.3 will be supported. No user-agent header. And so on. This is why I know there are many, many TLS sites that do not require SNI. As I learn about sites that require SNI or do not support TLS1.3, or have some other requirement, they are added to lists. These lists are "map" files that are loaded into the proxy's memory. Considering the two approaches, each involves a "guess" as to whether SNI is required before connecting. The first approach, i.e., assuming all sites require SNI, is, for me, wrong most of the time. I am not only accessing sites that require SNI. The second approach, i.e., "machine learning", uses stored information to educate the decision of whether to send SNI, use TLS1.3, etc. The second approach is, for me, right most of the time. And on a return visit to a site, the proxy is almost never wrong (unless perhaps the site has changed hosting providers); the proxy "knows" whether the site requires SNI.