5 ms·
> TLS worked before SNI was introduced, and the web worked before AWS and their ilk existed. There are still many, many, many websites that use TLS but no not r
by bawolff 4y ago
> TLS worked before SNI was introduced, and the web worked before AWS and their ilk existed. There are still many, many, many websites that use TLS but no not require SNI. If one only sees the web as a handful of CDNs, then it is easy to ignore these other TLS sites. However they exist, regardless of whether CDN devotees choose to acknowledge them.
CDNs are not the only use case for virtual hosts on the web.
Admittidly, subject-alt-name does help with that compared to the pre-sni web.
In any case, if SNI is not needed, than the ip address is neccesarily unique, in which case no privacy is lost by having the sni as nefarious people can just look at IP, so it doesnt really matter.
- pabs3 4y agoEven with CDNs and ESNI, discovering the site is possible using various techniques: https://blog.apnic.net/2019/08/23/what-can-you-learn-from-an-ip-address/ https://blog.apnic.net/2019/08/23/what-can-you-learn-from-an... https://news.ycombinator.com/item?id=28103770 https://news.ycombinator.com/item?id=28103770
- 1vuio0pswjnm7 4y ago"We also accounted for several resources that get loaded from different web servers due to the sub-queries performed when a website is requested. The set of all these IPs contacted is referred to as the Page Load Fingerprint (PLF) of the website." This method is restricted to clients that automatically load resources and/or run Javascript, e.g., so-called "modern" browsers. It will not work with non-browser clients. For example, I make HTTP requests using TCP clients and HTTP generators. There is no automatic loading of resources nor Javascript support. Another example is CCBot, the Common Crawl's Nutch crawler. It does not request CSS, images, Javascript, etc. Some researchers claim that sites can be fingerprinted by HTTP response size. People claim this is mitigated by HTTP/2 multiplexing and pipelining.^1 Because I use TCP clients not a browser to make HTTP requests, I can use HTTP/1.1 pipelining. Thus HTTP response size is not reliably predictive of any specific site. I can easily vary the size using pipelining. 1. https://arxiv.org/pdf/1707.00641 https://arxiv.org/pdf/1707.00641
- pabs3 4y agoI guess to mitigate this, you need something like a VPN, Apple Private Relay or the Tor network. Those don't solve the problem of a global passive adversary though.
- dfawcus 4y agoSo there is not a lot to be done about the ~50% of sites where the IP is sufficient to identify the domain. For the others, where they suggest a heuristic based on a PLF, it seems something like uMatrix (or a proxy based equivalent) could reduce, or change the fingerprint. Potentially at the cost of the page looking different (or ugly). I happen to use uMatrix simply to reduce b/w and make better use of my display (i.e. use the full width, not a silly narrow column), and most of the web is still usable. However such techniques would be beyond the likes of my parents and probably also my siblings.
- 1vuio0pswjnm7 4y agoNo one is using this theoretical method to support surveillance capitalism let alone censorship.^1 However many entities are using SNI for such purposes. I am not going to send SNI when it is not required and I am not going to send cleartext SNI if I can use ESNI. Why make surveillance and censorship easier. 1. "The real-world inference will be slightly different from our closed-world assumption because a wider dataset will be available to the adversary. It can happen that a PLF signature that might seem unique in our study can actually belong to two different websites; it's optimistic but we have identified IP addresses that have mappings to unique domains and these can potentially be used to uniquely profile websites." Sniffing SNI requires no such inferences nor assumptions. It works reliably in the real-world.
- pabs3 4y agoHow do you tell if a site requires SNI before connecting?
- 1vuio0pswjnm7 4y agoGenerally, you don't. With CDNs and other hosting providers, sometimes one can use heuristics. For example, all AWS sites require SNI and do not support TLS1.3. As such, before connecting, I can match the IP against a list of AWS IP ranges. If the site is using AWS then the page is retrieved from archive.org instead of Amazon. Internet Archive does not require SNI and supports TLS1.3. How do "tech" company-sponsored browsers tell if a site requires SNI before connecting. They don't. What so-called "modern" browsers do is they _assume_ all sites require SNI. That solution favours "tech" companies over user privacy. Same story with QUIC. I take a different approach. I assume no SNI is required and I train the proxy to recognise the sites I have accessed. I assume TLS TLS1.3 will be supported. No user-agent header. And so on. This is why I know there are many, many TLS sites that do not require SNI. As I learn about sites that require SNI or do not support TLS1.3, or have some other requirement, they are added to lists. These lists are "map" files that are loaded into the proxy's memory. Considering the two approaches, each involves a "guess" as to whether SNI is required before connecting. The first approach, i.e., assuming all sites require SNI, is, for me, wrong most of the time. I am not only accessing sites that require SNI. The second approach, i.e., "machine learning", uses stored information to educate the decision of whether to send SNI, use TLS1.3, etc. The second approach is, for me, right most of the time. And on a return visit to a site, the proxy is almost never wrong (unless perhaps the site has changed hosting providers); the proxy "knows" whether the site requires SNI.
- Lt_Riza_Hawkeye 4y ago> if SNI is not needed, than the ip address is neccesarily unique I think your are confused about what SNI does. SNI is not part of the request, and it does not indicate to the server which website is being requested (this is the HTTP Host header, still in use as a pseudo-header in HTTP/2 and HTTP/3). SNI is part of the response from the server back to the client. The idea is that if an attacker somehow poisoned your DNS to point the domain of one legitimate site (for example, paypal.com) to another (let's say pastebin.org) - your browser would notice the mismatch and refuse to load the page or send any potentially sensitive data to the server. Furthermore - there is no need for it to be sent in the clear. Cloudflare in particular has thrown their weight behind a proposal/protocol extension called ESNI (Encrypted SNI), in an effort that truly turned around my opinion of the company.
- pabs3 4y agoAccording to Wikipedia, SNI is sent by the client to the server: https://en.wikipedia.org/wiki/Server_Name_Indication https://en.wikipedia.org/wiki/Server_Name_Indication
- WatchDog 4y ago> SNI is not part of the request ... SNI is part of the response from the server back to the client. SNI is sent by the client, in the request, at the TLS layer. It's purpose is to let the server know which certificate it should use to respond to the request.
- Matthias247 4y agoNot in the request but as part of connection establishment (ClientHello). At that point in time the protocol (eg http/1 or /2 or something completely different) isn’t even determined - and so no [http] request exists yet.
- MattJ100 4y agoThis is largely incorrect. SNI is used to tell the server which domain it needs to send a certificate for (i.e. to securely prove it owns that domain). You are right that it is separate from the HTTP host header, but in practice they are always the same because (in lieu of other information) the client wants assurance that the server owns a valid certificate for the HTTP host it is requesting to. What makes SNI complicated to secure is that until the client has securely established the server's identity, it could be a MITM. But to prove the identity to the client, the server needs to know which proof the client needs. Alternative solutions like "send certificates for all domains" have other issues such as performance, scalability and trivially leaking to anyone the full list of domains served. ESNI is clever, but more complex to deploy. It's only in recent years that HTTPS has become widely used at all, and that's mainly thanks to deployment becoming largely trivial via ACME and free via Let's Encrypt.
- 1vuio0pswjnm7 4y agoHypothetical The IP address "185.199.108.153" is seen while sniffing the network. What is the domain name. This site does not require SNI. Visiting the address and/or examining the certificate will not reveal the domain name. Consider that sniffing the SNI from a TLS packet requires no guesswork and no additonal lookups. It is designed to be high throughput and used in making routing decisions. The question is not whether solving this hypothetical is possible. Of course the answer is, "Yes, it is". Rather, the question is whether solving it requires no more work than sniffing the SNI from a TLS packet. To use bawolff's words, "... people can just look at IP, so it does not really matter." Also, remember, the comment I made is about privacy from "tech" companies, resistance to surveillance capitalism by American companies, not censorship in China.
- zinekeller 4y agoSince that we're talking practicals, let's look at China's answer: just block the IP, we already knew what's behind it anyways. Besides, in an SNI-less world the censors would just visit the IP address - that's not significantly different on today's checking of SNI. Besides, some email filtering now also checks the IP address' associated domains, which not just includes the rDNS (that's easy) but also using a DNS datafeed that shows which domains point to that IP.
- bawolff 4y agoIn fairness - counterexample: at various times china has blocked zh.wikipedia.org, but doesnt want to block en.wikipedia.org as that makes them look draconian to visiting foreigners. Both sites are on the same IP with same wildcard cert. Their options are basically only SNI sniffing or DNS poisioning.
- zinekeller 4y ago> In fairness - counterexample: at various times china has blocked zh.wikipedia.org, but doesnt want to block en.wikipedia.org as that makes them look draconian to visiting foreigners. While I'll accept this as a hypothetical, I've just verified this one since I remember Wikimedia has a rather complicated structure (https://wikitech.wikimedia.org/wiki/Data_centers https://wikitech.wikimedia.org/wiki/Data_centers), and I can verify that in the US you'll see an Let's Encrypt-issued wildcard certificates, while the Singaporean datacenter sends Digicert certificates that is apparently separated (except for desktop and mobile one - so for example accessing en.wikipedia sends you en.wikipedia and en.m.wikipedia), and the European ones (if my understanting is the current setup) uses Globalsign-issued certs, so probably they're dealing with censorship in a methodical matter.