4 ms·

Wait, is this filtering based on IP or DNS? How do they make sure their whitelist remains up to date? (I assume it's HTTPS, so those are basically the only two

Wait, is this filtering based on IP or DNS? How do they make sure their whitelist remains up to date? (I assume it's HTTPS, so those are basically the only two options...) If it's DNS based, there should be simpler workarounds, so I guess it's just IP based?

cortesoft 4y ago
It is pretty simple... your filter just makes periodic DNS requests to the desired allowed host and updates it's IP restrictions to the returned address. You also need to run the DNS resolver to return that same cached IP to prevent having the upstream DNS server return a different address. You also need to make sure the DNS server will only resolve the domains you want it to, because if you allow unfiltered dns requests to arbitrary domains, anyone can then tunnel their traffic over DNS, as another comment on this thread pointed out.
- stefan_ 4y ago
  I don't understand, what's the domain of.. WhatsApp? Not to mention you are just as likely to hit some Round-Robin configuration.
  - axiosgunnar 4y ago
    This is a very good question. I would also assume that Whatsapp might change the servers used with updates of the app. How would Delta deal with that? Just wait for the complaints to come in?
    - cranekam 4y ago
      WhatsApp (now, since it moved into FB's infra) connects to g.whatsapp.net, which is a CNAME to chat.cdn.whatsapp.net, which in turn is an A record to a VIP on Facebook's edge network. The A record you're returned can change — it's intended to be one that's closest to you (as determined by your DNS resolver's location and probably EDNS Client Subnet) but traffic engineering policies might cause different responses over time. Since on a flight it's likely everyone will use the same resolver on the ground somewhere in Delta's infrastructure a simple mechanism to resolve the IP periodically and update a whitelist (or to cache one VIP location and always return that) might work. Alternatively, as other commenters have suggested, it'd be better to identify traffic with SNI or other profiling. Updates to the app almost certainly won't change the address it connects to.
      - toast0 4y ago
        > Updates to the app almost certainly won't change the address it connects to. Well, they have before. There's four generations of hostnames that were used before g. But g seems likely to work for quite a while.
- peeters 4y ago
  These days with the prevalance of HTTPS, another option is to inspect the TLS client hello packet. To work well with load balancers etc, clients typically indicate the server name that they're trying to connect to (SNI - https://en.wikipedia.org/wiki/Server_Name_Indication https://en.wikipedia.org/wiki/Server_Name_Indication). That information is not encrypted. So you have both the dst IP and hostname in the initial packet.
  - Matthias247 4y ago
    Zero-rating at ISPs via SNI inspection is pretty common practice
    - dapids 4y ago
      It's also a preferred hardware inspection as its dead easy to rip the header than to seek mid-message to do message entropy/fingerprinting.
      - Matthias247 4y ago
        It's indeed pretty simple for TLS over TCP, since the whole ClientHello is part of the first packet and relatively easy parse or seek for. With QUIC it becomes a major pain, since it's not obvious anymore for middleboxes which QUIC packet is the first in a connection, and since Crypto data can be fragmented and reordered (Chrome is doing that by purpose even inside single packets). Therefore hardware inspection would require a pretty full-featured QUIC protocol parser and understanding.
        toast0 4y ago
        It's easier to just block QUIC. (And UDP in general, might as well)
        dapids 4y ago
        Interesting, thanks for the insight.
    - 4y ago
Denatonium 4y ago
It could also be TLS-SNI/HTTP-HOST based. While this doesn't directly address Delta's captive portal implementation, on many TP-Link Omada wireless APs, there is a feature that allows you to create a captive portal, and when doing this, you can either whitelist a website by its hostname or by its IP address. I was curious as to how it was filtering by hostname, so I ran a few DNS queries, which all resolved normally, indicating that it wasn't a DNS-based whitelist. Seeing as the whitelisting also worked over HTTPS, I assumed it was TLS-SNI. It turns out that anyone can whitelist any IP address by visiting any website while sending the SNI of a whitelisted hostname. This caused the AP's software to create a firewall rule allowing access to the IP address associated with the spoofed SNI. After doing this, it was then possible to connect to any website hosted on that IP address with any SNI hostname.
toast0 4y ago
I worked on the technical side of WhatsApp's special pricing program (aka zero rating) from when it started, through integration with the Facebook Mobile Partner Portal until I left in late 2019. We provided partners IP addresses (well a list of cidr ip/subnet lengths) and email updates when IPs changed. Some partners wanted hostnames, but it wasn't usually effective to manage that way; hostnames seemed to work great for WAP based special pricing, but not for direct tcp. But AFAIK, airline programs were done by the airlines (or whoever does their internet service) without consultation with WhatsApp. (A special plan for messaging without multimedia wasn't within the WA policy, at least while I was there, so we wouldn't have helped them build the product they wanted anyway) There are lots of possible ways to identify WhatsApp traffic, but I never had a chance to figure out what they were really doing. During that time period, if I was on a flight, I was usually with my young child and it's hard to keep focus for debugging networking on a plane, anyway. What I saw when I was looking was more like what joshvm describes elsewhere. The messaging only plans seem to allow most low bandwidth connections, with high latency, but they'll actively supress some things, and others stall beyond some threshold; sometimes you could get a couple media files to transfer, but then it would stop, etc. WhatsApp was engineered to work with the world's terrible networks, so it will usually work ok for messaging as long as packets get through eventually; connection and ping timeouts are long on client and server, because sometimes it takes a lot of seconds. If DNS doesn't work, that's fine too. Multimedia would usually retry and resume enough to work even if connections didn't last long, so I'd guess there was something actively supressing that, but I don't really know. FWIW, chat isn't TLS, so SNI isn't the answer there, although at least in the past, the protocol was very identifyable. Been gone for a while and don't regularly tcpdump my connections to WA anymore, so I don't know if that changed though. Multimedia is https, and probably has SNI, although that used to vary by platform.
- mgarciaisaia 4y ago
  I always thought WhatsApp published their list of IPs (segregated by standard chats vs media), but it seems they only share CIDR blocks with operators privately: https://www.whatsapp.com/cidr.txt https://www.whatsapp.com/cidr.txt And you say there's no segregation between chat and media - and I trust your out of date info more than my completely made up guesses.

cortesoft 4y ago

It is pretty simple... your filter just makes periodic DNS requests to the desired allowed host and updates it's IP restrictions to the returned address. You also need to run the DNS resolver to return that same cached IP to prevent having the upstream DNS server return a different address. You also need to make sure the DNS server will only resolve the domains you want it to, because if you allow unfiltered dns requests to arbitrary domains, anyone can then tunnel their traffic over DNS, as another comment on this thread pointed out.

stefan_ 4y ago
I don't understand, what's the domain of.. WhatsApp? Not to mention you are just as likely to hit some Round-Robin configuration.
- axiosgunnar 4y ago
  This is a very good question. I would also assume that Whatsapp might change the servers used with updates of the app. How would Delta deal with that? Just wait for the complaints to come in?
  - cranekam 4y ago
    WhatsApp (now, since it moved into FB's infra) connects to g.whatsapp.net, which is a CNAME to chat.cdn.whatsapp.net, which in turn is an A record to a VIP on Facebook's edge network. The A record you're returned can change — it's intended to be one that's closest to you (as determined by your DNS resolver's location and probably EDNS Client Subnet) but traffic engineering policies might cause different responses over time. Since on a flight it's likely everyone will use the same resolver on the ground somewhere in Delta's infrastructure a simple mechanism to resolve the IP periodically and update a whitelist (or to cache one VIP location and always return that) might work. Alternatively, as other commenters have suggested, it'd be better to identify traffic with SNI or other profiling. Updates to the app almost certainly won't change the address it connects to.
    - toast0 4y ago
      > Updates to the app almost certainly won't change the address it connects to. Well, they have before. There's four generations of hostnames that were used before g. But g seems likely to work for quite a while.
peeters 4y ago
These days with the prevalance of HTTPS, another option is to inspect the TLS client hello packet. To work well with load balancers etc, clients typically indicate the server name that they're trying to connect to (SNI - https://en.wikipedia.org/wiki/Server_Name_Indication https://en.wikipedia.org/wiki/Server_Name_Indication). That information is not encrypted. So you have both the dst IP and hostname in the initial packet.
- Matthias247 4y ago
  Zero-rating at ISPs via SNI inspection is pretty common practice
  - dapids 4y ago
    It's also a preferred hardware inspection as its dead easy to rip the header than to seek mid-message to do message entropy/fingerprinting.
    - Matthias247 4y ago
      It's indeed pretty simple for TLS over TCP, since the whole ClientHello is part of the first packet and relatively easy parse or seek for. With QUIC it becomes a major pain, since it's not obvious anymore for middleboxes which QUIC packet is the first in a connection, and since Crypto data can be fragmented and reordered (Chrome is doing that by purpose even inside single packets). Therefore hardware inspection would require a pretty full-featured QUIC protocol parser and understanding.
      - toast0 4y ago
        It's easier to just block QUIC. (And UDP in general, might as well)
      - dapids 4y ago
        Interesting, thanks for the insight.
  - 4y ago

Denatonium 4y ago

It could also be TLS-SNI/HTTP-HOST based. While this doesn't directly address Delta's captive portal implementation, on many TP-Link Omada wireless APs, there is a feature that allows you to create a captive portal, and when doing this, you can either whitelist a website by its hostname or by its IP address. I was curious as to how it was filtering by hostname, so I ran a few DNS queries, which all resolved normally, indicating that it wasn't a DNS-based whitelist. Seeing as the whitelisting also worked over HTTPS, I assumed it was TLS-SNI. It turns out that anyone can whitelist any IP address by visiting any website while sending the SNI of a whitelisted hostname. This caused the AP's software to create a firewall rule allowing access to the IP address associated with the spoofed SNI. After doing this, it was then possible to connect to any website hosted on that IP address with any SNI hostname.

toast0 4y ago

I worked on the technical side of WhatsApp's special pricing program (aka zero rating) from when it started, through integration with the Facebook Mobile Partner Portal until I left in late 2019. We provided partners IP addresses (well a list of cidr ip/subnet lengths) and email updates when IPs changed. Some partners wanted hostnames, but it wasn't usually effective to manage that way; hostnames seemed to work great for WAP based special pricing, but not for direct tcp. But AFAIK, airline programs were done by the airlines (or whoever does their internet service) without consultation with WhatsApp. (A special plan for messaging without multimedia wasn't within the WA policy, at least while I was there, so we wouldn't have helped them build the product they wanted anyway) There are lots of possible ways to identify WhatsApp traffic, but I never had a chance to figure out what they were really doing. During that time period, if I was on a flight, I was usually with my young child and it's hard to keep focus for debugging networking on a plane, anyway. What I saw when I was looking was more like what joshvm describes elsewhere. The messaging only plans seem to allow most low bandwidth connections, with high latency, but they'll actively supress some things, and others stall beyond some threshold; sometimes you could get a couple media files to transfer, but then it would stop, etc. WhatsApp was engineered to work with the world's terrible networks, so it will usually work ok for messaging as long as packets get through eventually; connection and ping timeouts are long on client and server, because sometimes it takes a lot of seconds. If DNS doesn't work, that's fine too. Multimedia would usually retry and resume enough to work even if connections didn't last long, so I'd guess there was something actively supressing that, but I don't really know. FWIW, chat isn't TLS, so SNI isn't the answer there, although at least in the past, the protocol was very identifyable. Been gone for a while and don't regularly tcpdump my connections to WA anymore, so I don't know if that changed though. Multimedia is https, and probably has SNI, although that used to vary by platform.

mgarciaisaia 4y ago
I always thought WhatsApp published their list of IPs (segregated by standard chats vs media), but it seems they only share CIDR blocks with operators privately: https://www.whatsapp.com/cidr.txt https://www.whatsapp.com/cidr.txt And you say there's no segregation between chat and media - and I trust your out of date info more than my completely made up guesses.