4 ms·

I been seeing people on Twitter mocking the project, but I need it... You have no idea how much time I've wasted trying to block some products from pinging the

by nirui 3y ago

I been seeing people on Twitter mocking the project, but I need it... You have no idea how much time I've wasted trying to block some products from pinging their home server with curious data stream, but failed to do so because I can't be bothered to sit my ass in front of Wireshark to sniff out all their DoH servers. With this project, it's hopeful that in the future I can just not putting their domains in the TLS whitelist, even when they use DoH. --- Off topic but BTW: "Fully Encrypted Traffic" is really a confusing term that can only be understood correctly if the context is correct too. How about calling those protocols "High Entropy" (HighE)? Which IMO is more specific than calling it "Fully Encrypted". It's just my two-cents suggestion.

luke-stanley 3y ago
I briefly read a paper, I suspect it's more complex than "high entropy".
- nirui 3y ago
  The strategy used to detect "Fully Encrypted Traffic" is indeed complex, but the protocols investigated by the paper (at least Shadowsocks, VMess. Not really sure about Obfs4) works by transforming the traffic to make it "look like nothing". So I still believe "High Entropy" is a better description than "Fully Encrypted Traffic". I mean, you can pack the entire data stream in Base64 after sending them through a SHA256 pipeline, and it will still be "Fully Encrypted", but the entropy (in terms of traffic classification by content scanning) is not the same compare to doing it without the Base64 step.
  - luke-stanley 3y ago
    Yes, "High Entropy" or even "HighE" is a more accurate than "Fully Encrypted Traffic". It's interesting that the filter was observed to operate at specific ranges of entropy in the paper, and the repo has it mostly reproduced here: https://github.com/apernet/OpenGFW/blob/1dce82745d0bc8b3813aea147954ba3a2294ef30/analyzer/tcp/fet.go#L46 https://github.com/apernet/OpenGFW/blob/1dce82745d0bc8b3813a... But presumably to keep people on their toes, the real filter, only operates some of the time. It be a cursed, "hex ensemble".
  - nirui 3y ago
    Correction: instead of `SHA256`, I've should typed `AES256`. The last time when I wrote any encryption, it was still back in 2019... that's why I lost it... Sorry :)
kneoghau 3y ago
I've had luck finding pi-hole blocklists on github for various products, if it's something quite common like a branded smart TV someone will have already done the hard work of figuring out what IP's they're trying to dial home to.
- aacid 3y ago
  But pihole is dns only, any requests directly to ip address will still pass.
  - lightbritefight 3y ago
    Pi hole will happily block IPs. https://docs.pi-hole.net/ftldns/blockingmode/ https://docs.pi-hole.net/ftldns/blockingmode/
    - positr0n 3y ago
      None of those strategies block IPs. It's only a DNS server not a firewall or router.
benterix 3y ago
> You have no idea how much time I've wasted trying to block some products from pinging their home server with curious data stream, but failed to do so because I can't be bothered to sit my ass in front of Wireshark to sniff out all their DoH servers. You mean hardware products, right? In this case putting them in a separate VLAN would help. I you mean software running on your machine, you can set up a proxy and block all traffic not coming through it.
- nirui 3y ago
  Mainly TV, TV box and phones with non-open-source ROMs etc. These devices do need Internet connection for normal operations, which is why it's so hard to block suspicious traffic from them.
FuriouslyAdrift 3y ago
TLS 1.3 with cert pinning and end-to-end encryption is making life hell for corporate compliance. Our Palo Alto firewalls are about as good as it gets but it's a constant battle to de-obfuscate traffic. Google loves to mix traffic types (ad, telemetry, biz app) across protocols basically creating their own overlay which is a huge pain. For Apple, we basically have to exempt the entire 17.0.0.0/8 as that is theirs and they pin everything. Microsoft at least has several dynamic lists but I have my own list of their stuff that ISN'T on their lists (a few hundred IPs). At this point I don't think there's any way to secure a network. You can't trust or verify much of the traffic. Maybe using a synthetic NIC at the endpoint that is directly tied to the policy system is the only way forward.
- pests 3y ago
  How about the whole Zero Trust concept where network is assumed hostile and every service is properly authed?
  - zzyzxd 3y ago
    compliance != security. Good luck telling the auditor about zero trust.
    - Tknl 3y ago
      Your company hires the audit. Interview the to find one that gets zero trust.
creativeSlumber 3y ago
> You have no idea how much time I've wasted trying to block some products from pinging their home server with curious data stream, but failed to do so because I can't be bothered to sit my ass in front of Wireshark to sniff out all their DoH servers. I wonder if you can run this in observe only mode to analyze/log that traffic.
inemesitaffia 3y ago
Link? To Twitter?

by nirui 3y ago

luke-stanley 3y ago

I briefly read a paper, I suspect it's more complex than "high entropy".

nirui 3y ago
The strategy used to detect "Fully Encrypted Traffic" is indeed complex, but the protocols investigated by the paper (at least Shadowsocks, VMess. Not really sure about Obfs4) works by transforming the traffic to make it "look like nothing". So I still believe "High Entropy" is a better description than "Fully Encrypted Traffic". I mean, you can pack the entire data stream in Base64 after sending them through a SHA256 pipeline, and it will still be "Fully Encrypted", but the entropy (in terms of traffic classification by content scanning) is not the same compare to doing it without the Base64 step.
- luke-stanley 3y ago
  Yes, "High Entropy" or even "HighE" is a more accurate than "Fully Encrypted Traffic". It's interesting that the filter was observed to operate at specific ranges of entropy in the paper, and the repo has it mostly reproduced here: https://github.com/apernet/OpenGFW/blob/1dce82745d0bc8b3813aea147954ba3a2294ef30/analyzer/tcp/fet.go#L46 https://github.com/apernet/OpenGFW/blob/1dce82745d0bc8b3813a... But presumably to keep people on their toes, the real filter, only operates some of the time. It be a cursed, "hex ensemble".
- nirui 3y ago
  Correction: instead of `SHA256`, I've should typed `AES256`. The last time when I wrote any encryption, it was still back in 2019... that's why I lost it... Sorry :)

kneoghau 3y ago

I've had luck finding pi-hole blocklists on github for various products, if it's something quite common like a branded smart TV someone will have already done the hard work of figuring out what IP's they're trying to dial home to.

aacid 3y ago
But pihole is dns only, any requests directly to ip address will still pass.
- lightbritefight 3y ago
  Pi hole will happily block IPs. https://docs.pi-hole.net/ftldns/blockingmode/ https://docs.pi-hole.net/ftldns/blockingmode/
  - positr0n 3y ago
    None of those strategies block IPs. It's only a DNS server not a firewall or router.

benterix 3y ago

> You have no idea how much time I've wasted trying to block some products from pinging their home server with curious data stream, but failed to do so because I can't be bothered to sit my ass in front of Wireshark to sniff out all their DoH servers. You mean hardware products, right? In this case putting them in a separate VLAN would help. I you mean software running on your machine, you can set up a proxy and block all traffic not coming through it.

nirui 3y ago
Mainly TV, TV box and phones with non-open-source ROMs etc. These devices do need Internet connection for normal operations, which is why it's so hard to block suspicious traffic from them.

FuriouslyAdrift 3y ago

TLS 1.3 with cert pinning and end-to-end encryption is making life hell for corporate compliance. Our Palo Alto firewalls are about as good as it gets but it's a constant battle to de-obfuscate traffic. Google loves to mix traffic types (ad, telemetry, biz app) across protocols basically creating their own overlay which is a huge pain. For Apple, we basically have to exempt the entire 17.0.0.0/8 as that is theirs and they pin everything. Microsoft at least has several dynamic lists but I have my own list of their stuff that ISN'T on their lists (a few hundred IPs). At this point I don't think there's any way to secure a network. You can't trust or verify much of the traffic. Maybe using a synthetic NIC at the endpoint that is directly tied to the policy system is the only way forward.

pests 3y ago
How about the whole Zero Trust concept where network is assumed hostile and every service is properly authed?
- zzyzxd 3y ago
  compliance != security. Good luck telling the auditor about zero trust.
  - Tknl 3y ago
    Your company hires the audit. Interview the to find one that gets zero trust.

creativeSlumber 3y ago

> You have no idea how much time I've wasted trying to block some products from pinging their home server with curious data stream, but failed to do so because I can't be bothered to sit my ass in front of Wireshark to sniff out all their DoH servers. I wonder if you can run this in observe only mode to analyze/log that traffic.

inemesitaffia 3y ago

Link? To Twitter?