4 ms·

you'd be surprised what a "deep packet inspection" (better known as DPI or L7-aware) firewall can tell. some are quite sophisticated, and there are a surprising

by pyvpx 10y ago

you'd be surprised what a "deep packet inspection" (better known as DPI or L7-aware) firewall can tell. some are quite sophisticated, and there are a surprising number of techniques to identify even encrypted traffic (not the precise payload, per se)

Buge 10y ago
Well there's ALPN/NPN, but you could lie about those. I would indeed find it surprising if any commercial firewalls distinguish websockets over TLS versus raw bytes over TLS. Not that it's impossible, but I wouldn't think it would be done commercially.
- peterwwillis 10y ago
  Firewall packet inspection is basically just unpacking protocols layer by layer. Once you find the TCP or HTTP layer, you see if what follows looks like a WebSocket, which has a unique fingerprint and predictable elements. You can also look at streams and find patterns, like how one packet looks like HTTP, and then the following ones on the same stream don't look like HTTP. Initial versions may not be very accurate, but on "extremely paranoid" settings you can simply reject traffic that seems suspicious. Also, having the DoD as a client will buy you some serious R&D time.
  - Buge 10y ago
    You can't unpack protocols layer by layer when they're encrypted.
    - peterwwillis 10y ago
      True, you would typically use machine learning, which can be used to identify and separate layers in protocols.
      - Buge 10y ago
        Machine learning on random data won't get you anything. And is machine learning fast enough for live network filters?
        peterwwillis 10y ago
        It was fast enough 10 years ago. And it worked on random data. Luckily though, internet traffic is not random data, and encrypted internet traffic is much less random than you'd think. Of course a lot of it involves inferring information about different parts of flows, but high accuracy is not difficult. (What's difficult to detect is traffic you haven't got any trained models for)
        Buge 10y ago
        If it's actually random data then no it won't work. Actual random data is 100% noise and 0% signal. If the ML thinks it's picked up some signal, then it's wrong. You're right that TLS is not 100% random, there is some information to work with: the size of the encrypted blobs (they're rounded because of padding) and the timing of them. Are there any commercial firewalls that do analysis of this kind for the purpose of blocking the traffic? It wouldn't be able the block all the traffic, because it would have to wait a while to get more timing data that it can apply its heuristics, then end the connection. And it wouldn't be able to unpack protocol layers.
        peterwwillis 10y ago
        You're right, I misspoke, "random" data is nothing, but "random" as in "arbitrary selections of network data" it will work on. And no, it doesn't unpack layers per-se, but it detects signatures and can be used to find patterns and eventually separate layers of protocols. The two oldest and most successful methods that I know of are matching on payload length and models trained on the initialization of network application protocols, neither of which requires constantly re-sampling and re-classifying to get a hit. And blocking traffic is often more about terminating an existing connection once you detect something bad going over it (deep content inspection). Yes, commercial firewalls do look for tunneled applications. Palo Alto Networks has a patent on it (App-ID), Websense/Forcepoint does it (Content Gateway Analysis), Cisco sort of implements it (Network Based Application Recognition/Application Visibility and Control). A bunch of open source software implements it, too. Some commercial proprietary software is also out there, with PACE leading the pack. Wikileaks has one of their product data sheets: https://wikileaks.org/spyfiles/docs/IPOQUE-PACEProtAppl-en.pdf https://wikileaks.org/spyfiles/docs/IPOQUE-PACEProtAppl-en.p... Honestly, a lot of customers simply force proxies on their users and inspect all their traffic and drop anything that it can't inspect, so there probably isn't a lot of commercial need for this. But it is out there.

by pyvpx 10y ago

Buge 10y ago

Well there's ALPN/NPN, but you could lie about those. I would indeed find it surprising if any commercial firewalls distinguish websockets over TLS versus raw bytes over TLS. Not that it's impossible, but I wouldn't think it would be done commercially.

peterwwillis 10y ago
Firewall packet inspection is basically just unpacking protocols layer by layer. Once you find the TCP or HTTP layer, you see if what follows looks like a WebSocket, which has a unique fingerprint and predictable elements. You can also look at streams and find patterns, like how one packet looks like HTTP, and then the following ones on the same stream don't look like HTTP. Initial versions may not be very accurate, but on "extremely paranoid" settings you can simply reject traffic that seems suspicious. Also, having the DoD as a client will buy you some serious R&D time.
- Buge 10y ago
  You can't unpack protocols layer by layer when they're encrypted.
  - peterwwillis 10y ago
    True, you would typically use machine learning, which can be used to identify and separate layers in protocols.
    - Buge 10y ago
      Machine learning on random data won't get you anything. And is machine learning fast enough for live network filters?
      - peterwwillis 10y ago
        It was fast enough 10 years ago. And it worked on random data. Luckily though, internet traffic is not random data, and encrypted internet traffic is much less random than you'd think. Of course a lot of it involves inferring information about different parts of flows, but high accuracy is not difficult. (What's difficult to detect is traffic you haven't got any trained models for)
        Buge 10y ago
        If it's actually random data then no it won't work. Actual random data is 100% noise and 0% signal. If the ML thinks it's picked up some signal, then it's wrong. You're right that TLS is not 100% random, there is some information to work with: the size of the encrypted blobs (they're rounded because of padding) and the timing of them. Are there any commercial firewalls that do analysis of this kind for the purpose of blocking the traffic? It wouldn't be able the block all the traffic, because it would have to wait a while to get more timing data that it can apply its heuristics, then end the connection. And it wouldn't be able to unpack protocol layers.
        peterwwillis 10y ago
        You're right, I misspoke, "random" data is nothing, but "random" as in "arbitrary selections of network data" it will work on. And no, it doesn't unpack layers per-se, but it detects signatures and can be used to find patterns and eventually separate layers of protocols. The two oldest and most successful methods that I know of are matching on payload length and models trained on the initialization of network application protocols, neither of which requires constantly re-sampling and re-classifying to get a hit. And blocking traffic is often more about terminating an existing connection once you detect something bad going over it (deep content inspection). Yes, commercial firewalls do look for tunneled applications. Palo Alto Networks has a patent on it (App-ID), Websense/Forcepoint does it (Content Gateway Analysis), Cisco sort of implements it (Network Based Application Recognition/Application Visibility and Control). A bunch of open source software implements it, too. Some commercial proprietary software is also out there, with PACE leading the pack. Wikileaks has one of their product data sheets: https://wikileaks.org/spyfiles/docs/IPOQUE-PACEProtAppl-en.pdf https://wikileaks.org/spyfiles/docs/IPOQUE-PACEProtAppl-en.p... Honestly, a lot of customers simply force proxies on their users and inspect all their traffic and drop anything that it can't inspect, so there probably isn't a lot of commercial need for this. But it is out there.