12 ms·
Deep packet inspection is dead, and here's why (2017)
- drieddust 8y agoNot a very informative article. All it manages to say is that deep packet inspection does not work with encrypted traffic. I think author is not aware of transparent deep packet inspection of SSL traffic. Here is one such product doing it. https://www.sonicwall.com/en-us/products/firewalls/security-services/dpi-ssl https://www.sonicwall.com/en-us/products/firewalls/security-...
- detaro 8y agoThe article describes such products, so the author clearly is aware of their existence.
- mnw21cam 8y agoActually, that's kind of what the whole of the second half of the article is about.
- corebit 8y agoThat’s just a run-of-the-mill MITM privacy violator
- SideburnsOfDoom 8y agoYes, but I don't think the article explains why it is, or will be, "dead". Companies that have them don't want to give them up. What compelling reason would make them? "Employee privacy at work" is not one. Not with the level of perceived threat of malware downloads and trojaned NPM packages.
- yholio 8y agoBreaking TLS so you can do deep packet inspection is like a lifeguard throwing people in the water during winter so he can save them.
- hunter2_ 8y agoOr a lifeguard blowing their whistle at folks who specifically used the "no lifeguard on duty" beach so they could swim out far.
- lpcvoid 8y agoThe author suggests towards the end to analyze DNS queries, but that's on the best way [1] to be encrypted as well (finally). [1] https://wiki.mozilla.org/Trusted_Recursive_Resolver https://wiki.mozilla.org/Trusted_Recursive_Resolver
- dstjean 8y agoIn a corporate environment, managed devices can be configured to force the use of specific DNS settings. The same type of implementation (MITM) could be used to analyse the requests. That being said, this is at the OS level. An app such as Firefox could still override those settings or provide their own implementation.
- jopsen 8y agoIf you IT department is your adversary you should get a new job. Or at least use a personal device for personal matters :)
- dstjean 8y agoI don't think NOT performing packet inspection due to privacy concern is a good idea. (Good security controls should exist over its administration) One reason why organizations use packet inspection is to protect its staffs, customers and vendors from malicious actors who could cause data breaches leading to huge privacy issues. Privacy over Security? The right balance must be found
- LeifCarrotson 8y agoA user has no way of knowing whether a packet inspection will be performed by benevolent actors seeking to protect their security or by malicious actors seeking to invade their privacy. As in the good old post "What colour are your bits" [1] regarding the subject of copyright, the computer is colorblind when it comes to privacy vs. security tradeoffs. You seem to see color, believing compromise for security to be acceptable, and hoping you can allow your lawful and good security inspections to occur while disallowing nasty privacy invasion. The computer doesn't see color. It is impossible to build a security protocol that will distinguish between good third parties and malicious third parties. "Good security controls" come down to trusting people to do the right thing, and when there's big money coercing companies to do the wrong thing, the right thing too often loses. [1]: https://ansuz.sooke.bc.ca/entry/23/ https://ansuz.sooke.bc.ca/entry/23/
- xer 8y agoPDI is just one tool in the toolbox. It's never gonna die.
- bawana 8y agoI did not realize that squid could provide false certificates on the fly. The whole business of invalid certificates made people nervous about some sites. Now someone can sit in starbucks with a squid proxy in the middle and harvest everything, regardless of ssl encryption. Looking at the little lock in the URL means nothing to a MITM running squid. Will a VPN protect me by encrypting everything from my machine so that a squid in the middle will be thwarted?
- nickthemagicman 8y agoThe whole point of certificates is for the browser to check with a cert authority. How does squid circumvent the certificate authority? I think VPN may not be safe if the local machine has to negotiate encryption with the VPN server. Squid seems like it couldn't intercelt that.
- detaro 8y agoIt doesn't. In enterprise environments that use something like this, the sysadmins install their own CA certificate on all machines. You can't just MITM random machines at a coffeeshop.
- deleted 8y ago[deleted]
- Tepix 8y agoYour machine (browser) will only accept the false certificates without complaining a lot if you have previously added the certification authority of the attacker to your browsers list of valid CAs.
- mabbo 8y agoA few years ago, one of the best managers I ever worked for left to become the CTO of a company doing pattern analysis of network traffic, rather than Deep Packet Inspection. The premise was that most of the internet traffic on your network follows the same typical patterns, but nefarious traffic doesn't. Drop their system into the network and voila, you can start to find the weird things going on that seem out of the ordinary. At the time, I thought that it seemed a bit heavy-handed- just use DPI and you'll get the same results. This article is making me think he was very prescient in the matter.
- m-app 8y agoThis is exactly what has been researched at multiple security companies and productized by Cisco under "Encrypted Traffic Analytics". This is based on research from 2016 that can be found on arXiv: https://arxiv.org/abs/1607.01639 https://arxiv.org/abs/1607.01639 > We conclude that malware's usage of TLS is distinct from benign usage in an enterprise setting, and that these differences can be effectively used in rules and machine learning classifiers. Disclaimer: I work for Cisco
- vlovich123 8y agoNeat paper but as soon as this becomes more widespread malware authors are going to adapt to hide as regular traffic so the analysis is going to get more & more complex until it's not useful as malware traffic will look indistinguishable from real traffic. This is a fundamental evolutionary cat & mouse game that's impossible to win; antibiotics & bacteria, toxins in prey & toxicity resistance in predators, etc.
- ap0phenia 8y ago& autoimmune diseases...
- genpfault 8y ago> Drop their system into the network and voila, you can start to find the weird things going on that seem out of the ordinary. hmmm[1] [1]: https://everything2.com/user/The+Custodian/writeups/Seek+And+You+Shall+Find https://everything2.com/user/The+Custodian/writeups/Seek+And...
- mimixco 8y agoTL;DR = Because encryption.
- helen___keller 8y agoI think a more correct title would be "Deep packet inspection should be dead, and here's why" Schools, financial institutions, and more will pay big bucks to web gateway vendors who will help them deploy man in the middle attacks on their own machines, employ blacklists or whitelists (even on Google search terms not just at the DNS level), scan traffic for SSNs, and so on. It's not a dead market (quite the opposite, startups like Zscaler are fetching unicorn valuation). It also encourages terrifying but legal behavior for employers like monitoring which subreddits you read or what kind of YouTube videos you watch or how much time you spend slacking off at work. The arms race between security and exploitation isn't likely to stop, and I have no confidence that corporations with sensitive data will willingly take a privacy-granting approach when vendors promise them unmatched security by decrypting traffic. I think the two viable approaches are educating the public that your work machine is not private or looking for lawmakers to step in (but let's be real, that option is unlikely) During my time working for one of these web gateway vendors, I became highly sensitive to what browsing happened on my primary operating system (which had company certificates installed), and what went on my development VM (which I set up myself without corporate certificates)
- dillz 8y agoMy workplace has such a MitM gateway where every host has a company root CA installed and every SSL certificate we receive in the browser is an interchanged one. Fair enough. However, the huge problem is that employees are completely left in the dark about this privacy invasion... only the tech-savvy ones notice and understand it.
- rocqua 8y agoI'm worried about this development. One the one hand, ubiquitous encryption is simply required for security on the internet. Things like lets encrypt and warning on http are great improvements. On the other hand, the owner of a network has some right to look into the packets on that network. Especially if the owner of the network also owns the end-points of that traffic. My main use-case here isn't corporate networks, snooping there makes me uncomfortable. Really, my issue is stuff on my own network. I want to see what my TV sends home. Same with an amazon-echo, or really any IoT thing. Yet, if they all use SSL and don't allow me to add a root CA, I can't look at what they run. A user has no control over an amazon echo. You can't modify the software because the bootloader is locked down. You can't inspect the traffic because it is SSL cert-pinned. Amazon can push updates to it at any time. All a user gets to do is decide whether it is turned on, and whether it gets a network connection. Really, what I would want to see is the option to install a CA cert on any device I own. At the same time, that is a terrible idea. Every 14 year old with google is going to find some stack-overflow answer that'll tell them to MitM their TV to do some simple thing.
- k__ 8y agoWhile I understand your point of view, I think the plausible deniability network owners get frees them from much risk and burocracy.
- rocqua 8y agoTo be honest, I'm probably happy with the breaking of corporate blanket MitM-ing. It was never very effective because data exfiltration can be made very hard to detect. What worries me is that the full chain (Client-software -- Network -- Server-software) is totally opaque and immutable at the discretion of a vendor. That means we are left at the mercy of the vendors. I'm hoping for laws like 'Right to Repair' to help with this. The alternative would be to go full Richard Stallman, and I still think that is too radical.
- hsbaut76 8y agoI share your worry about not being able to inspect communications for various apps and devices purely from a privacy advocacy perspective. If you own a device, and said device is transmitting data from your environment, you should be able to know what information this device is communicating. It is not enough to trust a company privacy policy.
- kijin 8y agoDeep packet inspection seems to be alive and well, even outside of corporate networks. My ISP uses the User-Agent header in outgoing requests to guess how many computing devices I have at home, and tries to charge money if it's more than an undisclosed limit. This of course only works for plain HTTP, but there are still enough unencrypted sites out there that my ISP has an opportunity to intercept a request at least a couple of times a day. Meanwhile, my country is just beginning to roll out a system that detects the SNI hostname in encrypted connections, in order to block illegal sites that hide behind Cloudflare. Fortunately they can't spoof certificates on the public internet, so users just get a connection error. Too bad Cloudflare supports ESNI now ;)
- rcarmo 8y agoWhere do you live (if you can share the country name, of course)?
- kijin 8y agoSouth Korea. Nobody is under any threat of prosecution for talking about our ridiculous censorship regime, and the surveillance side of the program is probably no worse than in any other developed country. Which isn't much of a compliment, but at least we're not China-level evil -- just incompetent. DPI for blocking SNI hostnames is a particularly annoying way to waste taxpayers' money. It's almost as if they timed it to coincide with wide availability of DoH and ESNI!
- deleted 8y ago[deleted]
- anonymousisme 8y agoIt's not dead. Encryption has (unjustifiably) pushed the enterprise to install fake catchall certificates on proxies so they can snoop plain-text traffic. (Why anyone would ever think this is a good idea is beyond me.)
- robohoe 8y agoCorporate MITM devices/proxies are surely in a new business boom. Now we went from lack of encryption to encryption with MITM certificates on questionable appliances running questionable code.
- jandrese 8y agoHow else are you going to catch APT (Advanced Persistent Threat) data exfiltration/control channel traffic? Assumption 1: Machines on your network are already compromised and fully owned by a sophisticated and extremely difficult to detect rootkit. This is true of every large business. There is always that guy who will click on any link or open the document from what appears to be their co-worker. Assumption 2: APT tries to disguise their traffic as ordinary web traffic, because anything else is suspicious. Assumption 3: You have massive legal liabilities if your data is exfiltrated. Being able to do DPI and pattern matching on all TLS traffic (and firewall off anything you can't DPI) is pretty much mandatory.
- zrm 8y ago> There is always that guy who will click on any link or open the document from what appears to be their co-worker. Which is another reason why DPI is ineffective. The smart malware will identify when its connection is presenting a custom root certificate rather than the expected one and not proceed with its suspicious activities (if not deploy some kind of steganography). Then the same "that guy" will plug his personal phone into his computer, and now the malware has an unmonitored cellular data connection to the outside on a machine that's also connected to the internal network. Or a compromised laptop will hook up to the WiFi of the company on the adjacent floor or the coffee shop next door, or the user connects it to the coffee shop WiFi when they're in the coffee shop. In theory you can build a Faraday cage around your space and then strip-search employees for digital devices at the door, but if your data is that important then you probably ought to just not be connected to the internet at all.
- rcarmo 8y agoThere was a pre-2010 burst of interest in DPI in the carrier world, back when they thought it would be feasible to bill different kinds of traffic separately (i.e., beyond zero-rating traffic's to their walled gardens). That lead to an arms race from core networking vendors to push out all sorts of traffic sniffing and policing with insane degrees of intrusion that made me quite uneasy (I worked in core network planning), and it's been a relief to finally see LetsEncrypt take hold and TLS become de rigeur. I do have some qualms about the way legal interception can be abused (in general) and occasionally ponder how far those vendors may have progressed in MITM, though - carriers and exchange points are not as secure as they should (in sometimes surprising ways), and back then finding bugs in carrier equipment was relatively frequent. I wonder what's it like now that most of it are actually Linux VMs running someplace in their ancient datacenters.
- Scoundreller 8y agoThe other interest from carriers is protecting their media interests. Slowing torrents or streaming video directly helps maintain their « golden age of double dipping by running data over lines paid by audio/vidéo infrastructure »
- rcarmo 8y agoThe "policing" bit was actually about doing that. Strategies varied, from smooth shaping to randomly dropping packets to force TCP window resets and drastically lower throughput.
- suff 8y agoAuthor is dead wrong. Products exist today that perform DPI on SSL streams: https://www.a10networks.com/resources/articles/ssl-inspection-decryption-cisco-asa-firepower https://www.a10networks.com/resources/articles/ssl-inspectio...
- Florin_Andrei 8y agoThew author does mention that's doable if you break the SSL tunnel. They also mention some ethical issues with doing that.
- jimmychangas 8y agoNot related to the core of the article, but it taught me I can pipe random gibberish (such as tcpdump) to the audio output and I am finding it amazing.
- jordan314 8y agoThis sent me on a spiral of checking for MITM connections on my machine. You can compare the fingerprints of known sites with this list on this site: https://www.grc.com/fingerprints.htm https://www.grc.com/fingerprints.htm Though I think the facebook one is wrong (the one I see starts with BD 25 8C for SHA-1)
- chrischen 8y agoCool so how would I use these to circumvent the great chinese firewall with my SOCKS tunnel?
- adrianratnapala 8y agoThis sort of development seems good, not exactly from an moral point of view, but from the point of view of long-term reliability of the internet. The IP protocols have some expectation of end-to-end packet delivery. Over time we found ways in which networks could be kept "working" with this requirement relaxed. Except what could be known to "work" was just whatever was tested by the manufacturers of various middle-boxes, making change and development of new ways of solving problems harder than it should be. The less visibility middle-boxes have into what the the traffic is, the less they are able to selectively screw things up and the internet will be more reliable for it.
- shaklee3 8y agoLuca Deri, the author of nDPI did an excellent talk on this topic at the DPDK summit in December. The techniques they have to use now to apply heuristics on https is really cool: https://www.youtube.com/watch?v=4Vp8-UONhmM&t=0s&index=17&list=PLo97Rhbj4ceISWDa6OxsbEx2jBPaymJWL https://www.youtube.com/watch?v=4Vp8-UONhmM&t=0s&index=17&li...
- 75dvtwin 8y agoI think (and hope), that the next big thing (after https) -- will be VPNs by default. (and independent from the internet provider service). By default, nobody, and I mean, nobody needs to know ones home IP address, period. And nobody needs know what sites a person visit or when. So not only DPI should go away, but also IP address-based blacklisting/whitelisting, tracking/ advertising and so on.