4 ms·

I understand that these products has market demands from paranoid but not IP networking related businesses, but I never quite understood the fundamental basic p

I understand that these products has market demands from paranoid but not IP networking related businesses, but I never quite understood the fundamental basic premise of Palo Alto, F5 Networks, Fortinet, etc. brands of "MITM TLS firewall" products. These firewall boxes are on-prem white hat Mallory, reverse-reverse-proxying all TLS traffic. And of course the Linux stack it uses has tons of RCEs and misconfigurations. Isn't that just insecure???

p_l 3y ago
In case of Palo Alto Networks TLS interception wasn't the only, or even main, use in many places. Among reasons one might have seen them was centralised control plane, multipoint VPNs, yes deep-packet inspection (including for simply checking if the expected protocol was running on given traffic), they could be also simply used as pretty advanced router+firewall setup.
- amluto 3y ago
  If you have a TLS MITM proxy configured and an attacker pwns the proxy, it’s pretty much game over. Forget access to the internal network: any host that has the MITM proxy’s certificate installed will trust it to view and modify all TLS traffic. This gets a free attack on all web origins without even compromising anything else. AWS console, check. Configuration of other corporate appliances, check. Everyone’s communication tools, check. And you get to replace anything downloaded by anything that doesn’t use certificate pinning.
  - p_l 3y ago
    I'm not saying no. I'm just saying that not everyone used PAN NGFW for MITM proxying, and it's not necessary to enable/configure that to use them for other tasks.
WirelessGigabit 3y ago
Company I worked at turned on MITM TLS for everything. Suddenly a lot of stuff stopped working, because not every piece of software on my machine uses the OS's certificate store. For example Docker containers who then curl to set up stuff.
- unethical_ban 3y ago
  Yeah, it needs to be rolled out and tested a lot. And not all traffic should be decrypted, namely banking and health.
- sebazzz 3y ago
  I hate ZScaler with a passion.
  - Sohcahtoa82 3y ago
    At my last job, my CISO made the company trial ZScaler. When it caused tons of problems for Engineering, he cancelled the trial, then tried out CloudFlare's MitM proxy, which of course had the same problems. I had to talk to him and say look, what you're trying to do isn't possible without breaking things. It's not a limitation of the products, it's a limitation of the underlying security that you're trying to bypass, and you're making security WORSE, not BETTER. He wanted to log every employee's Internet traffic so that he could determine if someone was leaking company data, such as source code, customer data, etc. I said that there's nothing you can do to stop that, and even if someone knows their traffic is being monitored, it's easy to bypass by just adding a second layer of encryption.
- neilv 3y ago
  One company had a related (but less defensible) decision, coming down from the top, which broke CI runners and other things, and the poor overworked Git&CI infra lead was trying to work around it. They asked me to be a Git reviewer for their big workaround, and I found around a dozen new vulnerabilities and future build-breaking defects that the workaround introduced. I also told them that it's unreasonable for this other team to ask them to work around that mess, and the solution is for the other team to do the thing in a different, much simpler way. I also ended up raising some even bigger security decision problems with the C-suite. As one engineer from a different team told me (after they'd given notice of leaving, and kindly granted me an exit interview), the company had too many people making other people's jobs harder, for no reason. A too-entrenched-to-fail-soon company might be able to justify that (say, it was the easiest way to check off a compliance box, with their encumbered velocity at implementing anything). Most companies aren't that, though.
unethical_ban 3y ago
The premise is that a business needs to know what kind of data is passing through its networks. SSL forward proxy (man in the middle) helps this by letting the firewall see contents of data, more thoroughly govern access to websites and subcomponents of websites, and so on. For example, with MITM enabled, a Palo firewall can grant access to the viewing of Youtube videos, but not the upload or commenting of them. It can also scan files for malware, do analysis on shady looking javascript, and run DLP and credential checks to make sure no one is accidentally sending sensitive data to the wrong site. The SSL forward proxy feature of Palos has a number of things that preserve the security of TLS connections, such as requiring a minimum TLS version, "passing through" cert errors like expiration or bad CN, and so on. I've worked with PA firewalls for a while.
- dcow 3y ago
  Honest question: have you ever seen a corporate TLS middleware box stop an active threat? And I don’t mean drive-by crap like port scanning for sshd…
  - chelmzy 3y ago
    Its typically used for detecting malware, detecting data loss prevention, or forensics. "Drive by crap like port scanning for sshd" isn't even relevant to why companies mitm SSL. And yes I see it detect but not stop active threats on a daily basis.
    - dcow 3y ago
      These tools typically claim to block or catch stupidly high number of threats per unit time. And they usually bucket the drive-by stuff as a threat to achieve this narrative. I agree it’s hogwash. So you’re essentially admitting that these TLS boxes are more about controlling employees and “data loss prevention” than actually preventing real threats and doing honest security. Got it. https://honest.security https://honest.security
      - unethical_ban 3y ago
lormayna 3y ago
As someone that I have worked for years with this kind of products in a couple of F500, I hate them with all of my heart. I really prefer the approach based on DNS and IP filtering, it's easier to manage and doesn't have all the problems that SSL inspection has.
thephyber 3y ago
> Isn't that just insecure??? You seem to be singularly focused on the software quality of the VPN. It’s important, but is far from the only aspect of security. The company is worried both about keeping the bad guys out of their network and keeping their IP + secrets inside. The motivation for the TLS MITM blinks boxes is for the latter. It’s easy as an employee to simply discount the value of decrypting your internet traffic, but the truth is that the company has a responsibility to protect against malicious insiders, malware/ransomware exfiltration, etc.
- numpad0 3y ago
  > but the truth is that the company has a responsibility to ... These boxes provide a one-stop hacker shop for all data exfil and malware injection that normally don't exist. In theory they can _sign security updates_, send fake announcements, just intercept, redirect and drop emails, etc. It's just too hard for me to understand how it's supposed to add security, unless these would be NSA endorsed or something.
eemil 3y ago
Modern L7 firewalls do a lot more than just MITM, and they can do it without MITM. Take a look at Palo Alto's applipedia[1], it has 4419 protocols/services that you can detect and write security policy for without decrypting anything. One example, last week our firewall blocked a suspicious (but but in this case legitimate) site using DNS sinkholing, because the domain seemed suspicious (uncommon TLD + recently registered domain). [1]: https://applipedia.paloaltonetworks.com/ https://applipedia.paloaltonetworks.com/

p_l 3y ago

In case of Palo Alto Networks TLS interception wasn't the only, or even main, use in many places. Among reasons one might have seen them was centralised control plane, multipoint VPNs, yes deep-packet inspection (including for simply checking if the expected protocol was running on given traffic), they could be also simply used as pretty advanced router+firewall setup.

amluto 3y ago
If you have a TLS MITM proxy configured and an attacker pwns the proxy, it’s pretty much game over. Forget access to the internal network: any host that has the MITM proxy’s certificate installed will trust it to view and modify all TLS traffic. This gets a free attack on all web origins without even compromising anything else. AWS console, check. Configuration of other corporate appliances, check. Everyone’s communication tools, check. And you get to replace anything downloaded by anything that doesn’t use certificate pinning.
- p_l 3y ago
  I'm not saying no. I'm just saying that not everyone used PAN NGFW for MITM proxying, and it's not necessary to enable/configure that to use them for other tasks.

WirelessGigabit 3y ago

Company I worked at turned on MITM TLS for everything. Suddenly a lot of stuff stopped working, because not every piece of software on my machine uses the OS's certificate store. For example Docker containers who then curl to set up stuff.

unethical_ban 3y ago
Yeah, it needs to be rolled out and tested a lot. And not all traffic should be decrypted, namely banking and health.
sebazzz 3y ago
I hate ZScaler with a passion.
- Sohcahtoa82 3y ago
  At my last job, my CISO made the company trial ZScaler. When it caused tons of problems for Engineering, he cancelled the trial, then tried out CloudFlare's MitM proxy, which of course had the same problems. I had to talk to him and say look, what you're trying to do isn't possible without breaking things. It's not a limitation of the products, it's a limitation of the underlying security that you're trying to bypass, and you're making security WORSE, not BETTER. He wanted to log every employee's Internet traffic so that he could determine if someone was leaking company data, such as source code, customer data, etc. I said that there's nothing you can do to stop that, and even if someone knows their traffic is being monitored, it's easy to bypass by just adding a second layer of encryption.
neilv 3y ago
One company had a related (but less defensible) decision, coming down from the top, which broke CI runners and other things, and the poor overworked Git&CI infra lead was trying to work around it. They asked me to be a Git reviewer for their big workaround, and I found around a dozen new vulnerabilities and future build-breaking defects that the workaround introduced. I also told them that it's unreasonable for this other team to ask them to work around that mess, and the solution is for the other team to do the thing in a different, much simpler way. I also ended up raising some even bigger security decision problems with the C-suite. As one engineer from a different team told me (after they'd given notice of leaving, and kindly granted me an exit interview), the company had too many people making other people's jobs harder, for no reason. A too-entrenched-to-fail-soon company might be able to justify that (say, it was the easiest way to check off a compliance box, with their encumbered velocity at implementing anything). Most companies aren't that, though.

unethical_ban 3y ago

The premise is that a business needs to know what kind of data is passing through its networks. SSL forward proxy (man in the middle) helps this by letting the firewall see contents of data, more thoroughly govern access to websites and subcomponents of websites, and so on. For example, with MITM enabled, a Palo firewall can grant access to the viewing of Youtube videos, but not the upload or commenting of them. It can also scan files for malware, do analysis on shady looking javascript, and run DLP and credential checks to make sure no one is accidentally sending sensitive data to the wrong site. The SSL forward proxy feature of Palos has a number of things that preserve the security of TLS connections, such as requiring a minimum TLS version, "passing through" cert errors like expiration or bad CN, and so on. I've worked with PA firewalls for a while.

dcow 3y ago
Honest question: have you ever seen a corporate TLS middleware box stop an active threat? And I don’t mean drive-by crap like port scanning for sshd…
- chelmzy 3y ago
  Its typically used for detecting malware, detecting data loss prevention, or forensics. "Drive by crap like port scanning for sshd" isn't even relevant to why companies mitm SSL. And yes I see it detect but not stop active threats on a daily basis.
  - dcow 3y ago
    These tools typically claim to block or catch stupidly high number of threats per unit time. And they usually bucket the drive-by stuff as a threat to achieve this narrative. I agree it’s hogwash. So you’re essentially admitting that these TLS boxes are more about controlling employees and “data loss prevention” than actually preventing real threats and doing honest security. Got it. https://honest.security https://honest.security
    - unethical_ban 3y ago

lormayna 3y ago

As someone that I have worked for years with this kind of products in a couple of F500, I hate them with all of my heart. I really prefer the approach based on DNS and IP filtering, it's easier to manage and doesn't have all the problems that SSL inspection has.

thephyber 3y ago

> Isn't that just insecure??? You seem to be singularly focused on the software quality of the VPN. It’s important, but is far from the only aspect of security. The company is worried both about keeping the bad guys out of their network and keeping their IP + secrets inside. The motivation for the TLS MITM blinks boxes is for the latter. It’s easy as an employee to simply discount the value of decrypting your internet traffic, but the truth is that the company has a responsibility to protect against malicious insiders, malware/ransomware exfiltration, etc.

numpad0 3y ago
> but the truth is that the company has a responsibility to ... These boxes provide a one-stop hacker shop for all data exfil and malware injection that normally don't exist. In theory they can _sign security updates_, send fake announcements, just intercept, redirect and drop emails, etc. It's just too hard for me to understand how it's supposed to add security, unless these would be NSA endorsed or something.

eemil 3y ago

Modern L7 firewalls do a lot more than just MITM, and they can do it without MITM. Take a look at Palo Alto's applipedia[1], it has 4419 protocols/services that you can detect and write security policy for without decrypting anything. One example, last week our firewall blocked a suspicious (but but in this case legitimate) site using DNS sinkholing, because the domain seemed suspicious (uncommon TLD + recently registered domain). [1]: https://applipedia.paloaltonetworks.com/ https://applipedia.paloaltonetworks.com/