7 ms·

Use of firewalls in series from multiple vendors sounds like a good idea in theory but in practice, does it not make it easier for an attacker to exploit the ne

by dhx 5y ago

Use of firewalls in series from multiple vendors sounds like a good idea in theory but in practice, does it not make it easier for an attacker to exploit the network? Instead of the attacker having to find a vulnerability in a single product, they can instead find a vulnerability in one of multiple products (a much easier task). Given that every network device communicates with a central authentication/authorisation system, a central logging system and likely central patching, configuration deployment, etc systems, all it takes is to find a vulnerability in one of multiple firewall products, then find a vulnerability in one of the central systems used to mange the network. I'm also perplexed why there is mention of "traffic inspector" and "full-packet capture device" given that almost all traffic traversing a network nowadays is encrypted. Perhaps more useful today would be creating a good understanding of the normal traffic flows so that alarms can be configured for abnormal traffic. For example, perhaps no more than 100 requests to an authentication server occur per device per day. Or patches for a system are no more than 1GB so seeing 1.1GB or more transferred across the management network per day per device would be abnormal.

throwbigdata 5y ago
That’s not how “in series” works.
vlovich123 5y ago
> Instead of the attacker having to find a vulnerability in a single product, they can instead find a vulnerability in one of multiple products (a much easier task) I’m not sure I understand this argument you’re making. If you have them in series then you have to find vulnerabilities for all of them before you can talk to any other system. A vulnerability to one of them only lets you punch through that one layer. In theory of course. I think this advice may not be well flushed out though. For example, if the firewalls are running on the same appliance and have poor isolation/misconfigure settings, an escape in one can bypass the entire set of firewalls (which is maybe what you’re implying?). Having totally isolated firewall machines is inefficient from a capex perspective (more machines than needed for your load) and in general multiple vendors increased your opex costs too (you now need employees that are proficient with multiple vendors which ends up meaning you have teams with dedicated experts per firewall, you have to build automation for multiple vendors with non-standard APIs etc). The bigger problem though is you’ve sacrificed availability which is now divided by N times because every layer you’ve added has to be functioning perfectly or the entire system is down (and if it’s not you’ve implemented this advice incorrectly). I think not mentioning this security/availability/cost trade off is a disservice.
- dhx 5y ago
  The scenario I had in mind is Firewall A and Firewall B are in series and they're different products from two vendors. Firewall B is found to contain a vulnerability triggered by a specially crafted IPv6 TCP packet (that firewall A is happy to pass to firewall B), giving the attacker a level of control over Firewall B that allows them to then access the centralised authentication/authorisation system that would have otherwise been off limits to the attacker. The attacker communicates with firewall B using the same accepted protocol that firewall A and firewall B are configured to allow the attacker to access. Firewall B communicates to the authentication/authorisation system using the accepted protocol for doing so. Nothing suspicious appears to be going on unless you look at the patterns of traffic (throughput, duration of connections, number of connections in interval, etc) particularly to the authentication/authorisation system. I don't think in-band attacks on routers, switches, firewalls doing simple ACL checks are a risk worth spending much concern on because parsing IPv4/IPv6/TCP/UDP headers is not hard to securely implement. Riskier architecture would be intrusion detection systems that perform deep packet inspection in-band (e.g. not out-of-band via a beam splitter to a standalone IDS) where there is 1,000,000's of lines of potentially buggy code exposed to attackers. I agree with your point too that a complex network is harder to secure because you need more skill and expertise spread across more people. In such a scenario, it is easier for human mistakes to occur because of the difficulty in communicating, and difficulty in seeing the broader implication of what may appear to be a simple configuration change.
  - vlovich123 5y ago
    In your scenario though, if the company just used firewall B you’d have the same issue. It can’t be less secure. I’m not familiar enough with how enterprise firewalls work to evaluate things from that perspective. I just know operationally, the more moving pieces, the more issues you have to deal with.
    - dhx 5y ago
      The company would be less secure because they would also be wearing the risk that independent of the vulnerability in firewall B, there is potentially also an unknown vulnerability in firewall A being exploited by a different attacker.
jlgaddis 5y ago
> Instead of the attacker having to find a vulnerability in a single product, ... ... they must now find at least one vulnerability in each of the firewalls. It's not "pwn firewall A OR B", it's "pwn firewall A AND B".
wildzzz 5y ago
Many big companies have what is basically a MITM attack on SSL. They resign everything between you and the network endpoint using an enterprise certificate. It's still encrypted but at some point it's decrypted, inspected, and then re-encrypted. It's mostly seamless but sometimes you have to get a copy of the certificate to add it to things like pip.
- nobody9999 5y ago
  > Many big companies have what is basically a MITM attack on SSL. They resign everything between you and the network endpoint using an enterprise certificate. It's still encrypted but at some point it's decrypted, inspected, and then re-encrypted. It's mostly seamless but sometimes you have to get a copy of the certificate to add it to things like pip. Yes. And this is a good thing (in an enterprise environment) for a couple of reasons (a few others too, but these should make the point): Proxying connections is inherently more secure than packet filtering/stateful inspection; As was pointed out in another comment[0], most traffic is encrypted these days. Any traffic traversing the enterprise network needs to be visible (unencrypted) to network administrators. This allows them to identify, investigate and, potentially, block suspicious traffic. Any large organization (or even smaller ones with the need and the resources) should be able to decrypt and read every packet traversing their internal network and the ingress/egress points to/from that network. That requires ubiquitous use of proxies for all connections that have an external endpoint. It's likely that some sensitive internal networks/servers/applications should do so as well. While that's very intrusive, it's not your home network, nor is it (semi-)public wifi. It's the wholly owned property (I'm not addressing "cloud-based" resources here -- that's a different long discussion) of that organization. As such, they have every right to read every packet on their network and every byte on their storage. Which is why many organizations have "guest" networks which not only has no access to internal resources, but also only allows connections to external networks and bypasses all the proxies/firewalls as well. This allows employees, contractors and others to maintain their privacy on their own devices without impacting the security of the internal network. Of course, this requires device authentication at the network layer (e.g., 802.1x) to ensure that only authorized devices are permitted access to the internal network. None of the above is particularly new or particularly controversial. [0] https://news.ycombinator.com/item?id=30574794 https://news.ycombinator.com/item?id=30574794
brobinson 5y ago
They are describing a setup like this: https://imgur.com/a/R2jbfzj https://imgur.com/a/R2jbfzj
- waffleiron 5y ago
  When we are implementing this are there diminishing returns on the luck granted by the pfSense firewall? If not, wouldn’t it be better to stack pfSense and rely on luck alone. This would also save us money as we could run them all on the same server in VMs. Sincerely, your customer
- deleted 5y ago
  [deleted]
- dhx 5y ago
  In that example network, the Chinese, US, Russian, Israeli and other firewall vendors with their products in series could all have access to the network as each of them has implemented a backdoor that triggers on a hidden condition such as HMAC(secret_backdoor_key, CONCAT(ip.source_address, tcp.source_port, ip.destination_address, tcp.destination_port)) == tcp.sequence_number. Obviously if such a scenario were to play out, a backdoor would be designed to look like a mistaken bug in the software (also making it very hard to detect) rather than the simple example. Fault tolerant architectures such as N-modular redundancy[1] (typically used in the aerospace sector) may provide a more accurate architecture up front. Making protocols much more deterministic and only using nothing-up-my-sleeve numbers would be a good line of investigation. [1] https://en.wikipedia.org/wiki/Triple_modular_redundancy https://en.wikipedia.org/wiki/Triple_modular_redundancy [2] https://en.wikipedia.org/wiki/Nothing-up-my-sleeve_number https://en.wikipedia.org/wiki/Nothing-up-my-sleeve_number
SAI_Peregrinus 5y ago
WRT traffic inspection: if anything other than handshake packets show up unencrypted, it should be able to show an alert.

by dhx 5y ago

throwbigdata 5y ago

That’s not how “in series” works.

vlovich123 5y ago

> Instead of the attacker having to find a vulnerability in a single product, they can instead find a vulnerability in one of multiple products (a much easier task) I’m not sure I understand this argument you’re making. If you have them in series then you have to find vulnerabilities for all of them before you can talk to any other system. A vulnerability to one of them only lets you punch through that one layer. In theory of course. I think this advice may not be well flushed out though. For example, if the firewalls are running on the same appliance and have poor isolation/misconfigure settings, an escape in one can bypass the entire set of firewalls (which is maybe what you’re implying?). Having totally isolated firewall machines is inefficient from a capex perspective (more machines than needed for your load) and in general multiple vendors increased your opex costs too (you now need employees that are proficient with multiple vendors which ends up meaning you have teams with dedicated experts per firewall, you have to build automation for multiple vendors with non-standard APIs etc). The bigger problem though is you’ve sacrificed availability which is now divided by N times because every layer you’ve added has to be functioning perfectly or the entire system is down (and if it’s not you’ve implemented this advice incorrectly). I think not mentioning this security/availability/cost trade off is a disservice.

dhx 5y ago
The scenario I had in mind is Firewall A and Firewall B are in series and they're different products from two vendors. Firewall B is found to contain a vulnerability triggered by a specially crafted IPv6 TCP packet (that firewall A is happy to pass to firewall B), giving the attacker a level of control over Firewall B that allows them to then access the centralised authentication/authorisation system that would have otherwise been off limits to the attacker. The attacker communicates with firewall B using the same accepted protocol that firewall A and firewall B are configured to allow the attacker to access. Firewall B communicates to the authentication/authorisation system using the accepted protocol for doing so. Nothing suspicious appears to be going on unless you look at the patterns of traffic (throughput, duration of connections, number of connections in interval, etc) particularly to the authentication/authorisation system. I don't think in-band attacks on routers, switches, firewalls doing simple ACL checks are a risk worth spending much concern on because parsing IPv4/IPv6/TCP/UDP headers is not hard to securely implement. Riskier architecture would be intrusion detection systems that perform deep packet inspection in-band (e.g. not out-of-band via a beam splitter to a standalone IDS) where there is 1,000,000's of lines of potentially buggy code exposed to attackers. I agree with your point too that a complex network is harder to secure because you need more skill and expertise spread across more people. In such a scenario, it is easier for human mistakes to occur because of the difficulty in communicating, and difficulty in seeing the broader implication of what may appear to be a simple configuration change.
- vlovich123 5y ago
  In your scenario though, if the company just used firewall B you’d have the same issue. It can’t be less secure. I’m not familiar enough with how enterprise firewalls work to evaluate things from that perspective. I just know operationally, the more moving pieces, the more issues you have to deal with.
  - dhx 5y ago
    The company would be less secure because they would also be wearing the risk that independent of the vulnerability in firewall B, there is potentially also an unknown vulnerability in firewall A being exploited by a different attacker.

jlgaddis 5y ago

> Instead of the attacker having to find a vulnerability in a single product, ... ... they must now find at least one vulnerability in each of the firewalls. It's not "pwn firewall A OR B", it's "pwn firewall A AND B".

wildzzz 5y ago

Many big companies have what is basically a MITM attack on SSL. They resign everything between you and the network endpoint using an enterprise certificate. It's still encrypted but at some point it's decrypted, inspected, and then re-encrypted. It's mostly seamless but sometimes you have to get a copy of the certificate to add it to things like pip.

nobody9999 5y ago
> Many big companies have what is basically a MITM attack on SSL. They resign everything between you and the network endpoint using an enterprise certificate. It's still encrypted but at some point it's decrypted, inspected, and then re-encrypted. It's mostly seamless but sometimes you have to get a copy of the certificate to add it to things like pip. Yes. And this is a good thing (in an enterprise environment) for a couple of reasons (a few others too, but these should make the point): Proxying connections is inherently more secure than packet filtering/stateful inspection; As was pointed out in another comment[0], most traffic is encrypted these days. Any traffic traversing the enterprise network needs to be visible (unencrypted) to network administrators. This allows them to identify, investigate and, potentially, block suspicious traffic. Any large organization (or even smaller ones with the need and the resources) should be able to decrypt and read every packet traversing their internal network and the ingress/egress points to/from that network. That requires ubiquitous use of proxies for all connections that have an external endpoint. It's likely that some sensitive internal networks/servers/applications should do so as well. While that's very intrusive, it's not your home network, nor is it (semi-)public wifi. It's the wholly owned property (I'm not addressing "cloud-based" resources here -- that's a different long discussion) of that organization. As such, they have every right to read every packet on their network and every byte on their storage. Which is why many organizations have "guest" networks which not only has no access to internal resources, but also only allows connections to external networks and bypasses all the proxies/firewalls as well. This allows employees, contractors and others to maintain their privacy on their own devices without impacting the security of the internal network. Of course, this requires device authentication at the network layer (e.g., 802.1x) to ensure that only authorized devices are permitted access to the internal network. None of the above is particularly new or particularly controversial. [0] https://news.ycombinator.com/item?id=30574794 https://news.ycombinator.com/item?id=30574794

brobinson 5y ago

They are describing a setup like this: https://imgur.com/a/R2jbfzj https://imgur.com/a/R2jbfzj

waffleiron 5y ago
When we are implementing this are there diminishing returns on the luck granted by the pfSense firewall? If not, wouldn’t it be better to stack pfSense and rely on luck alone. This would also save us money as we could run them all on the same server in VMs. Sincerely, your customer
deleted 5y ago
[deleted]
dhx 5y ago
In that example network, the Chinese, US, Russian, Israeli and other firewall vendors with their products in series could all have access to the network as each of them has implemented a backdoor that triggers on a hidden condition such as HMAC(secret_backdoor_key, CONCAT(ip.source_address, tcp.source_port, ip.destination_address, tcp.destination_port)) == tcp.sequence_number. Obviously if such a scenario were to play out, a backdoor would be designed to look like a mistaken bug in the software (also making it very hard to detect) rather than the simple example. Fault tolerant architectures such as N-modular redundancy[1] (typically used in the aerospace sector) may provide a more accurate architecture up front. Making protocols much more deterministic and only using nothing-up-my-sleeve numbers would be a good line of investigation. [1] https://en.wikipedia.org/wiki/Triple_modular_redundancy https://en.wikipedia.org/wiki/Triple_modular_redundancy [2] https://en.wikipedia.org/wiki/Nothing-up-my-sleeve_number https://en.wikipedia.org/wiki/Nothing-up-my-sleeve_number

SAI_Peregrinus 5y ago

WRT traffic inspection: if anything other than handshake packets show up unencrypted, it should be able to show an alert.