3 ms·

Regarding the issue of people sending requests directly to your server, bypassing Cloudflare: While you can use IP allowlisting for this, Cloudflare can also be

Regarding the issue of people sending requests directly to your server, bypassing Cloudflare: While you can use IP allowlisting for this, Cloudflare can also be configured to send requests to your server exclusively over mTLS, with a client certificate chaining up to a root CA which they publish. If I were worried about this kind of attack, I'd probably turn that feature on (and then reject requests that don't present a valid client cert); I don't fully trust IP allowlisting. (On the other hand, I can't say for sure that the performance overhead wouldn't be a problem in a DDoS scenario. But it doesn't seem super likely?)

dx034 5y ago
Or you use Cloudflare Tunnel and don't open Ports 80/443 at all.
kungfufrog 5y ago
This is a good alternative when security is the priority. Wouldn't verifying an mTLS certs validity be significantly more expensive than whitelisting IP blocks though? Especially when a server is potentially under duress? Can you cache the verification so it wouldn't have to be done each time? All legitimate questions! I don't have the answers but am curious.
- fasterthanlime 5y ago
  mTLS certainly seems like the the most expensive option here (not that expensive outside attacks, though). S-tier implementations include: firewall rules or a BPF program, or a VPN-based approach (like Cloudflare Tunnel). The way I did it is fine for small-scale attacks like that one, but a large enough attack will have you spend too much time on syscalls and waste valuable kernel resources. I'd love to read a write-up about how these different approaches perform in practice, because this is largely gut feeling / the popular wisdom that "the sooner you block, the better".
- kevin_nisbet 5y ago
  > Wouldn't verifying an mTLS certs validity be significantly more expensive than whitelisting IP blocks though? I wouldn't expect this to be all that significant if already using TLS. So in the context of only allowing cloudflare to establish connections to the service, and be 100% fronted by cloudflare, mTLS isn't much more expensive than already using TLS, which we all do every day. And my read of the blog post, is most of the expense / optimizations are in the time it takes the server to generate and serve a page. > Especially when a server is potentially under duress? Well, it's a tradeoff. The desired property is only cloudflare can make requests to the actual web server, so compared to IP whitelisting there might be some tradeoffs. I think the biggest one is maintenance, IP whitelisting requires staying ontop of any changes from cloudflare. I'm sure cloudflare is good at pre-announcing new IPs, PoPs, etc, but there is a small risk to missing this, especially if it's a set it and forget approach. Although to be fair, mTLS also has this depending on when the root is set to expire. > Can you cache the verification so it wouldn't have to be done each time? Sort of, but it depends on some support on both the client and server, so it depends on whether both sides have opted into it. The terms you're looking for to do more research are TLS session resumption, which does involve it's own security concerns. More effective at this level, is clients will re-use connections. So as long as the client/server support Connection: keep-alive, you do the mTLS once and run hundreds or thousands of queries in the single TLS verification. My 2 cents is the biggest downside of the mTLS approach is you may still leak the location of the webserver. If the web server is on the same domain or a domain known to be used by the target, some certificate scans may turn it up. IIRC server cert is exchanged before client certificate, so it's possible to leak this without presenting a client certificate. I'd have to double check the spec to be sure though. Although to be fair, this problem also exists in the specific implementation of whitelisting from the OP, as it appears to be code embedded in the web server to drop the connection if not from an approved source. So someone scanning for certs / servers could still learn about the server from the TLS connection, and do a simple flood the server out of existence type attack, or some more limited resource type attacks on number of active connections, etc. Ideally, with this type of whitelisting you would want to do it in iptables or the host firewall, so that you just blackhole the unapproved IPs and reveal nothing about the existence of the server.
  - kevin_nisbet 5y ago

dx034 5y ago

Or you use Cloudflare Tunnel and don't open Ports 80/443 at all.

kungfufrog 5y ago

This is a good alternative when security is the priority. Wouldn't verifying an mTLS certs validity be significantly more expensive than whitelisting IP blocks though? Especially when a server is potentially under duress? Can you cache the verification so it wouldn't have to be done each time? All legitimate questions! I don't have the answers but am curious.

fasterthanlime 5y ago
mTLS certainly seems like the the most expensive option here (not that expensive outside attacks, though). S-tier implementations include: firewall rules or a BPF program, or a VPN-based approach (like Cloudflare Tunnel). The way I did it is fine for small-scale attacks like that one, but a large enough attack will have you spend too much time on syscalls and waste valuable kernel resources. I'd love to read a write-up about how these different approaches perform in practice, because this is largely gut feeling / the popular wisdom that "the sooner you block, the better".
kevin_nisbet 5y ago
> Wouldn't verifying an mTLS certs validity be significantly more expensive than whitelisting IP blocks though? I wouldn't expect this to be all that significant if already using TLS. So in the context of only allowing cloudflare to establish connections to the service, and be 100% fronted by cloudflare, mTLS isn't much more expensive than already using TLS, which we all do every day. And my read of the blog post, is most of the expense / optimizations are in the time it takes the server to generate and serve a page. > Especially when a server is potentially under duress? Well, it's a tradeoff. The desired property is only cloudflare can make requests to the actual web server, so compared to IP whitelisting there might be some tradeoffs. I think the biggest one is maintenance, IP whitelisting requires staying ontop of any changes from cloudflare. I'm sure cloudflare is good at pre-announcing new IPs, PoPs, etc, but there is a small risk to missing this, especially if it's a set it and forget approach. Although to be fair, mTLS also has this depending on when the root is set to expire. > Can you cache the verification so it wouldn't have to be done each time? Sort of, but it depends on some support on both the client and server, so it depends on whether both sides have opted into it. The terms you're looking for to do more research are TLS session resumption, which does involve it's own security concerns. More effective at this level, is clients will re-use connections. So as long as the client/server support Connection: keep-alive, you do the mTLS once and run hundreds or thousands of queries in the single TLS verification. My 2 cents is the biggest downside of the mTLS approach is you may still leak the location of the webserver. If the web server is on the same domain or a domain known to be used by the target, some certificate scans may turn it up. IIRC server cert is exchanged before client certificate, so it's possible to leak this without presenting a client certificate. I'd have to double check the spec to be sure though. Although to be fair, this problem also exists in the specific implementation of whitelisting from the OP, as it appears to be code embedded in the web server to drop the connection if not from an approved source. So someone scanning for certs / servers could still learn about the server from the TLS connection, and do a simple flood the server out of existence type attack, or some more limited resource type attacks on number of active connections, etc. Ideally, with this type of whitelisting you would want to do it in iptables or the host firewall, so that you just blackhole the unapproved IPs and reveal nothing about the existence of the server.
- kevin_nisbet 5y ago