3 ms·
:+1: for the effort to document this, and coordinating the disclosure with the vendors. This mainly talks about rate-limiting bypass/DoS, but if XFF is also use
by terom 5y ago
:+1: for the effort to document this, and coordinating the disclosure with the vendors. This mainly talks about rate-limiting bypass/DoS, but if XFF is also used for audit trail logging of IP addresses and/or IP-based access lists, then the security implications can be even more severe, with falsified audit logs and bypassed security controls.
Setting up an application server behind a reverse proxy to use the "real" client IP is unfortunately very typically just a trial-and-error based process, with very little room for this kind of nuanced security-conciousness, because the configuration and exact behavior is all so non-standardized across different implementations of reverse-proxies and application servers... Typically users will just try different configuration settings until they find a combination that seems to work, and you would actually need to dig in with curl and tshark to understand the edge cases, because the documentation of the application-specific implementation is typically just one brief sentence...
Getting XFF working correctly through a complicated HTTP stack with multiple layers of nginx/haproxy/apache proxies (yes, they have different non-overlapping feature sets), custom backends implementing custom XFF handling/forwarding, and jetty/spring backends upgraded across a major version bump that changed the implementation and configuration properties related to XFF handling was insanely difficult. And of course it broke when migrating from a F5 LB to an AWS ALB, because it behaved differently for that one edge-case for an important customer... highly recommended to just override the entire XFF header with a single value at the appropriate point in your stack, if at all possible.
If just the naive leftmost-first vs rightmost-ish-with-configurable-list-of-trusted-upstream-proxies wasn't enough, then yeah, HAProxy does the thing where it adds a new 100% standards-compliant header continuation line [1] that maybe 1% of backend application developers have ever tested with. And trying to configure HAProxy to interpret the incoming XFF headers for logging/access-control ~is~/was even more weird [2].
[1] https://github.com/haproxy/haproxy/issues/44 https://github.com/haproxy/haproxy/issues/44
[2] https://github.com/haproxy/haproxy/issues/90 https://github.com/haproxy/haproxy/issues/90
- adam-p 5y ago> Setting up an application server behind a reverse proxy to use the "real" client IP is unfortunately very typically just a trial-and-error based process This is very true, and I narrowly missed getting burned myself by thinking, "looks good to me!" after seeing my own IP. However... I take a dig in the conclusions at the implementers of security-related libraries. I don't think it's okay for them to stop at "seems to work". They should be taking the time to fully understand the problem space. > highly recommended to just override the entire XFF header with a single value at the appropriate point in your stack, if at all possible I agree, and probably should have emphasized that approach. (And maybe will, and will definitely add a note at the end.) I didn't really give any/enough attention to configuring your first proxy with a custom single-IP header. (Partly because I was writing more for people who are trying to use what's available.)