3 ms·

This article is all http daemon related which is a very small subset of server config... I don't want to devolve into all my own tips and then have arguments a

This article is all http daemon related which is a very small subset of server config... I don't want to devolve into all my own tips and then have arguments about fail2ban, but I have one tweak to the TRACE/TRACK note, really there are some silly things people can do with any requests other than POST/GET (also silly things people can do with those, but that's primarily App security). I disallow OPTIONS, HEAD, TRACE... everything, with this sort of an Apache config: <LimitExcept POST GET> Require valid-user </LimitExcept> The biggest place this can cause issues is with load balancers using a HEAD command to check the existence of a server.

apendleton 14y ago
This is probably over-broad. HEAD is also used by browsers to determine whether or not to re-request cached content; disabling it will still allow pages to load properly, but will waste bandwidth and slow down page loads as browsers re-request unchanged content. OPTIONS is necessary if you want to offer APIs that support CORS: http://en.wikipedia.org/wiki/Cross-origin_resource_sharing http://en.wikipedia.org/wiki/Cross-origin_resource_sharing
- ntoshev 14y ago
  Why would a browser use HEAD instead of conditional GET to re-request cached content? It would require one more round-trip if a refresh is actually required.
  - vsync 14y ago
    The best is to use GET with an If-Modified-Since header: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.25 http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14... This acts like HEAD if the resource has not been modified and like GET if it has. In practice this seems to be the go-to technique for browsers as well.
  - apendleton 14y ago
    I've definitely observed both behaviors in Firefox, and to be honest, I'm not entirely sure why. I'll see what I can dig up...
    - mbreese 14y ago
      HEAD just returns the headers, not the content. So, it is a faster way to see if content has been updated. If it has, then the client can request the content with a GET.
      - apendleton 14y ago
        Right, the question was why not use a conditional get, which either returns content if the thing has been modified, or a 304 Not Modified and no content if it hasn't -- this accomplishes the same thing as HEAD with only one request. Above, though, I wasn't sure why one or the other wasn't always used... I looked it up, though. It looks like you have to have an ETag to do a conditional GET, so browsers use HEAD if the original response (the cached one) didn't include one.
        vsync 14y ago
        > I looked it up, though. It looks like you have to have an ETag to do a conditional GET Nope, not the case. May I ask where you read that? It differs from my reading of RFC2616 and my experience. If the server gave you an ETag though you're supposed to include it in future cache-conditional requests. Section 13.3.4 (http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec13.3.4 http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec13...) describes the interaction between ETags and modification dates.
        apendleton 14y ago
        A blog post that's probably not worth linking to, since apparently it was wrong. Am I correctly inferring that you need at least a Last-Modified in the response to do a cache-conditional request, though?
- brokentone 14y ago
  It indeed is broad, as I said it is what I do. I've found that some of the URL security that people setup only apply to GET/POST, they're unaware of these other methods (TRACK? WTF?). As such, it's nice to have these off by default, and turn them on when needed. I saw a good note some time ago from people essentially stat'ing files they shouldn't have been able to see at all via OPTIONS or HEAD because they were secured against GET and POST only. There is a nice discussion going below on whether you need these for AJAX, conditional gets, etc. If you need this enabled for AJAX, consider enabling it for the URL or directory where your AJAX interactions occur. I run a large wordpress site and haven't had issues with any of this.
- twic 14y ago
  I don't think browsers do use HEAD to determine whether or not to re-request cached content; they make a conditional GET, ie a GET with an If-Modified-Since header (or one of its relatives). Are there some situations where browsers use HEAD instead of conditional GET? Or is my understanding wrong? Still, HEAD seems like it should generally be fairly safe, so not something i'd leap to block.
tptacek 14y ago
Why are you disabling OPTIONS and HEAD?
- euroclydon 14y ago
  Seriously. I've used AJAX libraries that won't POST data if the OPTIONS call fails.
  - jermy 14y ago
    Won't that cause issues when going through a typical paranoid corporate web proxy?
protomyth 14y ago
Why disable HEAD. HEAD has many great uses including figuring out what the web server is picking as a Content-Type.

apendleton 14y ago

This is probably over-broad. HEAD is also used by browsers to determine whether or not to re-request cached content; disabling it will still allow pages to load properly, but will waste bandwidth and slow down page loads as browsers re-request unchanged content. OPTIONS is necessary if you want to offer APIs that support CORS: http://en.wikipedia.org/wiki/Cross-origin_resource_sharing http://en.wikipedia.org/wiki/Cross-origin_resource_sharing

ntoshev 14y ago
Why would a browser use HEAD instead of conditional GET to re-request cached content? It would require one more round-trip if a refresh is actually required.
- vsync 14y ago
  The best is to use GET with an If-Modified-Since header: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.25 http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14... This acts like HEAD if the resource has not been modified and like GET if it has. In practice this seems to be the go-to technique for browsers as well.
- apendleton 14y ago
  I've definitely observed both behaviors in Firefox, and to be honest, I'm not entirely sure why. I'll see what I can dig up...
  - mbreese 14y ago
    HEAD just returns the headers, not the content. So, it is a faster way to see if content has been updated. If it has, then the client can request the content with a GET.
    - apendleton 14y ago
      Right, the question was why not use a conditional get, which either returns content if the thing has been modified, or a 304 Not Modified and no content if it hasn't -- this accomplishes the same thing as HEAD with only one request. Above, though, I wasn't sure why one or the other wasn't always used... I looked it up, though. It looks like you have to have an ETag to do a conditional GET, so browsers use HEAD if the original response (the cached one) didn't include one.
      - vsync 14y ago
        > I looked it up, though. It looks like you have to have an ETag to do a conditional GET Nope, not the case. May I ask where you read that? It differs from my reading of RFC2616 and my experience. If the server gave you an ETag though you're supposed to include it in future cache-conditional requests. Section 13.3.4 (http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec13.3.4 http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec13...) describes the interaction between ETags and modification dates.
        apendleton 14y ago
        A blog post that's probably not worth linking to, since apparently it was wrong. Am I correctly inferring that you need at least a Last-Modified in the response to do a cache-conditional request, though?
brokentone 14y ago
It indeed is broad, as I said it is what I do. I've found that some of the URL security that people setup only apply to GET/POST, they're unaware of these other methods (TRACK? WTF?). As such, it's nice to have these off by default, and turn them on when needed. I saw a good note some time ago from people essentially stat'ing files they shouldn't have been able to see at all via OPTIONS or HEAD because they were secured against GET and POST only. There is a nice discussion going below on whether you need these for AJAX, conditional gets, etc. If you need this enabled for AJAX, consider enabling it for the URL or directory where your AJAX interactions occur. I run a large wordpress site and haven't had issues with any of this.
twic 14y ago
I don't think browsers do use HEAD to determine whether or not to re-request cached content; they make a conditional GET, ie a GET with an If-Modified-Since header (or one of its relatives). Are there some situations where browsers use HEAD instead of conditional GET? Or is my understanding wrong? Still, HEAD seems like it should generally be fairly safe, so not something i'd leap to block.

tptacek 14y ago

Why are you disabling OPTIONS and HEAD?

euroclydon 14y ago
Seriously. I've used AJAX libraries that won't POST data if the OPTIONS call fails.
- jermy 14y ago
  Won't that cause issues when going through a typical paranoid corporate web proxy?

protomyth 14y ago

Why disable HEAD. HEAD has many great uses including figuring out what the web server is picking as a Content-Type.