4 ms·

"And it isn't like they were doing HTTP 'HEAD' requests -- no, they were doing 'GET' requests." Some httpds treat them the same -- they still send the file aft

by 10165 9y ago

"And it isn't like they were doing HTTP 'HEAD' requests -- no, they were doing 'GET' requests." Some httpds treat them the same -- they still send the file after HEAD requests instead of only the headers.

kupiakos 9y ago
Isn't that violating the standard?
jimktrains2 9y ago
1) Can you provide an example? 2) Wouldn't it still be best to do HEAD in any circumstance where you don't want the body?
- 10165 9y ago
  1) nautil.us 2) Yes. I was not implying one should do otherwise. I was just pointing out that servers that respond to GET will not always respond to HEAD as expected. Some sites treat it the same as GET. Others may not allow it. For example, Amazon responds with 405 MethodNotAllowed.
  - jimktrains2 9y ago
    http://nautil.us http://nautil.us appears to respect HEAD, and uses Apache. Do you have a specific example of it not respecting it? % curl -vX HEAD http://nautil.us http://nautil.us Warning: Setting custom HTTP method to HEAD with -X/--request may not work the Warning: way you want. Consider using -I/--head instead. * Rebuilt URL to: http://nautil.us/ http://nautil.us/ * Trying 107.20.148.228... * Connected to nautil.us (107.20.148.228) port 80 (#0) > HEAD / HTTP/1.1 > Host: nautil.us > User-Agent: curl/7.47.0 > Accept: / > < HTTP/1.1 200 OK < Access-Control-Allow-Origin: * < Cache-Control: post-check=0, pre-check=0, max-age=0 < Cache-control: no-cache="set-cookie" < Content-Type: text/html; charset=utf-8 < Date: Sun, 07 May 2017 04:55:46 GMT < Expires: Thu, 11 May 2017 00:00:00 GMT < Last-Modified: Sun, 07 May 2017 04:55:47 GMT < Pragma: no-cache < Server: Apache/2.4.25 (Amazon) PHP/5.5.38 < Set-Cookie: lbh_session=%2B67OvEeIwXsDYbsLxSZLVjlyp%2BWUj%2BOntgIOlRdx6qoOLqyx3WuVpd2ZEH074o5bxTr7IebRTJsGpVdyaw75GEir4ZwZwrmiKAojkoOkvduxZAtpg8D4SAqwNb1EB0l3eOb1gMt%2FMuYpGZsouFJtPHTXssM82%2FKFkU7Gxm%2BTAheHa%2F7VyQ%2BAysgzthDcDyd9RYvU7NXmFAwh596ZEk7TtkwzAGVcoL%2FLjImPvk5q6Xl%2BKMWQDvOkVPIc0JtuC1rWIy3DUsOas8vCM%2BWYdv9KW9lElqzk5IHS6L7kkWSNb7U%3D44229d0bd83cc6954cf8ad73bc14d08a1d039d9a; expires=Wed, 17-May-2017 04:55:46 GMT; Max-Age=864000; path=/ < Set-Cookie: lbh_session=eDfxDyIM%2BJFhuIpEml2KXA9B8Wcyc4Bo8GJfD0Xr3dNzGgh%2B2QdqgZWRhFVFBguslYrnQfnmrKorJjhwM47N969Qwx1NFLintVOKhP3ivrS5BVq4Kwos59OOpklaUifDEOH1FX9BG8%2BHGX9Fn8kb2duHS%2F1BRJFnGaEyOA1qmB7sFPhsjVPAL2%2BTYHNByRvxwnA2CqaY09uKs%2FC5ui6rnYCRYvI3Q7Z6KLL8QWVlT5rs71FQ%2BYXbdQyIHgiPR7yN8JnaHMgaz4qzETfr6heE04uLfUSKjIjQMM5v0YAEK0I%3D06738da1155f8af1a61c5d13cd8cee0513d4175c; expires=Wed, 17-May-2017 04:55:47 GMT; Max-Age=864000; path=/ < Set-Cookie: lbh_session=9jlaxMJdXhuYik9BjvgSVfG2Xp8HJBLTUNeI8HNcw52ORZC5bbei%2F22YgBTWHMmym1fSQHljSs9dwUbQE5Zgx%2FIWki3S8aakHI%2BXac30JU5eI3FFLeWORwFrsJDniM%2BKCDyhUi5i2zad8aYF%2FNnndhh4yISYk0ASjKa4%2BAnQxR3fZjqK1iw44K3Oe%2FoVc4weHIYCra6ecNnMWkFzBZkLUuJ%2F1gJN0w%2FNdjFs8DERSHLteTbg2OnqjOSEmn62fYXUb%2FW6YRQblJB0J%2BElbJ%2BKIn5v5NRXAerGcIT2O%2F6t08s%3D70ec7ec53459a33b68c9fda357cfbf634fcada85; expires=Wed, 17-May-2017 04:55:47 GMT; Max-Age=864000; path=/ < Set-Cookie: lbh_session=DUK2jE22vfFQmL5vZpV8LpqFsFD0%2F1aHV2mpi6MHNOw4oEastxJGbqL70Tlq79lpD%2F41%2Bl9P%2Bz4%2B8aNESLphAr4%2BlwkEn83jPGE2J83JazLGQJC07ndgXRL7Hf%2FsXbMnyaOwpFPGRwQ7AdLvuIfX8j0lQ7gEEoAF4NQmupcPo0PeQ41gTAf3tJbusD4ONNqkLVi3lGH1qhT%2FjXbu1mpPwYdcZyU18OU3qomqbWkx%2B1RsX8vsiHjoCADs%2FIHhZaY4rBH%2BDi6oDS8JR9vgBG5ll6jN3eTlXtvRblDHE1IMHMA%3D78fb89e5fda5a29bb58f6ab3b872d9150e7ecd9b; expires=Wed, 17-May-2017 04:55:47 GMT; Max-Age=864000; path=/ < Set-Cookie: AWSELB=E93BBFC71E4DF46DDD850E2C67B1FBE52FEAA0E103B670233CA20FC7694721647519A155E8C10ED0C96618595B97A7D45BA1E9EE061A86361B235D0E008D08712CA9113D57;PATH=/;MAX-AGE=604800 < X-Powered-By: PHP/5.5.38 < X-UA-Compatible: IE=Edge,chrome=1 < Connection: keep-alive * no chunk, no close, no size. Assume close to signal end <
    - 10165 9y ago
      There is no need for keep-alive for a single HEAD request. Why use HTTP/1.1? cat << eof |nc -vv nautil.us 80 HEAD / HTTP/1.0^M Host: nautil.us^M User-Agent: curl/7.47.0^M Accept: */*^M Connection: close^M ^M eof Anyway, it looks like they fixed the problem or I was mistaken. I will need to find another example. Meanwhile looking on stackexchange one can still see people running websites asking whether to block or "turn off" HEAD as recently as last year. If a user expects every website to respond properly to a HEAD request, then the user might be occasionally "surprised". This is because not every person running a website understands or agrees how HEAD can be useful. Sadly, GET is the only method that a user can expect to work on across all websites.

by 10165 9y ago

kupiakos 9y ago

Isn't that violating the standard?

jimktrains2 9y ago

1) Can you provide an example? 2) Wouldn't it still be best to do HEAD in any circumstance where you don't want the body?

10165 9y ago
1) nautil.us 2) Yes. I was not implying one should do otherwise. I was just pointing out that servers that respond to GET will not always respond to HEAD as expected. Some sites treat it the same as GET. Others may not allow it. For example, Amazon responds with 405 MethodNotAllowed.
- jimktrains2 9y ago
  http://nautil.us http://nautil.us appears to respect HEAD, and uses Apache. Do you have a specific example of it not respecting it? % curl -vX HEAD http://nautil.us http://nautil.us Warning: Setting custom HTTP method to HEAD with -X/--request may not work the Warning: way you want. Consider using -I/--head instead. * Rebuilt URL to: http://nautil.us/ http://nautil.us/ * Trying 107.20.148.228... * Connected to nautil.us (107.20.148.228) port 80 (#0) > HEAD / HTTP/1.1 > Host: nautil.us > User-Agent: curl/7.47.0 > Accept: / > < HTTP/1.1 200 OK < Access-Control-Allow-Origin: * < Cache-Control: post-check=0, pre-check=0, max-age=0 < Cache-control: no-cache="set-cookie" < Content-Type: text/html; charset=utf-8 < Date: Sun, 07 May 2017 04:55:46 GMT < Expires: Thu, 11 May 2017 00:00:00 GMT < Last-Modified: Sun, 07 May 2017 04:55:47 GMT < Pragma: no-cache < Server: Apache/2.4.25 (Amazon) PHP/5.5.38 < Set-Cookie: lbh_session=%2B67OvEeIwXsDYbsLxSZLVjlyp%2BWUj%2BOntgIOlRdx6qoOLqyx3WuVpd2ZEH074o5bxTr7IebRTJsGpVdyaw75GEir4ZwZwrmiKAojkoOkvduxZAtpg8D4SAqwNb1EB0l3eOb1gMt%2FMuYpGZsouFJtPHTXssM82%2FKFkU7Gxm%2BTAheHa%2F7VyQ%2BAysgzthDcDyd9RYvU7NXmFAwh596ZEk7TtkwzAGVcoL%2FLjImPvk5q6Xl%2BKMWQDvOkVPIc0JtuC1rWIy3DUsOas8vCM%2BWYdv9KW9lElqzk5IHS6L7kkWSNb7U%3D44229d0bd83cc6954cf8ad73bc14d08a1d039d9a; expires=Wed, 17-May-2017 04:55:46 GMT; Max-Age=864000; path=/ < Set-Cookie: lbh_session=eDfxDyIM%2BJFhuIpEml2KXA9B8Wcyc4Bo8GJfD0Xr3dNzGgh%2B2QdqgZWRhFVFBguslYrnQfnmrKorJjhwM47N969Qwx1NFLintVOKhP3ivrS5BVq4Kwos59OOpklaUifDEOH1FX9BG8%2BHGX9Fn8kb2duHS%2F1BRJFnGaEyOA1qmB7sFPhsjVPAL2%2BTYHNByRvxwnA2CqaY09uKs%2FC5ui6rnYCRYvI3Q7Z6KLL8QWVlT5rs71FQ%2BYXbdQyIHgiPR7yN8JnaHMgaz4qzETfr6heE04uLfUSKjIjQMM5v0YAEK0I%3D06738da1155f8af1a61c5d13cd8cee0513d4175c; expires=Wed, 17-May-2017 04:55:47 GMT; Max-Age=864000; path=/ < Set-Cookie: lbh_session=9jlaxMJdXhuYik9BjvgSVfG2Xp8HJBLTUNeI8HNcw52ORZC5bbei%2F22YgBTWHMmym1fSQHljSs9dwUbQE5Zgx%2FIWki3S8aakHI%2BXac30JU5eI3FFLeWORwFrsJDniM%2BKCDyhUi5i2zad8aYF%2FNnndhh4yISYk0ASjKa4%2BAnQxR3fZjqK1iw44K3Oe%2FoVc4weHIYCra6ecNnMWkFzBZkLUuJ%2F1gJN0w%2FNdjFs8DERSHLteTbg2OnqjOSEmn62fYXUb%2FW6YRQblJB0J%2BElbJ%2BKIn5v5NRXAerGcIT2O%2F6t08s%3D70ec7ec53459a33b68c9fda357cfbf634fcada85; expires=Wed, 17-May-2017 04:55:47 GMT; Max-Age=864000; path=/ < Set-Cookie: lbh_session=DUK2jE22vfFQmL5vZpV8LpqFsFD0%2F1aHV2mpi6MHNOw4oEastxJGbqL70Tlq79lpD%2F41%2Bl9P%2Bz4%2B8aNESLphAr4%2BlwkEn83jPGE2J83JazLGQJC07ndgXRL7Hf%2FsXbMnyaOwpFPGRwQ7AdLvuIfX8j0lQ7gEEoAF4NQmupcPo0PeQ41gTAf3tJbusD4ONNqkLVi3lGH1qhT%2FjXbu1mpPwYdcZyU18OU3qomqbWkx%2B1RsX8vsiHjoCADs%2FIHhZaY4rBH%2BDi6oDS8JR9vgBG5ll6jN3eTlXtvRblDHE1IMHMA%3D78fb89e5fda5a29bb58f6ab3b872d9150e7ecd9b; expires=Wed, 17-May-2017 04:55:47 GMT; Max-Age=864000; path=/ < Set-Cookie: AWSELB=E93BBFC71E4DF46DDD850E2C67B1FBE52FEAA0E103B670233CA20FC7694721647519A155E8C10ED0C96618595B97A7D45BA1E9EE061A86361B235D0E008D08712CA9113D57;PATH=/;MAX-AGE=604800 < X-Powered-By: PHP/5.5.38 < X-UA-Compatible: IE=Edge,chrome=1 < Connection: keep-alive * no chunk, no close, no size. Assume close to signal end <
  - 10165 9y ago
    There is no need for keep-alive for a single HEAD request. Why use HTTP/1.1? cat << eof |nc -vv nautil.us 80 HEAD / HTTP/1.0^M Host: nautil.us^M User-Agent: curl/7.47.0^M Accept: */*^M Connection: close^M ^M eof Anyway, it looks like they fixed the problem or I was mistaken. I will need to find another example. Meanwhile looking on stackexchange one can still see people running websites asking whether to block or "turn off" HEAD as recently as last year. If a user expects every website to respond properly to a HEAD request, then the user might be occasionally "surprised". This is because not every person running a website understands or agrees how HEAD can be useful. Sadly, GET is the only method that a user can expect to work on across all websites.