4 ms·

Going down a list of top websites, these URLs respond with HTTP 200 (possibly after redirections) when sent an ordinary HTTP/1.1 GET request with 0D0A line endi

Going down a list of top websites, these URLs respond with HTTP 200 (possibly after redirections) when sent an ordinary HTTP/1.1 GET request with 0D0A line endings, but respond with HTTP 400 when sent the exact same request with 0A line endings: https://br.pinterest.com/ https://www.pinterest.co.uk/ https://apps.apple.com/ https://support.apple.com/ https://podcasts.apple.com/ https://music.apple.com/ https://geo.itunes.apple.com/ https://ncbi.nlm.nih.gov/ https://www.salesforce.com/ https://www.purdue.edu/ https://www.playstation.com/ https://llvm.org/ https://www.iana.org/ https://www.gnu.org/ https://epa.gov/ https://justice.gov/ https://www.brendangregg.com/ http://heise.de/ https://www.post.ch/ http://hhs.gov/ https://oreilly.com/ https://www.thinkgeek.com/ https://www.constantcontact.com/ https://sciencemag.org/ https://nps.gov/ https://www.cs.mun.ca/ https://www.wipo.int/ https://www.unicode.org/ https://economictimes.indiatimes.com/ https://science.org/ https://icann.org/ https://caniuse.com/ https://w3techs.com/ https://chrisharrison.net/ https://www.universal-music.co.jp/ https://digiland.libero.it/ https://webaim.org/ https://webmd.com/ This URL responds with HTTP 505 on an 0A request: https://ed.ted.com/ These URLs don't respond on an 0A request: https://quora.com/ https://www.nist.gov/ Most of these seem pretty major to me. There are other sites that are public but responded with an HTTP 403, probably because they didn't like the VPN or HTTP client I used for this test. (Also, www.apple.com is tolerant of 0A line endings, even though its other subdomains aren't, which is weird.)

tptacek 2y ago
You sure about this? www.pinterest.com, for instance, does not appear to care whether I 0d0a or just 0a.
- LegionMammal978 2y ago
  My apologies, I was using a client which kept the connection alive between the 0D0A and 0A requests, which has an effect on www.pinterest.com. Rerunning the test with separate connections for 0D0A and 0A requests, www.pinterest.com and phys.org are no longer affected (I've removed the two from the list), but all other URLs are still affected.
  - tptacek 2y ago
    I picked one at random --- hhs.gov --- and it too appears to work? For what it's worth: I'm testing by piping the bytes for a bare-newline HTTP request directly into netcat.
    - LegionMammal978 2y ago
      Make sure you're contacting hhs.gov and not www.hhs.gov, the www. subdomain reacts differently. $ printf 'GET / HTTP/1.1\r\nHost: hhs.gov\r\n\r\n' | nc hhs.gov 80 HTTP/1.1 302 Found Date: Mon, 14 Oct 2024 01:38:29 GMT Server: Apache Location: http://www.hhs.gov/web/508// Content-Length: 212 Content-Type: text/html; charset=iso-8859-1 <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>302 Found</title> </head><body> <h1>Found</h1> <p>The document has moved <a href="http://www.hhs.gov/web/508//">here</a>.</p> </body></html> ^C $ printf 'GET / HTTP/1.1\nHost: hhs.gov\n\n' | nc hhs.gov 80 HTTP/1.1 400 Bad Request Date: Mon, 14 Oct 2024 01:38:40 GMT Server: Apache Content-Length: 226 Connection: close Content-Type: text/html; charset=iso-8859-1 <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>400 Bad Request</title> </head><body> <h1>Bad Request</h1> <p>Your browser sent a request that this server could not understand.<br /> </p> </body></html>
      - tptacek 2y ago
        Ahh, that was it, thanks.
        shadowgovt 2y ago
        And this whole exercise is an example of why this is a non-starter proposal (at least the "change existing implementations" part). How much do we expect the domain owners to invest in changing an implementation that already works? Hint: it's a number smaller than epsilon. Google might, but their volume is so high they care about the cost of individual bytes on the wire.
        2y ago

tptacek 2y ago

You sure about this? www.pinterest.com, for instance, does not appear to care whether I 0d0a or just 0a.

LegionMammal978 2y ago
My apologies, I was using a client which kept the connection alive between the 0D0A and 0A requests, which has an effect on www.pinterest.com. Rerunning the test with separate connections for 0D0A and 0A requests, www.pinterest.com and phys.org are no longer affected (I've removed the two from the list), but all other URLs are still affected.
- tptacek 2y ago
  I picked one at random --- hhs.gov --- and it too appears to work? For what it's worth: I'm testing by piping the bytes for a bare-newline HTTP request directly into netcat.
  - LegionMammal978 2y ago
    Make sure you're contacting hhs.gov and not www.hhs.gov, the www. subdomain reacts differently. $ printf 'GET / HTTP/1.1\r\nHost: hhs.gov\r\n\r\n' | nc hhs.gov 80 HTTP/1.1 302 Found Date: Mon, 14 Oct 2024 01:38:29 GMT Server: Apache Location: http://www.hhs.gov/web/508// Content-Length: 212 Content-Type: text/html; charset=iso-8859-1 <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>302 Found</title> </head><body> <h1>Found</h1> <p>The document has moved <a href="http://www.hhs.gov/web/508//">here</a>.</p> </body></html> ^C $ printf 'GET / HTTP/1.1\nHost: hhs.gov\n\n' | nc hhs.gov 80 HTTP/1.1 400 Bad Request Date: Mon, 14 Oct 2024 01:38:40 GMT Server: Apache Content-Length: 226 Connection: close Content-Type: text/html; charset=iso-8859-1 <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>400 Bad Request</title> </head><body> <h1>Bad Request</h1> <p>Your browser sent a request that this server could not understand.<br /> </p> </body></html>
    - tptacek 2y ago
      Ahh, that was it, thanks.
      - shadowgovt 2y ago
        And this whole exercise is an example of why this is a non-starter proposal (at least the "change existing implementations" part). How much do we expect the domain owners to invest in changing an implementation that already works? Hint: it's a number smaller than epsilon. Google might, but their volume is so high they care about the cost of individual bytes on the wire.
        2y ago