3 ms·
Routers treat HTTP and HTTPS traffic differently
- imtotallygay 17y agoFalse. Lies.
- wmf 17y agoThis is setting off alarms in my head. He's talking about path MTU discovery, but for some reason he doesn't actually use that term. http://en.wikipedia.org/wiki/Path_MTU_discovery http://en.wikipedia.org/wiki/Path_MTU_discovery Also, I've never heard of a system that sometimes uses PMTUD and sometimes doesn't. I can imagine that those SSL offload boxes may be doing weird stuff; maybe that's where the author saw this problem.
- mmaunder 17y agoThanks wmf I should have included the reference and have added it. There's a lot more that I don't cover e.g. PMTUD black hole routers, etc. But the article is targeted at an intermediate level audience, so leaving those out for now.
- axod 17y agoI just tried tcpdump of http vs https, and both had the flag unset :/ (As I'd expect) Would have been nice if the OP had included some example logs showing the issue.
- mmaunder 17y agoHi axod, on request from a commenter on my blog I've described how I captured packets and analyzed. I'm interested that you saw the DF flag for non-HTTP traffic. Can you tell me which OS and client or browser you used? I used Ubuntu 9.04 and curl, but have seen the same df/non-df behavior with a variety of other clients on the same OS. Mark.
- axod 17y agoOK perhaps I'm confused... Using firefox, on OSX, tcpdump. HTTP request - flag set HTTP response - flag not set HTTPS request - flag set HTTPS response - flag not set
- mmaunder 17y agoHi axod, I retested on OSX and Linux and same result. (I'm running Snow Leopard) I tested secure connection to https://help.ubuntu.com/ https://help.ubuntu.com/ and the data packets have the DF flag on both OS's. I tested insecure to www.ebay.com and the data packets have no DF flag. Make sure you ignore the first few packets that are setting up the connection because those will probably have the DF flag set both ways. Only check the packets further down about halfway through the connection - you can tell the data packets by their size. Mark.
- n-named 17y agoPlease don't sign your messages, this is discussed in the site guidelines: http://ycombinator.com/newsguidelines.html http://ycombinator.com/newsguidelines.html """Please don't sign comments, especially with your url. They're already signed with your username. If other users want to learn more about you, they can click on it to see your profile."""
- blasdel 17y agoYou don't need to greet the user you're replying to, and you sure as hell shouldn't sign your comments.
- ComputerGuru 17y agoYeah, but you don't need to be so harsh about it..
- aminuit 17y agoThis article is based on a false premise. Because HTTPS clients and servers set the DF or Do Not Fragment IP flag on packets and regular HTTP clients and servers do not. There is nothing anywhere in any standard that requires DF to be set for HTTPS traffic. There are certain links on the Internet that will set DF on your traffic in transit because it is more expensive to forward and even more expensive to do deep inspection on fragmented IP packets, but there is no reason this would be done for HTTPS and not for HTTP. EDIT: For completeness: links to RFC 2246 for TLS 1.0 and one of the draft specs for SSL 3.0 (it was never an official standard). There is nothing about setting the DF bit in the IP header in either document. TLS 1.0: http://www.ietf.org/rfc/rfc2246.txt http://www.ietf.org/rfc/rfc2246.txt SSL 3.0: http://tools.ietf.org/html/draft-ietf-tls-ssl-version3-00 http://tools.ietf.org/html/draft-ietf-tls-ssl-version3-00
- mmaunder 17y agoHere's how you set dont-fragment on a socket: http://manpages.ubuntu.com/manpages/jaunty/man7/ip.7.html http://manpages.ubuntu.com/manpages/jaunty/man7/ip.7.html See: IP_MTU_DISCOVER This premise isn't taken from an RFC. It's based on my own real-world observations of what is crossing the wire. Grab wireshark, fire up tcpdump and form your own opinion - and please do share what you find with us. Mark.
- aminuit 17y agoOn Linux (at least recent kernels), there are a number of variables that can determine whether or not DF is set on outgoing packets. I ran a few tests with (Ubuntu Hardy) firefox and it seems largely to vary based on the value of /proc/sys/net/ipv4/ip_no_pmtu_disc. If 1, no DF. If 0, DF set. But it has this behavior regardless of whether I'm using HTTPS or HTTP. I understand your objection: standards do not often reflect the real world. But the title of your post is misleading. Saying that routers treat https differently than http is just incorrect. When I read it, I thought you were trying to imply that routers were doing some sort of deep inspection, or at least looking at the TCP port numbers. What is true is that you can't expect PMTU discovery to work reliably over the Internet (another case of standards not reflecting the real world). I work at an ISP and it's a constant source of frustration. In general though, it's something that needs to be fixed on the client side (or at least on the client's ISP side), rather than on the server side.
- noonespecial 17y agoTinkering with your network cards MTU just to try to access a web site is probably a bad idea in most cases!