4 ms·

This article is based on a false premise. Because HTTPS clients and servers set the DF or Do Not Fragment IP flag on packets and regular HTTP clients and serve

by aminuit 17y ago

This article is based on a false premise. Because HTTPS clients and servers set the DF or Do Not Fragment IP flag on packets and regular HTTP clients and servers do not. There is nothing anywhere in any standard that requires DF to be set for HTTPS traffic. There are certain links on the Internet that will set DF on your traffic in transit because it is more expensive to forward and even more expensive to do deep inspection on fragmented IP packets, but there is no reason this would be done for HTTPS and not for HTTP. EDIT: For completeness: links to RFC 2246 for TLS 1.0 and one of the draft specs for SSL 3.0 (it was never an official standard). There is nothing about setting the DF bit in the IP header in either document. TLS 1.0: http://www.ietf.org/rfc/rfc2246.txt http://www.ietf.org/rfc/rfc2246.txt SSL 3.0: http://tools.ietf.org/html/draft-ietf-tls-ssl-version3-00 http://tools.ietf.org/html/draft-ietf-tls-ssl-version3-00

mmaunder 17y ago
Here's how you set dont-fragment on a socket: http://manpages.ubuntu.com/manpages/jaunty/man7/ip.7.html http://manpages.ubuntu.com/manpages/jaunty/man7/ip.7.html See: IP_MTU_DISCOVER This premise isn't taken from an RFC. It's based on my own real-world observations of what is crossing the wire. Grab wireshark, fire up tcpdump and form your own opinion - and please do share what you find with us. Mark.
- aminuit 17y ago
  On Linux (at least recent kernels), there are a number of variables that can determine whether or not DF is set on outgoing packets. I ran a few tests with (Ubuntu Hardy) firefox and it seems largely to vary based on the value of /proc/sys/net/ipv4/ip_no_pmtu_disc. If 1, no DF. If 0, DF set. But it has this behavior regardless of whether I'm using HTTPS or HTTP. I understand your objection: standards do not often reflect the real world. But the title of your post is misleading. Saying that routers treat https differently than http is just incorrect. When I read it, I thought you were trying to imply that routers were doing some sort of deep inspection, or at least looking at the TCP port numbers. What is true is that you can't expect PMTU discovery to work reliably over the Internet (another case of standards not reflecting the real world). I work at an ISP and it's a constant source of frustration. In general though, it's something that needs to be fixed on the client side (or at least on the client's ISP side), rather than on the server side.
  - mmaunder 17y ago
    So you're saying that with ip_no_pmtu_disc set to off (the default value), your HTTPS server does not set the DF flag on packets? What I've seen is that all HTTPS servers (that I've observed) set DF on their data packets. No HTTP servers (that I've observed) set DF for their data packets. What I would like to know is why SSL servers set DF on their data packets. As you've observed there's nothing in the FAQ and I've googled the socket options and couldn't find any recommendations. I'm curious if an underlying API like openssl is simply so widely used that it's a de facto standard.
    - aminuit 17y ago
      Did you try tweaking ip_no_pmtu_disc and run your tests? I played a bit with the openssl test client/server and it seems to respect ip_no_pmtu_disc. Test goes something like this: 1) In one window run openssl s_server 2) In another window run openssl s_client -connect localhost:4433 3) In another window run tcpdump -nvi lo port 4433 Try twiddling the ip_no_pmtu_disc sysctl and see what happens.
    - aminuit 17y ago
      Also, when you say "HTTPS servers set DF," are you observing from the server or from the client? Over the Internet, it's likely that you'd see different values for DF on opposite sides of the connection. Any router can set or clear the DF bit on your packets for any reason.

by aminuit 17y ago

mmaunder 17y ago

Here's how you set dont-fragment on a socket: http://manpages.ubuntu.com/manpages/jaunty/man7/ip.7.html http://manpages.ubuntu.com/manpages/jaunty/man7/ip.7.html See: IP_MTU_DISCOVER This premise isn't taken from an RFC. It's based on my own real-world observations of what is crossing the wire. Grab wireshark, fire up tcpdump and form your own opinion - and please do share what you find with us. Mark.

aminuit 17y ago
On Linux (at least recent kernels), there are a number of variables that can determine whether or not DF is set on outgoing packets. I ran a few tests with (Ubuntu Hardy) firefox and it seems largely to vary based on the value of /proc/sys/net/ipv4/ip_no_pmtu_disc. If 1, no DF. If 0, DF set. But it has this behavior regardless of whether I'm using HTTPS or HTTP. I understand your objection: standards do not often reflect the real world. But the title of your post is misleading. Saying that routers treat https differently than http is just incorrect. When I read it, I thought you were trying to imply that routers were doing some sort of deep inspection, or at least looking at the TCP port numbers. What is true is that you can't expect PMTU discovery to work reliably over the Internet (another case of standards not reflecting the real world). I work at an ISP and it's a constant source of frustration. In general though, it's something that needs to be fixed on the client side (or at least on the client's ISP side), rather than on the server side.
- mmaunder 17y ago
  So you're saying that with ip_no_pmtu_disc set to off (the default value), your HTTPS server does not set the DF flag on packets? What I've seen is that all HTTPS servers (that I've observed) set DF on their data packets. No HTTP servers (that I've observed) set DF for their data packets. What I would like to know is why SSL servers set DF on their data packets. As you've observed there's nothing in the FAQ and I've googled the socket options and couldn't find any recommendations. I'm curious if an underlying API like openssl is simply so widely used that it's a de facto standard.
  - aminuit 17y ago
    Did you try tweaking ip_no_pmtu_disc and run your tests? I played a bit with the openssl test client/server and it seems to respect ip_no_pmtu_disc. Test goes something like this: 1) In one window run openssl s_server 2) In another window run openssl s_client -connect localhost:4433 3) In another window run tcpdump -nvi lo port 4433 Try twiddling the ip_no_pmtu_disc sysctl and see what happens.
  - aminuit 17y ago
    Also, when you say "HTTPS servers set DF," are you observing from the server or from the client? Over the Internet, it's likely that you'd see different values for DF on opposite sides of the connection. Any router can set or clear the DF bit on your packets for any reason.