5 ms·

So I've been writing a web server recently (mostly for learning purposes of HTTP and web stuff again as I've been out of that field for over a decade), and I've

by berkut 8y ago

So I've been writing a web server recently (mostly for learning purposes of HTTP and web stuff again as I've been out of that field for over a decade), and I've discovered that Firefox seems to be the only browser I've tried which still really utilises it fully and respects the headers: Safari sort of does, but only ever once, regardless of the header params, and Chrome never does, again, regardless of what the Connection header item says in the first response from the server. https://www.chromium.org/developers/design-documents/network-stack/http-pipelining https://www.chromium.org/developers/design-documents/network... shows they removed it in Chrome due to issues, and weirdly this Firefox ticket: https://bugzilla.mozilla.org/show_bug.cgi?id=264354 https://bugzilla.mozilla.org/show_bug.cgi?id=264354 (last updated 5 years ago), seems to show they're not going to enable keep alive for HTTP 1.1, but Firefox is most definitely utilising it.

iforgotpassword 8y ago
These tickets talk about pipelining, not keep-alive. The chrome link specifically points out why they decided to disable it. Chrome and Firefox still support regular keep-alive and do reuse connections on http 1.1
- berkut 8y ago
  Fair point, however I've seen no evidence Chrome is doing keep-alive on Linux or Mac OS - it always seems to send a FIN ACK after the response from the server to close the connection. Firefox most definitely is utilising it fully, and obeys the Keep-Alive params specified, which I can't see any of the other browsers doing.
  - pvg 8y ago
    I've seen no evidence Chrome is doing keep-alive on Linux or Mac OS That sounds like a bug with your server implementation. The web would melt down if Chrome's keep-alive support didn't work.
    - berkut 8y ago
      Possibly, and that's what I assumed at first, but I don't really see how: after sending the response the socket waits for more data in the form of another request from the user agent (I've tried with and without a poll / socket timeout). Firefox does this perfectly. Chrome sends a FIN ACK in response to the response from the server, and it closes the connection its side, triggering the next recv() within the server to return 0 (or poll() to fail, depending on how I do it). But then it opens another connection after that with another request which could have been within the previous connection. I also can't see Chrome doing it on non HTTPS websites in general when monitoring through wireguard, which is why I don't believe it's an issue with my server's implementation: I haven't actually observed Chrome utilise it at all on several computers on both Linux and Mac OS.
      - pvg 8y ago
        I don't know why you're seeing what you're seeing but your testing/monitoring setup sounds like it's at a bit too low level for the thing you're trying to figure out. A few basic things you might want (or should, really, if you're implementing HTTP from scratch) to try: - Serve real content, say, a basic HTML page with a couple of external resources. - Serve same and monitor (turn on all teh logs) with apache. - Use the Chrome dev tools. In the network tab, right click on the column headers in the request list and enable the 'Connection ID' column. Keep in mind any modern browser will open concurrent connections in the base HTTP 1.1 case.
        berkut 8y ago
        Thanks, but I've done this (although used Nginx instead). The web server I've written is now serving several websites I'm hosting, some of which have thousands of images, so there's plenty of scope for connection re-use. Each connection ID is different within dev tools in Chrome - although errors have 0 - (often sequential, sometimes there are huge gaps). And this is the same with HTTP websites on the net (just tried www.briscoes.co.nz and am still seeing the same thing in Chrome 72). Using Wireshark I can see that Firefox is doing the correct thing, so I'm not sure why you think what I'm looking at is too low-level: I think it's pretty clearly something the client's doing at the TCP level to close the connection its end after its received the content length. It may well be that I'm not sending a header Chrome's looking for, but I'm not sure what it is, as in its request it's sending "Connection: keep-alive", and the response has the same, and Firefox works fine. It's almost like it's a client configuration or something, but I don't know what that would be, as I've tried on several machines.
  - fps_doug 8y ago
    Just checked with Chromium 71 on Linux against lighttpd - keepalive works just fine, seeing html, css and js requested using the same connection. I'd also suspect something's up with your server's implementation. FWIW, this is the reply header lighttpd sends: HTTP/1.1 200 OK Vary: Accept-Encoding Content-Encoding: gzip Last-Modified: Wed, 16 Sep 2015 08:50:41 GMT ETag: "713114043" Content-Type: application/javascript Accept-Ranges: bytes Content-Length: 9743 Date: Fri, 08 Mar 2019 10:34:12 GMT Server: lighttpd/1.4.35

by berkut 8y ago

iforgotpassword 8y ago

These tickets talk about pipelining, not keep-alive. The chrome link specifically points out why they decided to disable it. Chrome and Firefox still support regular keep-alive and do reuse connections on http 1.1

berkut 8y ago
Fair point, however I've seen no evidence Chrome is doing keep-alive on Linux or Mac OS - it always seems to send a FIN ACK after the response from the server to close the connection. Firefox most definitely is utilising it fully, and obeys the Keep-Alive params specified, which I can't see any of the other browsers doing.
- pvg 8y ago
  I've seen no evidence Chrome is doing keep-alive on Linux or Mac OS That sounds like a bug with your server implementation. The web would melt down if Chrome's keep-alive support didn't work.
  - berkut 8y ago
    Possibly, and that's what I assumed at first, but I don't really see how: after sending the response the socket waits for more data in the form of another request from the user agent (I've tried with and without a poll / socket timeout). Firefox does this perfectly. Chrome sends a FIN ACK in response to the response from the server, and it closes the connection its side, triggering the next recv() within the server to return 0 (or poll() to fail, depending on how I do it). But then it opens another connection after that with another request which could have been within the previous connection. I also can't see Chrome doing it on non HTTPS websites in general when monitoring through wireguard, which is why I don't believe it's an issue with my server's implementation: I haven't actually observed Chrome utilise it at all on several computers on both Linux and Mac OS.
    - pvg 8y ago
      I don't know why you're seeing what you're seeing but your testing/monitoring setup sounds like it's at a bit too low level for the thing you're trying to figure out. A few basic things you might want (or should, really, if you're implementing HTTP from scratch) to try: - Serve real content, say, a basic HTML page with a couple of external resources. - Serve same and monitor (turn on all teh logs) with apache. - Use the Chrome dev tools. In the network tab, right click on the column headers in the request list and enable the 'Connection ID' column. Keep in mind any modern browser will open concurrent connections in the base HTTP 1.1 case.
      - berkut 8y ago
        Thanks, but I've done this (although used Nginx instead). The web server I've written is now serving several websites I'm hosting, some of which have thousands of images, so there's plenty of scope for connection re-use. Each connection ID is different within dev tools in Chrome - although errors have 0 - (often sequential, sometimes there are huge gaps). And this is the same with HTTP websites on the net (just tried www.briscoes.co.nz and am still seeing the same thing in Chrome 72). Using Wireshark I can see that Firefox is doing the correct thing, so I'm not sure why you think what I'm looking at is too low-level: I think it's pretty clearly something the client's doing at the TCP level to close the connection its end after its received the content length. It may well be that I'm not sending a header Chrome's looking for, but I'm not sure what it is, as in its request it's sending "Connection: keep-alive", and the response has the same, and Firefox works fine. It's almost like it's a client configuration or something, but I don't know what that would be, as I've tried on several machines.
- fps_doug 8y ago
  Just checked with Chromium 71 on Linux against lighttpd - keepalive works just fine, seeing html, css and js requested using the same connection. I'd also suspect something's up with your server's implementation. FWIW, this is the reply header lighttpd sends: HTTP/1.1 200 OK Vary: Accept-Encoding Content-Encoding: gzip Last-Modified: Wed, 16 Sep 2015 08:50:41 GMT ETag: "713114043" Content-Type: application/javascript Accept-Ranges: bytes Content-Length: 9743 Date: Fri, 08 Mar 2019 10:34:12 GMT Server: lighttpd/1.4.35