3 ms·

Understanding HTTP/2 Fingerprinting

3 points by rl1987 4y ago

1vuio0pswjnm7 4y ago
"HTTP/1.1 requires a single TCP connection per request (with possibility for connection reuse), but HTTP/2 supports request-response flow multiplexing across a single connection." I send multiple HTTP requests over a single TCP connection every day. I have been using HTTP/1.1 pipelining for the last 15 years. It is optimal for information retrieval, e.g., retrieving many web pages or files from the same website, such as sitemap XML files. Needless to say, it is also good for so-called "scraping". Most "developers" doing "web scraping" will make large numbers of TCP connections, sending one HTTP request per connection, and as a result they will draw bans for consuming server resources. Then they fiddle around with proxies. Then they complain about problems. Maybe "modern" web browsers are incapable of using HTTP/1.1 pipelining, but virtually all HTTP servers are capable. I do not use a web browser to do pipelining. "HTTP/1.1 requires a single TCP connection per request" is false. It seems to be a popular refrain from the HTTP/2 crowd (Google and other companies interested in eyeballs for advertisers and therefore graphical browsers). Bloggers have picked this up and have proceeded to regurgitate the lie. The whole point of HTTP/1.1's "keep-alive" value in the Connection header is pipelining multiple HTTP requests over s single TCP connection. HTTP/1.1 pipelining works beautifully for requesting multiple resources from the same host: information retrieval. It just does not work for Google's online advertising playground, which requires a cornucopia of resources from multiple hosts, including pure ad servers and trackers, just to "load" a single page. Hence Google and HTTP/2 proponents want to pretend HTTP/1.1 cannot be used to pipeline requests. But the truth is that only their web browsers and their idea of web pages designed for advertising cannot use HTTP/1.1 to pipeline requests. There is no such limitation in the protocol. Trying to "fingerprint" HTTP/1.1 pipelining is left as an excercise for the reader. No doubt it could be done, but it would also be easy to evade.

3 points by rl1987 4y ago

"HTTP/1.1 requires a single TCP connection per request (with possibility for connection reuse), but HTTP/2 supports request-response flow multiplexing across a single connection." I send multiple HTTP requests over a single TCP connection every day. I have been using HTTP/1.1 pipelining for the last 15 years. It is optimal for information retrieval, e.g., retrieving many web pages or files from the same website, such as sitemap XML files. Needless to say, it is also good for so-called "scraping". Most "developers" doing "web scraping" will make large numbers of TCP connections, sending one HTTP request per connection, and as a result they will draw bans for consuming server resources. Then they fiddle around with proxies. Then they complain about problems. Maybe "modern" web browsers are incapable of using HTTP/1.1 pipelining, but virtually all HTTP servers are capable. I do not use a web browser to do pipelining. "HTTP/1.1 requires a single TCP connection per request" is false. It seems to be a popular refrain from the HTTP/2 crowd (Google and other companies interested in eyeballs for advertisers and therefore graphical browsers). Bloggers have picked this up and have proceeded to regurgitate the lie. The whole point of HTTP/1.1's "keep-alive" value in the Connection header is pipelining multiple HTTP requests over s single TCP connection. HTTP/1.1 pipelining works beautifully for requesting multiple resources from the same host: information retrieval. It just does not work for Google's online advertising playground, which requires a cornucopia of resources from multiple hosts, including pure ad servers and trackers, just to "load" a single page. Hence Google and HTTP/2 proponents want to pretend HTTP/1.1 cannot be used to pipeline requests. But the truth is that only their web browsers and their idea of web pages designed for advertising cannot use HTTP/1.1 to pipeline requests. There is no such limitation in the protocol. Trying to "fingerprint" HTTP/1.1 pipelining is left as an excercise for the reader. No doubt it could be done, but it would also be easy to evade.