4 ms·

I was scraping an HTTP/2 site the other day, and discovered that Python requests library does not support it :S HTTPX and Hyper didn't work either (probably du

I was scraping an HTTP/2 site the other day, and discovered that Python requests library does not support it :S HTTPX and Hyper didn't work either (probably due to ALPN?) Had to call Curl from Python to make it work

tyingq 5y ago
The bundled urllib doesn't even support HTTP/1.1 properly. It sends "Connection: close" and uses a connection-per-request model. I suppose since it's hard to scrape anything without a headless browser these days, the http[s] scraping libraries are stagnated.
- hackcasual 5y ago
  The flip side to that is headless browsers have gotten a lot better