4 ms·

Note that the popular "requests" module depends on urllib3. This space is a little messy for python. The old urllib ships with python, but it doesn't support s

by tyingq 4y ago

Note that the popular "requests" module depends on urllib3. This space is a little messy for python. The old urllib ships with python, but it doesn't support some normal expected things like HTTP/1.1 pipelined requests. It adds a "Connection: close" header to each request.

dec0dedab0de 4y ago
I think both of them should be in the standard library at this point. The API has been fairly stable for over 5 years.
- hotfixguru 4y ago
  Web and release cycles don’t always go hand in hand, though.
- Frotag 4y ago
  stdlib is where things go to die. Backwards compatibility, limits on dependencies, release cycles, etc all make development a pain and discourage potential contributions. Or so I've read. Here's one discussion about it in the requests repo https://github.com/psf/requests/issues/2424 https://github.com/psf/requests/issues/2424
  - tyingq 4y ago
    Stdlib may not be the answer, but the current setup does make urllib3 suffer in anonymity...since for many it's hidden under the requests module.
  - dec0dedab0de 4y ago
    I read that at the time, but that was 7 years ago. Requests has been pretty much backward compatible for that whole time. They could still have active developments available in pypi for people who need the newer features, but python has been dropping minor releases about once a year, that should be plenty fast for new features in requests.
  - bodyfour 4y ago
    stdlib is also where things go to get used, though. Coding against a pile of external python modules is perfectly reasonable when you're building something "app scale" -- i.e. something that is going to be spread across many files and installed in a container or venv. However, when writing something at "script scale" I just don't want to deal with all of that. I want to write something that I can deploy as a single file and not end up dealing with missing python dependencies every time. This means I'm using the old urllib and such more often than I'd like. It's a shame that there doesn't seem to be much flow of functionality into stdlib at all any more. For my needs, I wish "requests" and "yaml" were in that set, although I'm sure other people have their own opinions on that.
kstrauser 4y ago
Does using requests buy you much on top of using urllib3 directly? I could look for myself, but it might be more fun as an open discussion topic.
- xavdid 4y ago
  Ergonomics, mostly. To make a basic POST request w/ a JSON body in urllib3, you write: import json import urllib3 http = urllib3.PoolManager() data = {'key': 'value'} encoded_data = json.dumps(data).encode('utf-8') r = http.request( 'POST', 'http://httpbin.org/post', body=encoded_data, headers={'Content-Type': 'application/json'} ) in requests, it's: import requests requests.post('http://httpbin.org/post', json={'key': 'value'})
  - progmetaldev 4y ago
    Does the variable "r" in your first example afford you anything more than what you can get from requests.post? e.g. Can you capture a return value from requests.post, and does it give you the same information to help debug that http.request gives?
    - xavdid 4y ago
      > e.g. Can you capture a return value from requests.post Yes, I just didn't in my toy example. That's the response object, which will typically include all the response info, plus reponse.request, which has request info. > and does it give you the same information to help debug that http.request gives? I'm not positive, but I'd bet it's similar.
  - SethMLarson 4y ago
    In v2.0 urllib3 it's: import urllib3 urllib3.request("POST", "https://httpbin.org/post", json={"key": "value"})
    - xavdid 4y ago
      That's fair- I was summarizing latest stable: https://urllib3.readthedocs.io/en/1.26.13/user-guide.html#json https://urllib3.readthedocs.io/en/1.26.13/user-guide.html#js...
expazl 4y ago
Relevant to that is the fact that the author of urrlib3 stopped collaborating with the author of "requests" because the latter was misusing the popularity of the library in fundraising for what seemed entirely personal goals, while advertising "new features in requests" that where in fact just urrlib3 features he planned to also expose in "requests". That and a bunch of other stuff, the blog is here: https://vorpus.org/blog/why-im-not-collaborating-with-kenneth-reitz/ https://vorpus.org/blog/why-im-not-collaborating-with-kennet...

by tyingq 4y ago

dec0dedab0de 4y ago

I think both of them should be in the standard library at this point. The API has been fairly stable for over 5 years.

hotfixguru 4y ago
Web and release cycles don’t always go hand in hand, though.
Frotag 4y ago
stdlib is where things go to die. Backwards compatibility, limits on dependencies, release cycles, etc all make development a pain and discourage potential contributions. Or so I've read. Here's one discussion about it in the requests repo https://github.com/psf/requests/issues/2424 https://github.com/psf/requests/issues/2424
- tyingq 4y ago
  Stdlib may not be the answer, but the current setup does make urllib3 suffer in anonymity...since for many it's hidden under the requests module.
- dec0dedab0de 4y ago
  I read that at the time, but that was 7 years ago. Requests has been pretty much backward compatible for that whole time. They could still have active developments available in pypi for people who need the newer features, but python has been dropping minor releases about once a year, that should be plenty fast for new features in requests.
- bodyfour 4y ago
  stdlib is also where things go to get used, though. Coding against a pile of external python modules is perfectly reasonable when you're building something "app scale" -- i.e. something that is going to be spread across many files and installed in a container or venv. However, when writing something at "script scale" I just don't want to deal with all of that. I want to write something that I can deploy as a single file and not end up dealing with missing python dependencies every time. This means I'm using the old urllib and such more often than I'd like. It's a shame that there doesn't seem to be much flow of functionality into stdlib at all any more. For my needs, I wish "requests" and "yaml" were in that set, although I'm sure other people have their own opinions on that.

kstrauser 4y ago

Does using requests buy you much on top of using urllib3 directly? I could look for myself, but it might be more fun as an open discussion topic.

xavdid 4y ago
Ergonomics, mostly. To make a basic POST request w/ a JSON body in urllib3, you write: import json import urllib3 http = urllib3.PoolManager() data = {'key': 'value'} encoded_data = json.dumps(data).encode('utf-8') r = http.request( 'POST', 'http://httpbin.org/post', body=encoded_data, headers={'Content-Type': 'application/json'} ) in requests, it's: import requests requests.post('http://httpbin.org/post', json={'key': 'value'})
- progmetaldev 4y ago
  Does the variable "r" in your first example afford you anything more than what you can get from requests.post? e.g. Can you capture a return value from requests.post, and does it give you the same information to help debug that http.request gives?
  - xavdid 4y ago
    > e.g. Can you capture a return value from requests.post Yes, I just didn't in my toy example. That's the response object, which will typically include all the response info, plus reponse.request, which has request info. > and does it give you the same information to help debug that http.request gives? I'm not positive, but I'd bet it's similar.
- SethMLarson 4y ago
  In v2.0 urllib3 it's: import urllib3 urllib3.request("POST", "https://httpbin.org/post", json={"key": "value"})
  - xavdid 4y ago
    That's fair- I was summarizing latest stable: https://urllib3.readthedocs.io/en/1.26.13/user-guide.html#json https://urllib3.readthedocs.io/en/1.26.13/user-guide.html#js...

expazl 4y ago

Relevant to that is the fact that the author of urrlib3 stopped collaborating with the author of "requests" because the latter was misusing the popularity of the library in fundraising for what seemed entirely personal goals, while advertising "new features in requests" that where in fact just urrlib3 features he planned to also expose in "requests". That and a bunch of other stuff, the blog is here: https://vorpus.org/blog/why-im-not-collaborating-with-kenneth-reitz/ https://vorpus.org/blog/why-im-not-collaborating-with-kennet...