7 ms·
Requests: HTTP for Humans
- sayrer 15y agoThis library seems like a good idea. I hope it is or will be based on Joe Gregorio's httplib2, so it gets close to correct. Also, I can't believe it includes PATCH support. Surprising, but good, to see that out in the wild.
- StavrosK 15y agoWhy not just include arbitrary verbs? The rest of the protocol doesn't change at all...
- izak30 15y agoThere is support for arbitrary verbs, just more direct API support for the common ones. here's the api for a generic request, get is right under it: https://github.com/kennethreitz/requests/blob/develop/requests/api.py#L21 https://github.com/kennethreitz/requests/blob/develop/reques...
- sdfjkl 15y agoThis was much needed.
- rubergly 15y agoI'm almost frustrated to see this. I just learned to use urllib2, and found that it took a great deal of effort to get any reasonable behavior. I wish I had known about this before. It really looks great and I will try to use it in any future work.
- kenneth_reitz 15y agoYou could be a contributor then ;)
- abecedarius 15y agoFrom the title I thought this would be some kind of mechanical-Turk webserver.
- dickeytk 15y agoWow I was just about to start going through the pain of using urllib2 for the millionth time right before I read this.
- waterside81 15y agoAwesome stuff. How many hours I've wasted futzing around with urllib2 and creating my own Request handlers and all that other nonsense. This lends itself nicely to working with 3rd party APIs (Facebook, Twitter) in particular.
- espeed 15y agoI used httplib2 (http://code.google.com/p/httplib2/ http://code.google.com/p/httplib2/) for the REST module in Bulbflow -- httplib2 is by Joe Gregorio of Google, and it's not bad.
- RyanMcGreal 15y agohttplib2 has a very nice API, but be warned that it's not thread safe.
- notmyname 15y agohow does this compare with webob (http://docs.webob.org/en/latest/index.html http://docs.webob.org/en/latest/index.html)?
- ericflo 15y agoIt's completely different.
- StavrosK 15y agoIt doesn't. Webob is a server framework, requests is a client.
- ianb 15y agoPeople will rightly tell you that it does not. But I must admit that this did get me to thinking about a few small changes to WebOb along with the integration of WSGIProxy, at which point there would be some real similarities.
- dshah 15y agoHave been using this for a little while now. Totally awesome. The simple example on the home page should have one more line of code that shows that you use r.content to get to the content that came back from the request. That would seem to be the most common use case.
- beaumartinez 15y agoSo that's why I couldn't get it to work—have a look at this[1]. Thanks for teaching me what the documentation failed to—it's neither in the front page example, or in the API documentation for Response. [1] https://github.com/kennethreitz/requests/issues/105 https://github.com/kennethreitz/requests/issues/105
- kenneth_reitz 15y agoFixing now :)
- danielpal 15y agoFascinating, I am really glad that "we" the new wave of programmers are not only conserned about code quality but are also deeply concerned with "usability". Everyday, as I go back and forth using old libs vs new libs, its obvious that the quality of old libs is pretty damn good but they have crappy "usability". New libs have a bit more bugs, being that they are also "newer", but they are a pleasure to use.
- maxogden 15y agoequivalent in node: http://github.com/mikeal/request http://github.com/mikeal/request equivalent in jquery: https://github.com/iriscouch/request_jquery https://github.com/iriscouch/request_jquery
- losvedir 15y agoHow do those handle SSL and user authentication? I skimmed your links but didn't see anything. Python's urllib2 isn't that bad for the simple stuff shown in your links, but it's a mess when you have to try to log in somewhere. Seeing in the example that logging in is as straightforward as requests.get('https://api.github.com', auth=('user', 'pass')) just blew me away.
- EiZei 15y agoYeah the authentication parts of urllib2 remind me of the horrible 20-letter classes glued together in Java, just the thing I wanted to escape from by using Python. Seeing that I'm trying to make this script interface with a website that uses both cookies and http auth this library seems like a godsend.
- izak30 15y agoFor those interested in getting to know requests vs urllib2, I gave a talk @ PyOhio, covered some simple examples in each (and why requests is super great) http://python.mirocommunity.org/video/4388/pyohio-2011-consuming-web-apis http://python.mirocommunity.org/video/4388/pyohio-2011-consu...
- stock_toaster 15y agoThis looks neat. I have been using restkit[1] for a little while now, but I am certainly going to check this out. [1]: http://benoitc.github.com/restkit/ http://benoitc.github.com/restkit/
- jokull 15y ago+1 for restkit. It has a Pythonic interface, shortcuts and completely replaces urllib, urllib2 and httplib all of which I've had some beef with before. Love restkit!
- smiley325 15y agoI've noticed that the layout of the site is uncannily similar to flask's documentation (http://flask.pocoo.org/docs/ http://flask.pocoo.org/docs/), down to the "Fork me on Github" in the top right corner. Is there some sort of software producing this kind of documentation?
- paxswill 15y agoI don't have much more info than this, but it looks like they're both generated with Sphinx (http://sphinx.pocoo.org/ http://sphinx.pocoo.org/), going by the footer from flask, and the common source (both have `<div class="sphinxsidebar">`).
- jmreardon 15y agoIt's pulling in a copy of the css file the flask site uses.
- akavlie 15y agoThe "Fork me on Github" is pretty common on a lot of sites (Github made those graphics), but that aside I agree -- I was thinking it was another Pocoo project until I saw the author. Looks like he just co-opted the theme (Pocoo might have it available somewhere, but I couldn't find it). Nothing wrong with that, it's a great theme.
- kenneth_reitz 15y agoMy modification of the Flask theme is available here: https://github.com/kennethreitz/kr-sphinx-themes The original is here: https://github.com/mitsuhiko/flask-sphinx-themes
- forsaken 15y agoYes. They're both created using Sphinx: http://sphinx.pocoo.org/ http://sphinx.pocoo.org/ Requests is hosted on a site called Read the Docs, which does documentation hosting for Sphinx projects: http://readthedocs.org/ http://readthedocs.org/ Sphinx has become the defacto documentation tool for Python projects, and a lot of other projets as well. Notably a number of PHP projects, and Varnish the web cache use it also.
- watmough 15y agoI don't know much Python, but I'd likely be correct in assuming there are quite a few proven http client libraries in existence. This looks like another reason to believe that if you build a better solution, even when the space is well-covered, that you can still win adoption, so maybe, a lot of people can stop worrying about being second.
- perfunctory 15y ago> so maybe, a lot of people can stop worrying about being second. Indeed.
- SoftwareMaven 15y agoBeing second can often be beneficial. You get to learn from the mistakes of the first without being locked into those mistakes.
- marcamillion 15y agoNot to start a language pissing war...but that syntax looks awfully like Ruby. That means it's pretty elegant and nice :)
- bialecki 15y agoQuick question on the code. I'm looking at this commit: https://github.com/kennethreitz/requests/commit/773630b0109c7671930b7ab38bb71c22ac8a94c5#L0R16 https://github.com/kennethreitz/requests/commit/773630b0109c... and wondering what "from .status_codes import codes" does? Specifically the fact that it's not "status_codes" and has a dot prepended. Is that something special in Python (guessing no) or some convention that is used? Just curious.
- bialecki 15y agoJust needed to look at the docs: http://docs.python.org/reference/simple_stmts.html#the-import-statement http://docs.python.org/reference/simple_stmts.html#the-impor.... The relevant bit: "When specifying what module to import you do not have to specify the absolute name of the module. When a module or package is contained within another package it is possible to make a relative import within the same top package without having to mention the package name. By using leading dots in the specified module or package after from you can specify how high to traverse up the current package hierarchy without specifying exact names. One leading dot means the current package where the module making the import exists. Two dots means up one package level. Three dots is up two levels, etc. So if you execute from . import mod from a module in the pkg package then you will end up importing pkg.mod. If you execute from ..subpkg2 import mod from within pkg.subpkg1 you will import pkg.subpkg2.mod. The specification for relative imports is contained within PEP 328." Cool stuff.
- dheerosaur 15y agoReally cool stuff indeed. Thank you.
- pangram 15y agoIt's a relative import -- where "relative" here is like relative vs. absolute paths. See: http://docs.python.org/tutorial/modules.html http://docs.python.org/tutorial/modules.html
- anoopks 15y agoDoes anyone know of such an API in Java - HTTP for Humans in Java?
- ollysb 15y agoRestlet is a good attempt at writing an api that matches the http spec well. You can find docs for a client at http://wiki.restlet.org/docs_2.1/13-restlet/21-restlet/318-restlet/320-restlet.html http://wiki.restlet.org/docs_2.1/13-restlet/21-restlet/318-r.... It's still more verbose than it might be, would be a good starting point for writing a decent api though.
- nl 15y agoRestlet is a server framework. I think the parent poster is looking for a client API that is nicer to use than java.net.Url and/or the various versions of HTTPClient
- ollysb 15y agoFrom the restlet home page "Do you want to blend your web services, web sites and web clients into unified web applications exposing and consuming RESTful web APIs?" The link I provided was to an example of using restlet on the client side. I would agree that the api is not as human friendly as it could be but it does describe interactions in a restful which is why I suggested it would be a good starting point for writing a friendly api.
- deno 15y agoJetty's HTTP client is pretty decent. http://wiki.eclipse.org/Jetty/Tutorial/HttpClient http://wiki.eclipse.org/Jetty/Tutorial/HttpClient
- perfunctory 15y agoThis seems to be a fantastic project. I wish more libraries were designed this way. Library's usability is just as important as performance or feature completeness.
- alexatkeplar 15y agoUsing this with DictShield (https://github.com/j2labs/dictshield https://github.com/j2labs/dictshield) or Colander (https://docs.pylonsproject.org/projects/colander/dev/ https://docs.pylonsproject.org/projects/colander/dev/) would be very cool methinks
- pestaa 15y agoQuickly judging by the look, I thought "hey, Armin Ronacher just released another library!", but this one seems as well thought out as if it was designed by Armin himself.
- ludwigvan 15y agoAnother useful Python library by Kenneth Reitz is tablib, "Python Module for Tabular Datasets in XLS, CSV, JSON, YAML, &c. ". See https://github.com/kennethreitz/tablib https://github.com/kennethreitz/tablib
- ez77 15y agoThat looks extremely useful. Thanks! Project webpage: http://tablib.org http://tablib.org
- tsycho 15y agoDoes any one know of a good Ruby equivalent of such a library?
- kennystone 15y agohttps://github.com/archiloque/rest-client https://github.com/archiloque/rest-client
- st3fan 15y agoThis is a great library. I've used it in a couple of projects now and it is great. This is the kind of stuff that I would love to see in the Python standard library.
- serichsen 15y agoBy the way, Common Lisp has Drakma for this: http://weitz.de/drakma/ http://weitz.de/drakma/
- joeshaw 15y agoA coworker introduced me to Requests a couple of weeks ago. It makes life a whole lot easier. It does highlight my biggest gripe about Python, though. I love the language, but the class library is hit-and-miss, and anything to do with HTTP is a pretty big miss. httplib, urllib, urllib2, cookielib -- yuck. I like that Requests has a nice API, but it's still built on top of pretty bad class libs underneath. (And the Requests API does expose some warts, like having to use cookielib's CookieJars.) I'd really love to see someone's take on a clean break from the standard modules with a simple, maleable API.
- jokull 15y agoYou are describing restkit. Check it out. Goes past the broken standard lib stuff and straight to socket. Very pythonic.
- joeshaw 15y agolooks interesting! thanks for the pointer. (Here's the URL for the project: http://benoitc.github.com/restkit/ http://benoitc.github.com/restkit/)
- kevingessner 15y agoIf you're looking for a "clean break", Google's httplib2 [1] might be what you want. It's somewhat lower-level than Requests, but adds a bunch of advanced stuff like caching support and keep-alives. As you might guess from the name, httplib2 is more like httplib, while Requests appears to be more like urllib2. 1: http://code.google.com/p/httplib2/ http://code.google.com/p/httplib2/
- kenneth_reitz 15y agoIt should be noted that httpbib2 is not a Google project. A lot of people prefer it though.
- acdha 15y agohttplib2 is part of why you should use requests: it's far more respectable as a client but not as well documented and it still takes way too much code for basic operations. I appreciate what httplib2 is trying to do, that there's a ton of hard low-level annoyances in building a modern HTTP client, but really, just use requests instead. Kenneth Reitz is very motivated and he gets the degree to which simple things should be simple whereas httplib2 feels more like an academic exercise than something people should use to build production systems[1]. Disclosure: I'm listed in the requests AUTHORS file but can claim credit for, oh, about 0.0001% of the awesomeness. 1. http://code.google.com/p/httplib2/issues/detail?id=96 http://code.google.com/p/httplib2/issues/detail?id=96 is a good example: an annoying bug which affect many people, there was a fix available for months, which worked great when I applied it in a fork and pounded a couple TB of data through it, but it took over a year to make it into trunk and even longer to make it onto PyPI where any other project which required "httplib2" would get the working version.
- simonw 15y agoI wish someone would write an equivalent of this for Python's myriad of horrible process management functions. popen, popen2, popen3 etc were a nightmare, and the newer subprocess module still doesn't have an API that I'd call memorable or obvious.
- kenneth_reitz 15y agoI am planning on this actually! HTTP lends itself well to running a process. It will likely be very call+result oriented at first (no interaction w/ a running process)
- llimllib 15y agoFor such a new module, the API for subprocess is remarkably bad. I have to look it up absolutely every time.
- pault 15y agoCheck out psutil (http://code.google.com/p/psutil/ http://code.google.com/p/psutil/). I discovered it the other day when I needed to get a list of open ports used by an ssh client running in a subprocess, and it was as easy as this: sshc = psutil.Process(pid) sshc.get_connections() It allows you to upconvert process ids, and it has its own wrapper of Popen that returns Process objects. It also seems to have well thought out windows compatibility, if that matters to you. :D <- My face when I found it.
- eddieplan9 15y agoThe example written in urllib2 instantly reminds me of Java and the classy query joke [http://ejohn.org/blog/classy-query/ http://ejohn.org/blog/classy-query/]. Just a thought: are most Java-related APIs over-engineered, and does Java have a bad influence on many API design work?