14 ms·

Python for Humans

427 points by craigkerstiens 15y ago

nvictor 15y ago
seems the guy nailed down many issues i have had in the past :)
ovi256 15y ago
Wow those libraries are indeed great, amazing compared to the standard libs. Hope they'll be included in the standard libs one day.
- dextorious 15y ago
  +1 I'd like to see some community effort to build a collection of similar "better than the standard" libs. Which, at some point, could replace the standard libs. Or be the de facto standard, a pip install call away...
- stock_toaster 15y ago
  While python has always been 'batteries included' I think some of the batteries should not have been included. Libraries tend to move more quickly than the language and interpreter/compiler. Tying them together, while convenient, often leads to rot, clunky libraries, slow moving updates, and libraries being build to the interpreter/compiler instead of to the needs of the users. I would like to see instead a somewhat canonical (widely accepted) list of the highest quality libraries for a given set of needs, with information and pro/con/caveats listed for each, instead of them being included in the mainline trunk. I really applaud what Kenneth Reitz has been doing lately. -- a very happy user of the `requests` library
kenneth_reitz 15y ago
Audio is available here: https://github.com/kennethreitz/python-for-humans https://github.com/kennethreitz/python-for-humans
drivingmenuts 15y ago
Hangs on the first slide unless I'm doing something really wrong.
- kenneth_reitz 15y ago
  Try the arrow keys
  - Symmetry 15y ago
    Or SPACE and SHIFT-SPACE. But it also took me a bit to figure out how to navigate the slides.
  - WayneDB 15y ago
    It took me at least 30 seconds to figure that out. I clicked darn near every pixel on the page before that occurred to me :) I guess you really stick by that rule of "one way to do it", huh? I kid. Great presentation, I like your passion and I hope that your call to arms is answered.
  - drivingmenuts 15y ago
    You know, some sort of clue that the arrow keys are used for navigation would be helpful.
  - brown9-2 15y ago
    A clear sign that the one doing something wrong is the website's designer. Also, it breaks the back button.
- Heliosmaster 15y ago
  Am I weird if I say that such badly-written presentation make me much less enthusiastic about their content?
y3di 15y ago
The slides don't fit vertically on my screen, so some of the content is cut off. There's no scroll bar so initially it was difficult to figure how to see the info cut off from the bottom. Chrome's text zoom out didn't work either. I had to highlight the text and drag downwards in order to see the content. But it was annoying having to do this for every slide with a lot of content. Otherwise, these libraries seem really useful. Thanks for this.
- minikomi 15y ago
  Had the same problem but Chrome zoomed out for me.. (Chrome 16 -> [command -] on a mac.)
- cbs 15y ago
  I had the same problem too. I had no idea it was supposed to be a slide show, then once I gave my browser as much screen real estate as the site wanted I had to figure out how to navigate the damn thing. Whats so wrong with just sticking a bunch of static slides on a page one after another?
rmc 15y ago
This makes a lot of good points, but some bad ones. Esp. the "installing python" one. Just use your package manager to install all the versions you need. And for "Packaging and Dependencies", just use pip.
- engtech 15y ago
  I agree that people should use a good package manager, but the reality of the corporate env world is that you'll be trying to use the latest Python on some ridiculously old unix install without root access, and you'll never be able to get IT to install the latest for all users.
  - joshbaptiste 15y ago
    +1 for you sir, we are constantly fighting with Unix admins who refuse to install anything but the generic terribly old 2.3 - 2.4 Python versions that are bundled with their default enterprise Suse packages, which only leads to full /home issues since everyone just downloads their own versions, compiling extensions manually.
    - antoncohen 15y ago
      Or you could write the applications for the systems they will run on. Why develop in the latest Ubuntu with Python 2.7 when you know your company runs RHEL 5 or SUSE with Python 2.4? I know it was annoying how long RHEL lasted with Python 2.4, but now the latest RHEL, Debian-stable, and Ubuntu LTS all have Python 2.6, so write to Python 2.6 if your company uses those OSs, there's not excuse to write incompatible code targeted at Python 2.7. In many Linux distributions it is impossible to replace the Python version, core parts of the OS like package management will break. It is possible to install another version in parallel and use "virtualenv --python=" to use it for an application. If you are using RHEL/CentOS 5 you can install the python26 package from the EPEL repository.
- tbatterii 15y ago
  What's the package manager for windows? Mostly I agree with you, but installing python on windows is a PITA compared to *nix. PIP works until you need something that requires compiling extensions which is also a PITA on windows. IIRC easy_install can install binaries, but I could be wrong about that since I don't do windows development anymore and thus pip works for me 99% of the time. Regardless, my point is, though I think we all wish it was as simple as using your package manager and pip, it simply isn't that way for everyone. And it probably never will be, but we can always make it simpler. :)
tuananh 15y ago
The name says it all. This is much more readable compared to urllib2
skrebbel 15y ago
I like the presentation, but for me it just underlines how it's smart to stay far away from Python. It's great that it's improving, but many other languages have a much better library/API situation than Python has had for years already. Will Python catch up fast enough?
- ak217 15y ago
  It's quite the opposite, actually. Python already has an extremely good and very capable standard library, arguably better than any other language. Python also has a community with a strong sense of what kind of API design best fits with the Python philosophy (which usually boils down to readability and consistency). So there are many efforts going on to improve the standard libraries. For example, the standard library already has urllib2, which provides HTTP support much better than most other languages' standard libraries. But Requests is a rewrite of it, taking ease of use to a whole new level without compromising any of the features. There is no equivalent in any other language.
  - frodwith 15y ago
    "There is no equivalent in any other language." I don't know, LWP is pretty easy to use.
    - pbreit 15y ago
      Does Perl come with LWP?
  - adeelk 15y ago
    If Python has a better standard library than the other languages you know, then you should learn some more languages.
    - teach 15y ago
      Do you have some examples? I thought I knew a fair number of programming languages, and Python's got one of the most comprehensive. IMO, Python's stdlib beats C++, Ruby, Java, javascript and Perl....
edna_piranha 15y ago
yes please and thank you muchly. puts away yak shaving contraption
dspeyer 15y ago
Anybody have a text version of this? I got maybe 30 slides in before I got too annoyed to continue.
- bpierre 15y ago
  You can read the Markdown file: https://github.com/kennethreitz/python-for-humans/blob/master/python-for-humans/1_content.md https://github.com/kennethreitz/python-for-humans/blob/maste...
  - jonmulholland 15y ago
    Just use arrow keys on your keyboard to navigate.
- tripzilch 15y ago
  Turn off your stylesheets and scroll down (the TOC links don't seem to work anymore when you do that).
joshbaptiste 15y ago
The portion that explains of how subprocess shuns dev/ops guys in the beginning is so true. Perl/Bash colleagues at work would basically ask me how to perform output=`command`. Once they seen subprocess, they would continue writing their script in Bash/Perl.
- bobbyi 15y ago
  If backticks are good enough for them, then they don't need the more complex usecases that Popen allows, so just tell them to use check_call or check_output. As far as they should be concerned, the subprocess module has two functions that are straightforward to use. And those functions are more convenient than the default behavior of backticks, because they handle for you raising an exception if the subprocess fails. Python: import subprocess output = subprocess.check_output('command') Perl: $output = `command` die "failed: $output" if $?
- brendano 15y ago
  Very true. I spent quite a while trying to learn subprocess, then gave up and just use os.popen() now. It's a shame -- there are certain subprocess features I really would like to have, but it's too hard to remember how to use it.
  - dagw 15y ago
    I used subprocess in a program as recently as this afternoon and I'm quite baffled about the amount of negativity it attracts. Sure it's not quite as simple as `ls -l` but it's not too far off and in return it gives you far more control. Is there anything in particular that you found hard or confusing?
  - codejoust 15y ago
    I've done a considerable amount of work in subprocess. It's API isn't amazing, but it does have quite a few great features and once you spend a little time with it really isn't too bad. Some of the piping features, along with shell arguments work quite well.
- d0mine 15y ago
prolepunk 15y ago
I wish that the developers of requests module stop changing it's API -- code that was working just fine with 0.6.4 suddenly began finding missing methods in version 0.8.5
- kenneth_reitz 15y ago
  All changes are for the best, I promise. API will stable by 1.0.
mattdeboard 15y ago
Great presentation on a library I've loved (and used) for awhile. However according to slide 42 I need to rewrite the regex module? I'm so busy this week though. edit: If anyone wants to see a real-world refactor from HTTPLib/2 to requests, I did so with Pysolr here: https://github.com/mattdeboard/pysolr/commit/db63d8910dec42ddfa8c8626dbbf73dbd3853c51 https://github.com/mattdeboard/pysolr/commit/db63d8910dec42d...
prolepunk 15y ago
On installing python -- the most practical way to work with it, is to have a moderately recent os-level python install and then build all the other python versions from source if required -- https://github.com/collective/buildout.python https://github.com/collective/buildout.python After that use virtualenv with virtualenvwrapper.
sktrdie 15y ago
What did they use to make the presentation?
- shuzchen 15y ago
  Looks like markdown and a sinatra based presentation framework. See the code at: https://github.com/kennethreitz/python-for-humans https://github.com/kennethreitz/python-for-humans
loevborg 15y ago
I was intrigued by the author's library "envoy", which is intended to provide a more intuitive interface to running processes from python. (https://github.com/kennethreitz/envoy https://github.com/kennethreitz/envoy) The back story is that the older APIs that Python comes with -- os.popen and os.system -- are deprecated. Programmers are urged to use the "subprocess" module instead. Although this doesn't have the problems of the original functions, it has a rather arcane interface, in particular if you want to read the output (stdout or stderr) of a subprocess. "envoy" seems to aim at fixing this, by providing sane defaults and being optimized for the common case. However, these defaults have drawbacks of their own. 1. envoy defaults to keeping the process output in memory, as a giant string. This can be a bad choice with regard to memory usage and performance. 2. You can run several processes in a pipe using ("cat foo | grep bla"). But otherwise as far as I can see, run() ignores regular Shell semantics, such as quotes. I imagine this can lead to unexpected results. The amount of data passed from one process to the next is capped at 10 MB -- recipe for bugs that are hard to find. 3. subprocess.call() accepts an array in the style of ["ls", "-l", "/mnt/My SD card"]. This has obvious advantages over having to deal with escaping shell characters. A good API should preserve this advantage over os.system(). 4. The defaults cannot be overridden, and no preperations have been made to allow changing them. Of course this can be changed in the future. However, one of the reasons the subprocess.* API is convoluted is that it allows all kinds of flexibility, much of which is needed in many serious programs. It may be difficult to add this flexibility to envoy at a later stage. The point is that a flexible API is hard. None of this is to discourage this initiative, which seems to me a much-needed improvement over Python's built-in API. Also, with a version number as low as 0.0.2, there is probably little need to worry about API compatibility.
- bobbyi 15y ago
  > it has a rather arcane interface, in particular if you want to read the output (stdout or stderr) of a subprocess. oputput = subprocess.check_output(my_command)
- Silhouette 15y ago
  > subprocess.call() accepts an array in the style of ["ls", "-l", "/mnt/My SD card"]. This has obvious advantages over having to deal with escaping shell characters. Unless you're running on Windows, in which case IME it will corrupt your carefully constructed parameters in completely inappropriate ways that can be debugged only at the cost of (a) changing the call() to execute a script that dumps the actual parameters supplied verbatim, and (b) at least an hour of your life that you're never getting back. This "feature" is about one step above MS Word's default autoreplace behaviour in irritation level. What happened to "Explicit is better than implicit" and "Special cases aren't special enough to break the rules"?
socratic 15y ago
This presentation brings up a tangential point that has always confused me: how error-prone is starting a subprocess, really? I agree with the author's goals of making common tasks easier and more obvious. urllib2 is an easy target, as it was added to the standard library over a decade ago, long before REST was something people talked about. The best tools for packaging, versioning, and testing have always been a bit ambiguous in any language, including Python. However, the author points out something that has always bothered me about Python: it is way harder to start a subprocess with an external command in Python than almost any other language. This has been true whether using sys or os or even subprocess, which is quite recent. I always felt that this had something to do with the constant warnings in the documentation about how a pipe between the subprocess and the Python process might fill and cause the subprocess to block. Or how running the program through shell rather than exec or something might cause some sort of security issue. Are these real issues that other languages ignore in the name of user convenience, or has Python just never been able to make the right API (as the author seems to argue)?
- bretthoerner 15y ago
  It's funny you mention that. The author/speaker wrote a "Subprocesses for humans" module, too: https://github.com/kennethreitz/envoy https://github.com/kennethreitz/envoy There's no fundamental problem that's stopped Python from doing this before. For some reason, all of the ways to spawn a subprocess in Python have tried to map almost directly to the underlying C API... which is pretty awful.
  - cturner 15y ago
    > For some reason, all of the ways to spawn a subprocess > in Python have tried to map almost directly to the > underlying C API I think both are good and necessary. One of the strengthes of python is that if you have a copy of Stevens you can usually work out how to do something in Python. And this is awesome. I've written things on top of unix that in times past would have been written in C. However, that mechanism is usually not very "pythonic". In the early days python had a principle that there should be one way to do things. You don't hear this so much any more: we're long past that now, with some things different between 2.6 and 2.7 (arg handling), and with multiple broken libraries in the stdlib. When you're working on your own computer and your own time with root access you can always hand-roll outcmes. But it's common to have to deal with a spread of python's and cater to the most obsolete version. Yet I suspect some people still aspire to the one-way-to-do-it, and pretend it's true. I think we should dump the principle. A good example of why compromise is not the right outcome is the curses library - it's not quite Stevens, but it's not friendly either. It's hard to do good work with the curses library. We'd be better off if there was (1) a close curses mapping to the C ncurses mechanisms and (2) a nice-to-use abstraction layer that hid far more away from you.
RShackleford 15y ago
python sucks!
mixmastamyk 15y ago
I guess I agree things could be simpler, although the cries of "garbage!" were a bit much. I wrote a wrapper function around urllib2 about 5 years ago and haven't looked back.
- adeelk 15y ago
  Wrappers are handy, but as soon as you need something beyond the basic use case they become useless. What’s great about Requests is that it seems to have minimal leakiness as an abstraction over HTTP.
  - mixmastamyk 15y ago
    My wrapper is quite robust after almost five years. e.g. It can save headers to alternate data streams (on NTFS) for proper 304 handling. If there is anything left to implement it could be done pretty quickly. Still I like these new projects; it's a shame they missed the python 3.x boat by only a year or two. That would have been a great time to include them in the stdlib.
    - adeelk 15y ago
      It sounds like you should release this wrapper :)
      - mixmastamyk 15y ago
        It is online, but embedded in my employers software. I could probably do some extraction of it, but not sure if it is worth the effort.
- pbreit 15y ago
  That's precisely the problem: your wrapper helps you...and only you (ie, it's worthless).
  - mixmastamyk 15y ago
teyc 15y ago
I blame GOF for making Python Standard Libs hard. The patterns described were for an OO system where functions were not first class. Python didn't need to be complicated. If you have a look at the older libraries, most of them were written in a procedural style. Not only that, it is very amenable to testing in the REPL. import smtplib s=smtplib.SMTP("localhost") s.sendmail("me@my.org",tolist,msg) note the absence of doers like "Adapters", "Handler", "Manager", "Factory" If you have a look at the XML library, roughly when "patterns" became popular, this style of thinking infested standard library contributions. It also coincides with a time when camelCased function names crept into the python standard library. Here's one in xml/dom/pulldom.py: self.documentFactory = documentFactory Once you see this, you know you are in for some subclassing. You can no longer REPL your way to figure out how things work, and you now have to consult the manual. Here's more pain from libraries of the same era, some of these I'd argue un-Pythonic: #xml/sax/xmlreader.py: def setContentHandler(self, handler): #wsgiref/simple_server.py: class ServerHandler(SimpleHandler): #urllib2.py: class HTTPDigestAuthHandler(BaseHandler, AbstractDigestAuthHandler): The last example is especially jarring. Abstract classes have a place in strongly typed world to declare interfaces, and help with vtable-style dispatch. In Python, where you have duck-typing and monkey patching, a class that virtually "does nothing" on its own stands out like a guy in a tux at a beach party. Even logging is infected by the same over-patterning. logging/__init__.py: class StreamHandler(Handler) LoggerAdapter(someLogger, dict(p1=v1, p2="v2")) "Managers" - what a pain when plain function handles would have done the job. Does this name even tell you what task the class performs? #multiprocessing/managers.py: class BaseManager(object) If anyone remembers, Java had to do OO in a big-style with OO everywhere -- there were no alternatives. Initially, buttons had to be subclassed just to handle click events, since functions were not first class objects. Then someone came up with a MouseListener interface, which proved too unwieldy to handle a single click. So the MouseEventAdapters came into being. Therefore, to handle a click in a "pattern" manner involves an anonymous class which subclasses MouseAdapter which implements MouseListener, which overrides MouseClick. Publishing how industry solves this problem of "MouseClick" over and over as a pattern [design pattern is a general reusable solution to a commonly occurring problem within a given context in software design] only gives legitimacy to an approach that has dubious wider applicability. Heavens help the future developers who are forced to do it because it is now recognized as being industrially "good practice" and codified in a reknowned book. It isn't! It was a style that was forced by the constraints of a language. This is neither pythonic nor necessary: panel.addMouseListener ( new MouseAdapter () { public void mouseEntered (MouseEvent e) { System.out.println (e.toString ()); } } ); Embracing "foolish, unschooled" thinking, this would be rendered in Python as: def mouseEntered(event): print event panel.mouseEntered = mouseEntered or for multiple event handlers panel.mouseEntered.append(mouseEntered) This style of API again allows effective exploration on the REPL.
josefrichter 15y ago
Why don't you just switch to Ruby?
- tdfx 15y ago
  I didn't find making moderately complex HTTP requests in Ruby to be much more fun than urllib2 in Python. Do you have a library you recommend?
  - josefrichter 15y ago
    I like httparty or mechanize, but feel free make your own choice over here https://www.ruby-toolbox.com/categories/http_clients https://www.ruby-toolbox.com/categories/http_clients
    - tdfx 15y ago
      I've tried rest-client, em-http-request, and mechanize. I don't recall their specific failings except for mechanize, which seemed to work awesome in 95% of cases and produce cryptic errors with the latter 5%. I will take a look at httparty, though, thanks.
arjn 15y ago
I can confirm that the python subprocess api is a pain to use and also documented poorly. I recently had to use (no choice) python 2.5.x to write a script that extensively called external programs and ran into several problems. It strange that a language such as python which I find so easy to use in many cases does not already have a good as in simple, safe and well documented subprocess api.
pbreit 15y ago
It's really inexcusable that in 2012 (or 1992 even) a language that otherwise is well-suited for internet programming does not come with a first class httpclient.
dhalexander 15y ago
The Python standard library has gotten worse over time, as it got loaded up with more and more features, obfuscating the common use cases. The irony now is that to do simple, everyday things (like http requests) you are now better off installing a third party package like "requests" than using the standard library. So much for "batteries included." The standard library needs a reboot. Why not do it in Python 3? Nobody's using it yet anyway ;-)

427 points by craigkerstiens 15y ago

seems the guy nailed down many issues i have had in the past :)

Wow those libraries are indeed great, amazing compared to the standard libs. Hope they'll be included in the standard libs one day.

dextorious 15y ago
+1 I'd like to see some community effort to build a collection of similar "better than the standard" libs. Which, at some point, could replace the standard libs. Or be the de facto standard, a pip install call away...
stock_toaster 15y ago
While python has always been 'batteries included' I think some of the batteries should not have been included. Libraries tend to move more quickly than the language and interpreter/compiler. Tying them together, while convenient, often leads to rot, clunky libraries, slow moving updates, and libraries being build to the interpreter/compiler instead of to the needs of the users. I would like to see instead a somewhat canonical (widely accepted) list of the highest quality libraries for a given set of needs, with information and pro/con/caveats listed for each, instead of them being included in the mainline trunk. I really applaud what Kenneth Reitz has been doing lately. -- a very happy user of the `requests` library

kenneth_reitz 15y ago

Audio is available here: https://github.com/kennethreitz/python-for-humans https://github.com/kennethreitz/python-for-humans

drivingmenuts 15y ago

Hangs on the first slide unless I'm doing something really wrong.

kenneth_reitz 15y ago
Try the arrow keys
- Symmetry 15y ago
  Or SPACE and SHIFT-SPACE. But it also took me a bit to figure out how to navigate the slides.
- WayneDB 15y ago
  It took me at least 30 seconds to figure that out. I clicked darn near every pixel on the page before that occurred to me :) I guess you really stick by that rule of "one way to do it", huh? I kid. Great presentation, I like your passion and I hope that your call to arms is answered.
- drivingmenuts 15y ago
  You know, some sort of clue that the arrow keys are used for navigation would be helpful.
- brown9-2 15y ago
  A clear sign that the one doing something wrong is the website's designer. Also, it breaks the back button.
Heliosmaster 15y ago
Am I weird if I say that such badly-written presentation make me much less enthusiastic about their content?

y3di 15y ago

The slides don't fit vertically on my screen, so some of the content is cut off. There's no scroll bar so initially it was difficult to figure how to see the info cut off from the bottom. Chrome's text zoom out didn't work either. I had to highlight the text and drag downwards in order to see the content. But it was annoying having to do this for every slide with a lot of content. Otherwise, these libraries seem really useful. Thanks for this.

minikomi 15y ago
Had the same problem but Chrome zoomed out for me.. (Chrome 16 -> [command -] on a mac.)
cbs 15y ago
I had the same problem too. I had no idea it was supposed to be a slide show, then once I gave my browser as much screen real estate as the site wanted I had to figure out how to navigate the damn thing. Whats so wrong with just sticking a bunch of static slides on a page one after another?

rmc 15y ago

This makes a lot of good points, but some bad ones. Esp. the "installing python" one. Just use your package manager to install all the versions you need. And for "Packaging and Dependencies", just use pip.

engtech 15y ago
I agree that people should use a good package manager, but the reality of the corporate env world is that you'll be trying to use the latest Python on some ridiculously old unix install without root access, and you'll never be able to get IT to install the latest for all users.
- joshbaptiste 15y ago
  +1 for you sir, we are constantly fighting with Unix admins who refuse to install anything but the generic terribly old 2.3 - 2.4 Python versions that are bundled with their default enterprise Suse packages, which only leads to full /home issues since everyone just downloads their own versions, compiling extensions manually.
  - antoncohen 15y ago
    Or you could write the applications for the systems they will run on. Why develop in the latest Ubuntu with Python 2.7 when you know your company runs RHEL 5 or SUSE with Python 2.4? I know it was annoying how long RHEL lasted with Python 2.4, but now the latest RHEL, Debian-stable, and Ubuntu LTS all have Python 2.6, so write to Python 2.6 if your company uses those OSs, there's not excuse to write incompatible code targeted at Python 2.7. In many Linux distributions it is impossible to replace the Python version, core parts of the OS like package management will break. It is possible to install another version in parallel and use "virtualenv --python=" to use it for an application. If you are using RHEL/CentOS 5 you can install the python26 package from the EPEL repository.
tbatterii 15y ago
What's the package manager for windows? Mostly I agree with you, but installing python on windows is a PITA compared to *nix. PIP works until you need something that requires compiling extensions which is also a PITA on windows. IIRC easy_install can install binaries, but I could be wrong about that since I don't do windows development anymore and thus pip works for me 99% of the time. Regardless, my point is, though I think we all wish it was as simple as using your package manager and pip, it simply isn't that way for everyone. And it probably never will be, but we can always make it simpler. :)

tuananh 15y ago

The name says it all. This is much more readable compared to urllib2

skrebbel 15y ago

I like the presentation, but for me it just underlines how it's smart to stay far away from Python. It's great that it's improving, but many other languages have a much better library/API situation than Python has had for years already. Will Python catch up fast enough?

ak217 15y ago
It's quite the opposite, actually. Python already has an extremely good and very capable standard library, arguably better than any other language. Python also has a community with a strong sense of what kind of API design best fits with the Python philosophy (which usually boils down to readability and consistency). So there are many efforts going on to improve the standard libraries. For example, the standard library already has urllib2, which provides HTTP support much better than most other languages' standard libraries. But Requests is a rewrite of it, taking ease of use to a whole new level without compromising any of the features. There is no equivalent in any other language.
- frodwith 15y ago
  "There is no equivalent in any other language." I don't know, LWP is pretty easy to use.
  - pbreit 15y ago
    Does Perl come with LWP?
- adeelk 15y ago
  If Python has a better standard library than the other languages you know, then you should learn some more languages.
  - teach 15y ago
    Do you have some examples? I thought I knew a fair number of programming languages, and Python's got one of the most comprehensive. IMO, Python's stdlib beats C++, Ruby, Java, javascript and Perl....

edna_piranha 15y ago

yes please and thank you muchly. puts away yak shaving contraption

dspeyer 15y ago

Anybody have a text version of this? I got maybe 30 slides in before I got too annoyed to continue.

bpierre 15y ago
You can read the Markdown file: https://github.com/kennethreitz/python-for-humans/blob/master/python-for-humans/1_content.md https://github.com/kennethreitz/python-for-humans/blob/maste...
- jonmulholland 15y ago
  Just use arrow keys on your keyboard to navigate.
tripzilch 15y ago
Turn off your stylesheets and scroll down (the TOC links don't seem to work anymore when you do that).

joshbaptiste 15y ago

The portion that explains of how subprocess shuns dev/ops guys in the beginning is so true. Perl/Bash colleagues at work would basically ask me how to perform output=`command`. Once they seen subprocess, they would continue writing their script in Bash/Perl.

bobbyi 15y ago
If backticks are good enough for them, then they don't need the more complex usecases that Popen allows, so just tell them to use check_call or check_output. As far as they should be concerned, the subprocess module has two functions that are straightforward to use. And those functions are more convenient than the default behavior of backticks, because they handle for you raising an exception if the subprocess fails. Python: import subprocess output = subprocess.check_output('command') Perl: $output = `command` die "failed: $output" if $?
brendano 15y ago
Very true. I spent quite a while trying to learn subprocess, then gave up and just use os.popen() now. It's a shame -- there are certain subprocess features I really would like to have, but it's too hard to remember how to use it.
- dagw 15y ago
  I used subprocess in a program as recently as this afternoon and I'm quite baffled about the amount of negativity it attracts. Sure it's not quite as simple as `ls -l` but it's not too far off and in return it gives you far more control. Is there anything in particular that you found hard or confusing?
- codejoust 15y ago
  I've done a considerable amount of work in subprocess. It's API isn't amazing, but it does have quite a few great features and once you spend a little time with it really isn't too bad. Some of the piping features, along with shell arguments work quite well.
d0mine 15y ago

prolepunk 15y ago

I wish that the developers of requests module stop changing it's API -- code that was working just fine with 0.6.4 suddenly began finding missing methods in version 0.8.5

kenneth_reitz 15y ago
All changes are for the best, I promise. API will stable by 1.0.

mattdeboard 15y ago

Great presentation on a library I've loved (and used) for awhile. However according to slide 42 I need to rewrite the regex module? I'm so busy this week though. edit: If anyone wants to see a real-world refactor from HTTPLib/2 to requests, I did so with Pysolr here: https://github.com/mattdeboard/pysolr/commit/db63d8910dec42ddfa8c8626dbbf73dbd3853c51 https://github.com/mattdeboard/pysolr/commit/db63d8910dec42d...

prolepunk 15y ago

On installing python -- the most practical way to work with it, is to have a moderately recent os-level python install and then build all the other python versions from source if required -- https://github.com/collective/buildout.python https://github.com/collective/buildout.python After that use virtualenv with virtualenvwrapper.

sktrdie 15y ago

What did they use to make the presentation?

shuzchen 15y ago
Looks like markdown and a sinatra based presentation framework. See the code at: https://github.com/kennethreitz/python-for-humans https://github.com/kennethreitz/python-for-humans

loevborg 15y ago

I was intrigued by the author's library "envoy", which is intended to provide a more intuitive interface to running processes from python. (https://github.com/kennethreitz/envoy https://github.com/kennethreitz/envoy) The back story is that the older APIs that Python comes with -- os.popen and os.system -- are deprecated. Programmers are urged to use the "subprocess" module instead. Although this doesn't have the problems of the original functions, it has a rather arcane interface, in particular if you want to read the output (stdout or stderr) of a subprocess. "envoy" seems to aim at fixing this, by providing sane defaults and being optimized for the common case. However, these defaults have drawbacks of their own. 1. envoy defaults to keeping the process output in memory, as a giant string. This can be a bad choice with regard to memory usage and performance. 2. You can run several processes in a pipe using ("cat foo | grep bla"). But otherwise as far as I can see, run() ignores regular Shell semantics, such as quotes. I imagine this can lead to unexpected results. The amount of data passed from one process to the next is capped at 10 MB -- recipe for bugs that are hard to find. 3. subprocess.call() accepts an array in the style of ["ls", "-l", "/mnt/My SD card"]. This has obvious advantages over having to deal with escaping shell characters. A good API should preserve this advantage over os.system(). 4. The defaults cannot be overridden, and no preperations have been made to allow changing them. Of course this can be changed in the future. However, one of the reasons the subprocess.* API is convoluted is that it allows all kinds of flexibility, much of which is needed in many serious programs. It may be difficult to add this flexibility to envoy at a later stage. The point is that a flexible API is hard. None of this is to discourage this initiative, which seems to me a much-needed improvement over Python's built-in API. Also, with a version number as low as 0.0.2, there is probably little need to worry about API compatibility.

bobbyi 15y ago
> it has a rather arcane interface, in particular if you want to read the output (stdout or stderr) of a subprocess. oputput = subprocess.check_output(my_command)
Silhouette 15y ago
> subprocess.call() accepts an array in the style of ["ls", "-l", "/mnt/My SD card"]. This has obvious advantages over having to deal with escaping shell characters. Unless you're running on Windows, in which case IME it will corrupt your carefully constructed parameters in completely inappropriate ways that can be debugged only at the cost of (a) changing the call() to execute a script that dumps the actual parameters supplied verbatim, and (b) at least an hour of your life that you're never getting back. This "feature" is about one step above MS Word's default autoreplace behaviour in irritation level. What happened to "Explicit is better than implicit" and "Special cases aren't special enough to break the rules"?

socratic 15y ago

This presentation brings up a tangential point that has always confused me: how error-prone is starting a subprocess, really? I agree with the author's goals of making common tasks easier and more obvious. urllib2 is an easy target, as it was added to the standard library over a decade ago, long before REST was something people talked about. The best tools for packaging, versioning, and testing have always been a bit ambiguous in any language, including Python. However, the author points out something that has always bothered me about Python: it is way harder to start a subprocess with an external command in Python than almost any other language. This has been true whether using sys or os or even subprocess, which is quite recent. I always felt that this had something to do with the constant warnings in the documentation about how a pipe between the subprocess and the Python process might fill and cause the subprocess to block. Or how running the program through shell rather than exec or something might cause some sort of security issue. Are these real issues that other languages ignore in the name of user convenience, or has Python just never been able to make the right API (as the author seems to argue)?

bretthoerner 15y ago
It's funny you mention that. The author/speaker wrote a "Subprocesses for humans" module, too: https://github.com/kennethreitz/envoy https://github.com/kennethreitz/envoy There's no fundamental problem that's stopped Python from doing this before. For some reason, all of the ways to spawn a subprocess in Python have tried to map almost directly to the underlying C API... which is pretty awful.
- cturner 15y ago
  > For some reason, all of the ways to spawn a subprocess > in Python have tried to map almost directly to the > underlying C API I think both are good and necessary. One of the strengthes of python is that if you have a copy of Stevens you can usually work out how to do something in Python. And this is awesome. I've written things on top of unix that in times past would have been written in C. However, that mechanism is usually not very "pythonic". In the early days python had a principle that there should be one way to do things. You don't hear this so much any more: we're long past that now, with some things different between 2.6 and 2.7 (arg handling), and with multiple broken libraries in the stdlib. When you're working on your own computer and your own time with root access you can always hand-roll outcmes. But it's common to have to deal with a spread of python's and cater to the most obsolete version. Yet I suspect some people still aspire to the one-way-to-do-it, and pretend it's true. I think we should dump the principle. A good example of why compromise is not the right outcome is the curses library - it's not quite Stevens, but it's not friendly either. It's hard to do good work with the curses library. We'd be better off if there was (1) a close curses mapping to the C ncurses mechanisms and (2) a nice-to-use abstraction layer that hid far more away from you.

RShackleford 15y ago

python sucks!

mixmastamyk 15y ago

I guess I agree things could be simpler, although the cries of "garbage!" were a bit much. I wrote a wrapper function around urllib2 about 5 years ago and haven't looked back.

adeelk 15y ago
Wrappers are handy, but as soon as you need something beyond the basic use case they become useless. What’s great about Requests is that it seems to have minimal leakiness as an abstraction over HTTP.
- mixmastamyk 15y ago
  My wrapper is quite robust after almost five years. e.g. It can save headers to alternate data streams (on NTFS) for proper 304 handling. If there is anything left to implement it could be done pretty quickly. Still I like these new projects; it's a shame they missed the python 3.x boat by only a year or two. That would have been a great time to include them in the stdlib.
  - adeelk 15y ago
    It sounds like you should release this wrapper :)
    - mixmastamyk 15y ago
      It is online, but embedded in my employers software. I could probably do some extraction of it, but not sure if it is worth the effort.
pbreit 15y ago
That's precisely the problem: your wrapper helps you...and only you (ie, it's worthless).
- mixmastamyk 15y ago

teyc 15y ago

I blame GOF for making Python Standard Libs hard. The patterns described were for an OO system where functions were not first class. Python didn't need to be complicated. If you have a look at the older libraries, most of them were written in a procedural style. Not only that, it is very amenable to testing in the REPL. import smtplib s=smtplib.SMTP("localhost") s.sendmail("me@my.org",tolist,msg) note the absence of doers like "Adapters", "Handler", "Manager", "Factory" If you have a look at the XML library, roughly when "patterns" became popular, this style of thinking infested standard library contributions. It also coincides with a time when camelCased function names crept into the python standard library. Here's one in xml/dom/pulldom.py: self.documentFactory = documentFactory Once you see this, you know you are in for some subclassing. You can no longer REPL your way to figure out how things work, and you now have to consult the manual. Here's more pain from libraries of the same era, some of these I'd argue un-Pythonic: #xml/sax/xmlreader.py: def setContentHandler(self, handler): #wsgiref/simple_server.py: class ServerHandler(SimpleHandler): #urllib2.py: class HTTPDigestAuthHandler(BaseHandler, AbstractDigestAuthHandler): The last example is especially jarring. Abstract classes have a place in strongly typed world to declare interfaces, and help with vtable-style dispatch. In Python, where you have duck-typing and monkey patching, a class that virtually "does nothing" on its own stands out like a guy in a tux at a beach party. Even logging is infected by the same over-patterning. logging/__init__.py: class StreamHandler(Handler) LoggerAdapter(someLogger, dict(p1=v1, p2="v2")) "Managers" - what a pain when plain function handles would have done the job. Does this name even tell you what task the class performs? #multiprocessing/managers.py: class BaseManager(object) If anyone remembers, Java had to do OO in a big-style with OO everywhere -- there were no alternatives. Initially, buttons had to be subclassed just to handle click events, since functions were not first class objects. Then someone came up with a MouseListener interface, which proved too unwieldy to handle a single click. So the MouseEventAdapters came into being. Therefore, to handle a click in a "pattern" manner involves an anonymous class which subclasses MouseAdapter which implements MouseListener, which overrides MouseClick. Publishing how industry solves this problem of "MouseClick" over and over as a pattern [design pattern is a general reusable solution to a commonly occurring problem within a given context in software design] only gives legitimacy to an approach that has dubious wider applicability. Heavens help the future developers who are forced to do it because it is now recognized as being industrially "good practice" and codified in a reknowned book. It isn't! It was a style that was forced by the constraints of a language. This is neither pythonic nor necessary: panel.addMouseListener ( new MouseAdapter () { public void mouseEntered (MouseEvent e) { System.out.println (e.toString ()); } } ); Embracing "foolish, unschooled" thinking, this would be rendered in Python as: def mouseEntered(event): print event panel.mouseEntered = mouseEntered or for multiple event handlers panel.mouseEntered.append(mouseEntered) This style of API again allows effective exploration on the REPL.

josefrichter 15y ago

Why don't you just switch to Ruby?

tdfx 15y ago
I didn't find making moderately complex HTTP requests in Ruby to be much more fun than urllib2 in Python. Do you have a library you recommend?
- josefrichter 15y ago
  I like httparty or mechanize, but feel free make your own choice over here https://www.ruby-toolbox.com/categories/http_clients https://www.ruby-toolbox.com/categories/http_clients
  - tdfx 15y ago
    I've tried rest-client, em-http-request, and mechanize. I don't recall their specific failings except for mechanize, which seemed to work awesome in 95% of cases and produce cryptic errors with the latter 5%. I will take a look at httparty, though, thanks.

arjn 15y ago

I can confirm that the python subprocess api is a pain to use and also documented poorly. I recently had to use (no choice) python 2.5.x to write a script that extensively called external programs and ran into several problems. It strange that a language such as python which I find so easy to use in many cases does not already have a good as in simple, safe and well documented subprocess api.

pbreit 15y ago

It's really inexcusable that in 2012 (or 1992 even) a language that otherwise is well-suited for internet programming does not come with a first class httpclient.

dhalexander 15y ago

The Python standard library has gotten worse over time, as it got loaded up with more and more features, obfuscating the common use cases. The irony now is that to do simple, everyday things (like http requests) you are now better off installing a third party package like "requests" than using the standard library. So much for "batteries included." The standard library needs a reboot. Why not do it in Python 3? Nobody's using it yet anyway ;-)