6 ms·
Python for Humans
- slurgfest 14y agoThis keeps getting reposted, http://news.ycombinator.com/item?id=3419671 http://news.ycombinator.com/item?id=3419671 http://news.ycombinator.com/item?id=4117035 http://news.ycombinator.com/item?id=4117035 http://news.ycombinator.com/item?id=4651882 http://news.ycombinator.com/item?id=4651882
- ColinWright 14y agoAre you sure these are all the same? I can't check from here (on my mobile, poor reception) but the URLs look significantly different. This one is a slide deck, one of the ones you quote is a video. Added in edit: It does look like this one - http://news.ycombinator.com/item?id=4651882 http://news.ycombinator.com/item?id=4651882 - really is a duplicate, but it got no votes and no comments, so perhaps a repost is in order.
- jwpeddle 14y agoIt was reposted today because he gave the talk again this morning at Pycon Canada, I suspect.
- peter_l_downs 14y agoSlightly off topic: in an effort to become a better programmer, and to improve the Python ecosystem, I've tried to write a "for Humans" style logging library. Would anyone mind giving me feedback? http://peterdowns.com/lggr/ http://peterdowns.com/lggr/
- Goopplesoft 14y agoLooks great. Can't get to testing it today but I will get back to you soon.
- peter_l_downs 14y agoThanks! Is the API simple enough?
- jwpeddle 14y agoSome feedback from reading this page: - Why is your example logger named d? It might seem nitpicky but it's hard to read an example with a meaningless single character variable. - "d.close() # stop logging" - what is this? What does it mean to "stop logging" and why do I want to? - "Add a coroutine" - I would wager the lay Python developer doesn't even know what this word means. - lggr.Lggr() - Why not just name the class Logger? - The default format variables are inconsistent about when words are separated with an underscore. - I can't make sense of the example logging calls. In one you pass a dictionary as the second argument, in another you pass three strings as separate arguments. Why would anyone pass a message like this instead of just using standard string formatting? Especially when there's other legit arguments like extra. It's not even really clear why you'd want to pass something in extra instead of in the message. I fully support your goal, but at a glance this seems like a confusing alternative to logging. EDIT: hot dang it's hard to format a bulleted list on HN
- peter_l_downs 14y agoHi jwpeddle, thanks for all of the feedback! All of the changes I've made are on a branch called 'hn-fix' (https://github.com/peterldowns/lggr/tree/hn-fix https://github.com/peterldowns/lggr/tree/hn-fix) if you'd like to take a look. > Why is your example logger named d? It might seem nitpicky but > it's hard to read an example with a meaningless single character variable. No particular reason -- you're right, I will change this now. > "d.close() # stop logging" - what is this? What does it mean to "stop logging" > and why do I want to? This is a way to "close" the logger and clean up all of its associated coroutines. An example of when you might want to is upon catching a fatal error to your program, you might want to log the error and then safely clean up all open files or network sockets to which the logger is writing. The method name is clearly confusing -- what do you think of using "shutdown" instead? > "Add a coroutine" - I would wager the lay Python developer doesn't > even know what this word means. Maybe not, but they should! The readme now includes a link to dabeaz's coroutines page, and I'll add a quick overview in a couple of minutes. > lggr.Lggr() - Why not just name the class Logger? For consistency's sake. Maybe if I had to start again I would call the project `logger`, but I decided to be "hip" and use a vowel-less name instead :) > The default format variables are inconsistent about when words are separated > with an underscore. The initial idea was to mimic the variable names from the default logging module (http://docs.python.org/2/library/logging.html#logrecord-attributes http://docs.python.org/2/library/logging.html#logrecord-attr...). You're right that it is confusing though! I think I will rename everything to be lower case, one word, instead of the default module's mix of camelCase and underscore_separated names. Thoughts? > I can't make sense of the example logging calls. In one you pass a > dictionary as the second argument, in another you pass three strings > as separate arguments. Why would anyone pass a message like this > instead of just using standard string formatting? Especially when > there's other legit arguments like extra. It's not even really clear > why you'd want to pass something in extra instead of in the message. I should definitely clarify what formats are allowed and aren't. The log message format is using standard string formatting -- string.format(), to be exact. The 'extra' argument is a result of trying to imitate the default library (see http://docs.python.org/2/library/logging.html#logging.Logger.debug http://docs.python.org/2/library/logging.html#logging.Logger... for a description of the 'extra' kwarg), and can be useful when you'd like to pass information to every single log message.
- unshift 14y agoyour library has no tests! it's a good effort but I honestly wouldn't use it yet. a few nit picks: i can't imagine a scenario where i'd want my logger to close stdout (or any other file descriptor) for me. there is a lot of missing error handling, which is really important for something critical like a logger. what happens when disk space runs out? no timeouts on network operations? also your SMTP and Gmail loggers don't form valid MIME messages (I can't log non-ascii?). you also seem to just swallow exceptions which is totally not what I would want or expect from a logging library. keep at it though, the logging module's API (inspired by log4j) is fairly painful.
- peter_l_downs 14y ago> your library has no tests! it's a good effort but I > honestly wouldn't use it yet. You're right, and I'd love to have you help me add them! > i can't imagine a scenario where i'd want my logger to > close stdout (or any other file descriptor) for me. By default it won't -- see https://github.com/peterldowns/lggr/blob/master/lggr/__init__.py#L297 https://github.com/peterldowns/lggr/blob/master/lggr/__init_.... > there is a lot of missing error handling, which is really > important for something critical like a logger. what > happens when disk space runs out? no timeouts on > network operations? > [...] > you also seem to just swallow exceptions which is totally > not what I would want or expect from a logging library. Lggr will fail silently by default, because your logging library shouldn't cause your code to crash. If you'd like, it's quite easy to stop it from suppressing errors. For more complex error handling, I think users should write their own coroutines for handling log messages -- the default file printing and network sending coroutines are merely the most basic case. > also your SMTP and Gmail loggers don't form valid MIME messages > (I can't log non-ascii?). Please help me and add loggers that do format valid MIME message! The included loggers/coroutines are just a few examples I came up with, but I'd love if other people were to contribute more :) > keep at it though, the logging module's API (inspired by log4j) > is fairly painful. Thank you, and thanks for your feedback :)
- mercurial 14y agoI like the idea of delegating logging to its own process in theory (in practice, it would depend on your configuration, I guess, it may mean a lot of context switching), but your built-in coroutines seem to rely on yield instead. Have you experimented with all three styles? A big feature missing, IMHO, is file-based configuration. You definitely want this for larger projects.
- antihero 14y agoInteresting. This is weird as fuck, though: d.critical("Someone {} us {} the {}!", "set", "up", "bomb") See that seems ambiguous - does it actually use the string formating mini-lanaguage?
- Evbn 14y agoMost unpythonic module name ever. Change it to logger.
- ak217 14y agoWow, this is cool. Kenneth has started to identify and call out other libraries that are "barriers to entry" in Python. If this is a call to arms to scale the Requests experience, it could grow into something really powerful - a movement to systematically reinvent and rejuvenate the Python standard library.
- andybak 14y agoYep. I was looking for exactly this list. Incidentally Python could use an more equivalent to djangopackages.com to help in identifying popular alternatives to standard lib packages.
- pyre 14y agoWhile not a table of "stdlib package => 3rd party package", there is this: http://docs.python-guide.org/en/latest http://docs.python-guide.org/en/latest [ Also by Kenneth Reitz ]
- wylie 14y agoThat link doesn't appear to work for me, so I think the same content is available at http://docs.python-guide.org/ http://docs.python-guide.org/
- pydanny 14y agoI'm of the leads on that project. We're working on upgrading the code base to support all of Python, and hope to have that ready by PyCon US.
- wrath 14y agoI was at his talk this morning at pycon canada. It was a very good talk, the highlight of the conference. I like his approach, although I think it'll be a challenge to get there since not everyone has his "visionary" and api building skills, but someone has to start somewhere. Well done.
- jwpeddle 14y agoAlso there- loved it. I think the key is what he said about writing the readme first. Write the docs on how you'd want to use the software, then hold yourself to that API when you actually write it.
- pav3l 14y ago>lxml is awesome, but difficult to install So true... I often need to write something that will run on a Windows machine and I just end up having to use the old version of Beautiful Soup (before it became dependent on lxml). Pretty frustrating.
- tnuc 14y agolxml is great. The docs and the installation aren't the best. For a windows install. http://www.lfd.uci.edu/~gohlke/pythonlibs/ http://www.lfd.uci.edu/~gohlke/pythonlibs/
- pav3l 14y agoThanks so much! Just tried it and the setup binaries worked like a charm! 20 seconds for the whole experience including downloading and testing. Very different from how I remember my experience with lxml on Windows when I tried it before! They should definitely have a link to this website from http://lxml.de/installation.html#ms-windows http://lxml.de/installation.html#ms-windows
- revolutions 14y agoA bit off-topic, but I couldn't find this and was very curious: What font are the slides using in the code snippets? It looks very clean--I didn't realize it was monospace at first.
- jamsterdam 14y ago+1. Would love to know the name of that font.
- human_error 14y agoLooks like Ubuntu Mono.
- willcodeforfoo 14y agoIt is Monofur: http://www.dafont.com/monofur.font http://www.dafont.com/monofur.font, IMO it looks better on slides in small doses than in an editor all day long.
- porker 14y agoAnd also, what's the main font used for all the text? It's a lovely serif
- revolutions 14y agoLooks like this one: http://www.azfonts.net/load_font/high_tower_text.html http://www.azfonts.net/load_font/high_tower_text.html
- porker 14y agoSpot on. Thanks! http://www.myfonts.com/fonts/fontbureau/hightower/textroman/ http://www.myfonts.com/fonts/fontbureau/hightower/textroman/
- inovica 14y agoHere's a video: http://www.youtube.com/watch?v=Y97D8j6CAog http://www.youtube.com/watch?v=Y97D8j6CAog
- codexon 14y agoHis requests module is an improvement, but he says being able to specify the outbound IP of an HTTP request is out of the scope of his module??? https://github.com/kennethreitz/requests/issues/394 https://github.com/kennethreitz/requests/issues/394 That's why I am still using urllib2.
- jwpeddle 14y agoI'm not sure why you feel this niche requirement requires three question marks. The point of modules like this is to make it easier to do the tasks that 95% of developers are likely to need to do regularly, at the cost of not covering all rare use cases.
- codexon 14y agoI think it is a fairly common requirement to use multiple IPs to avoid throttling or for APIs that restrict access to individual IPs. The fact that he was confused by it does not help his credibility when he is trying to tell people how to design their libraries.
- aiiane 14y agoI have yet to ever have to use multiple outbound IPs from the same machine for an HTTP request from Python. It might be "fairly common" for whatever you do, but in terms of the general population, it's not.
- maaku 14y agoStill, it's a fair point that it at least should be possible. It's fine by me if the uncommon 5% tasks are just as difficult to do in requests or even a little more so, so long as the most common 95% of tasks are a breeze.
- akkartik 14y agoIt's a reasonable tradeoff for a specific library. But if he's trying to claim that the entire eco-system should put the API above all, then it's worth asking how the 5% use case integrates with this worldview. If I suddenly find I need multiple IPs, do I now need to switch my http client library?
- waterside81 14y agorequests & envoy have saved me a huge amount of hours. I wish I met Kenneth when he was in Toronto this weekend, I would have bought him anything he wanted.
- kenneth_reitz 14y agoI'm still in Toronto :)
- waterside81 14y agoHaha, alright you name the place & time, I will bring said "anything you want" object.
- ersii 14y agoGoing straight to the Python Stockholm Meetup after yer flight? :-O Mmmmh, jetlag
- falcolas 14y agoWhile I appreciate the effort to make some more simplistic APIs available alongside their more complex counterparts - libraries like urllib2 are not expendable in the process. In the act of making an API simpler and abstracting the details - you loose a lot of the detailed control provided by these lower level libraries. Perhaps you won't miss the 10% as outlined in the slides, but someone out there will. Do we really need both? I think no, but then again for most of what I do, urllib2 just works; I'm not really the target for requests. Just don't expect that requests will ever replace urllib2.
- scottbruin 14y agoIf requests hits the 90% and urllib2 the 10%, then requests should be the standard library and urllib2 a module. I think his point is more to point out good modules that most people should use, and should become common knowledge of what people should turn to when trying to do something. Java, for example, has several date and time options but Jodatime actually does what most people want to do with dates and times.
- emillon 14y agoFor new projects the use HTTP at a high level, I believe that there is no reason to prefer urllib2 to requests. Probably it won't replace urllib2, but it has attained critical mass and won't go away soon.
- kenneth_reitz 14y agoAnyone needing more power/flexibility should use httplib.
- 3pt14159 14y agoWas there today at PyCon Canada and not only was the presentation Kenneth gave very good, but he was very awesome to talk to afterwards. I randomly came across his 500px account, so we sent him a shirt, and he wore it on Friday just to say thanks :3 (I actually didn't get to see him on the Friday night party, probably because I know him best by his pixelated twitter pic, but it is super cool that he take the time to think of stuff like that).
- jklio 14y agoI can't express how depressing dealing with text encoding issues in any language is.
- maaku 14y agoAbsolutely love `requests`. First I've heard of `envoy`; will certainly be using that in the future. Thank you.
- js2 14y agourllib2 is so bad (and I know how to use it) that I have at times used subprocess + curl, which says a lot, because subprocess ain't exactly pretty either. But the thing is, urllib2 is included and is the devil I know.
- batgaijin 14y agoI remember writing raw socket stuff and I actually thought that was better than urllib2.
- andybak 14y agoI was googling to see the status of the 'requests should be in stdlib' debate and discovered PEP411: http://www.python.org/dev/peps/pep-0411/ http://www.python.org/dev/peps/pep-0411/ Basically this helps mitigate the downside of a module getting included in stdlib.
- rmccue 14y agoAs much as I know everyone dislikes PHP, there's lots of people still working with it. I love Requests in Python, so I built Requests for PHP: http://requests.ryanmccue.info/ http://requests.ryanmccue.info/ I have to agree with Kenneth's points, in that pragmatism should outweigh the theoretical points. For example, in PHP, using a class as a grouping for static methods is a bad idea. I agree to a point, but there's something to be said for the ability to do `Requests::get('http://google.com/)` http://google.com/)` versus all the crap you have to do with cURL. (Plus, it's always >90% test coverage, which I stick to religiously.)
- dave1010uk 14y agoIf autoloading namespaces or namespaced functions was possible then static classes wouldn't be needed and things like this would be commonplace: \Requests\get('http://google.com');
- antihero 14y agoAnother reminder of why in the fuck did they use the backslash character.
- rmccue 14y agoThat's true, and it's something that's being discussed on the PHP core mailing list. However, I support PHP 5.2 (pragmatic reasons; 5.2 is still used by a huge percentage of hosts), so I wouldn't be able to use that anyway.
- RegEx 14y agoI'll check this out. I use python at home but have to use PHP at work. Thanks :)
- antihero 14y agoA little nitpick. In your tutorial you do: $request = Requests::get(...); This should really be: $response = Requests::get(...); As you get a Response object back.
- dagw 14y agoReminds me of a chat session I had with a dev at another office a few days ago: him: "have you ever used urllib2" me: "not if I can avoid it" him: "I've just spent 3 days trying to do X" me: "have you heard of Requests" him: "nope" ...send URL... him: "god damn it! Why didn't I talk to you three days ago" 20 minutes later he's replaced three days work with a dozen lines of easy to read Requests code.
- dmritard96 14y agospot on - if only I knew many others felt this way when building projects. lol Maybe libraries should have a rating or something just for fun. Might help direct attention at problems like this.
- jaimebuelta 14y agoThere is a video of him giving this presentation on the PyCon Ireland, in case someone want to hear it. http://www.youtube.com/watch?v=IN5AlsEHdIM&feature=plcp http://www.youtube.com/watch?v=IN5AlsEHdIM&feature=plcp