13 ms·

5 Years of Bad Ideas -- Python tips, tricks, and chicanery

130 points by jnazario 14y ago

Smerity 14y ago
This presentation is by Armin Ronacher, the author (amongst many other things) of Flask, Werkzeug and Jinja2. If people find this presentation interesting he catalogues and provides slides (and recordings where appropriate) of all his previous presentations: http://lucumr.pocoo.org/talks/ http://lucumr.pocoo.org/talks/ For those who prefer straight PDF over Scribd, Armin has a PDF provided for this presentation[1]. There's also a video but it seems you need to be a member of EuroPython[2]. [PDF]: http://dev.pocoo.org/~mitsuhiko/badideas.pdf http://dev.pocoo.org/~mitsuhiko/badideas.pdf [EuroPython recording if you're a member]: https://ep2012.europython.eu/conference/talks/5-years-of-bad-ideas https://ep2012.europython.eu/conference/talks/5-years-of-bad...
- jnazario 14y ago
  i started using flask this past winter. i'm enjoying it as a lightweight framework. armin's got some good code and the reason i posted this link is that the topics and ideas are real and presently simply, useful (in some situations), and great to understand.
sirclueless 14y ago
Oh man, there is so much black magic in those slides. I hope to high heaven that as few people as possible read that. Especially not any future employers. Judging by the number of projects I have seen recently that are predicated on defining new keywords and recognizing them through magic decorators, it may be too late.
- luriel 14y ago
  This is one of the reasons I love Go: simple, clean, concise, expressive, and completely free of dark magic. Code does what it says, and says what it does. You don't get to feel so clever doing magic tricks, but you gain a lot of time by not having decipher and debug such magic tricks.
  - marcus 14y ago
    I believe in a balance, too little magic, and you're working with one hand tied behind your back, too much magic and you're working with blindfolds.
    - luriel 14y ago
      Seriously, try Go for a while, it wont take long before you feel that you have three hands, and more importantly, an extra third eye.
      - hythloday 14y ago
        Can you give an example in Go of how you'd solve the same problem as any of the python solutions?
        sirclueless 14y ago
        Well actually, people with bizarre build requirements are starting to use clean Go source code as a build target of their increasingly diverse preprocessors. Can't be healthy. [1]: https://groups.google.com/d/msg/golang-nuts/LQMv7Zsmsi0/8Aax1XSekjAJ https://groups.google.com/d/msg/golang-nuts/LQMv7Zsmsi0/8Aax... [2]: https://github.com/jteeuwen/go-bindata https://github.com/jteeuwen/go-bindata [3]: http://stackoverflow.com/questions/9838304/resource-bundling/9840851#9840851 http://stackoverflow.com/questions/9838304/resource-bundling... [4]: https://github.com/chanwit/kgc https://github.com/chanwit/kgc [5]: https://github.com/droundy/gotgo https://github.com/droundy/gotgo
      - batista 14y ago
        I use Go and I don't see any particular expressivity or ability to code easier/faster, compared to, say Python. The benefits are: type checking (if you need it), more speed (but not as much as advertised), and easy concurrency (if you need it). I think one of the main "marketing" points of Go, in a grassroots way, is that some people coming from Python/Ruby/js etc, that have never (or only very lghtly) used C/C++ before, feel suddenly like they are using a "hard, grown up's language", what with pointers and everything. It's mostly an illusion, though.
- jnazario 14y ago
  here's where i disagree with you, and here's why. there comes times - hopefully rarely - where tricks like metaclasses and monkey patching actually REDUCE overall code complexity, if done right and in a limited way. it takes a lot of experience, education, and skill to do this, but when you see it done right it greatly reduces overall code complexity by isolating the "odd" bits into small, confined spaces, and this can lead to more readable code overall. the other day a link was posted here on HN about metaclasses in python, and dpkt was one of the examples (see the _metapacket class). i think that's a good example of the above: the scary, hard to grok stuff is isolated, and overall the code becomes more readable for it. i've seen - and written - code that could benefit from these things, and it's less maintainable than code with them used appropriately. also, i think it's great if people read this and learn. i want more talented programmers on my team, people who know how to do these things, when to do them (e.g. limited), and where to do them. i've seen the opposite, and it's not pretty. that said i'm unqualified to use the "bad ideas" in this presentation appropriately, and so i wont. one facet of wisdom is knowing your limits.
seigenblues 14y ago
yikes. These are impressive (and terrifying). I particularly like the "see if you use my return value and dynamically change my behavior" trick, I bet debugging a library that used that would be fuuuun.
deleted 14y ago
[deleted]
tomrod 14y ago
Wow. I'm a numpy aficionado and connoisseur.. I'm sad to say I really don't understand most of whats going on in this code...
- irahul 14y ago
  > I'm sad to say I really don't understand most of whats going on in this code.. Don't be:) Most of the dark voodoo there is not used(macros, implicit self), some of it is used only by toolmakers(finding request, changing traceback messages, import hooks), and then a small part is used here and there by regular applications(monkey patching; "names of variables" sounds useful but it won't work with shared pool. `a = 10; b = 10; show_names(a)` will show both "a" and "b")
  - the_mitsuhiko 14y ago
    > but it won't work with shared pool. `a = 10; b = 10; show_names(a)` will show both "a" and "b") That will actually show nothing because strings are not tracked by the cyclic garbage collector because they can never be part of cycles.
    - irahul 14y ago
      My bad. I was talking about this alternate implementation: https://gist.github.com/2688055 https://gist.github.com/2688055
vosper 14y ago
Without the (assumed) accompanying presentation these slides aren't all that illuminating
notaddicted 14y ago
Everything is a first class object? What about the if-statement? Anyway, my personal gripes aside, I've been looking for something like this: http://www.scribd.com/doc/58306088/Bad-Ideas#outer_page_54 http://www.scribd.com/doc/58306088/Bad-Ideas#outer_page_54 ... a way to print "name=value" for variables for debugging.
- MaxGabriel 14y ago
  Your link just goes to the first page. Is this presumably the code you're referencing? import gc, sys def find_names(obj): frame = sys._getframe(1) while frame is not None: frame.f_locals frame = frame.f_back result = set() for referrer in gc.get_referrers(obj): if isinstance(referrer, dict): for k, v in referrer.iteritems(): if v is obj: result.add(k) return tuple(result)
  - notaddicted 14y ago
    yup, thanks for the heads up, it worked for me. I realized that most of that was unnecessary, this is what I was talking about: import gc, sys def show(*args): for obj in args: frame = sys._getframe(1) for k,v in frame.f_locals.iteritems(): if v is obj: print "%s=%r" % (k,v) a, b, c, d, e, = False, 1, [1,2,3], {'x':2 }, map show(a, b, c, d, e) outputs: a=False b=1 c=[1, 2, 3] d={'x': 2} e=<built-in function map> [ http://codepad.org/4JogU069 http://codepad.org/4JogU069 ] [ edit: slight cleanup]
    - irahul 14y ago
      Not so useful when you have shared pool. a = 10 b = 10 show(a) Or a = 'test' b = 'test' show(a)
jay_kyburz 14y ago
Hey Scribd and or Google Chrome team. I was just tricked into clicking an ad that simply said "play now" underneath the first slide. I thought it was going to start a slide show of the presentation. Scribd, if you are going to let just anybody advertise on your site you need to clearly mark what is an ad and what isn't. Google team, clicking the add opened a tab that when attempting to close opened a custom dialog with two buttons at the bottom. "leave now" and "cancel" (or something like that). The page had populated that dialog with a bunch of text and ascii art pointing to the cancel button. I didn't feel confident enough to click either buttons on this dialog because I thought be may have been some co-oped permissions dialog.I was unable to close the tab. I ended up using the task manager to kill chrome. I took the time to make a note here because it's been ages (5 years perhaps) since I had to deal with this kind of crap. Has it always been around and I just haven't seen it or is there a resurgence these days.
- packetslave 14y ago
  What made you believe this is the support site for either Scribd or Google Chrome?
  - Danieru 14y ago
    The guy who founded Scridb is here as well as tons of googlers.
    - oinksoft 14y ago
      As well as a ton of AdBlock users who'd prefer discussion of the presentation's content, and have no idea about the obtrusive adverts others mention ...
      - batista 14y ago
        So what are they doing following this particular sub-thread and giving other people condescending advice? Why aren't they busy reading other sub-threads where the presentation's content is discussed?
Mizza 14y ago
Just got a full page ad with sound from this page. Fuck Scribd. Any links to a video *or a PDF of these slides?
- endgame 14y ago
  I assume you mean a "video OR a PDF", but since it's scribd we're dealing with here, a video of the PDF would still be better.
kroger 14y ago
Here's the video of the presentation and the slides as a pdf: http://www.youtube.com/watch?v=8e0l_Dt28MQ http://www.youtube.com/watch?v=8e0l_Dt28MQ https://ep2012.europython.eu/media/conference/slides/5-years-of-bad-ideas.pdf https://ep2012.europython.eu/media/conference/slides/5-years...
downvoteme 14y ago
Here's a shell function. Pipe the scribd html page into it and, voila!, out pops .jpg urls for each page of the PDF document. Feed them to curl or whatever you program you use. Open them with your browser or your image viewer. Pipe them to ghostscript, remake the PDF to your own specs. Whatever. But for the love of God, stay away from scribd.com That site is an annoyance and I doubt it will ever improve. scrib(){ sed -n ' s,http://html.scribd.com,\ http://html.scribd.com,\ &,g; s,http://html[1-9].scribdassets.com,\ http://html[1-9].scribdassets.com,\ &,g; s,assets,,; s,pages,images,; s,jsonp,jpg, ; s,html[1-9].scribd.com,html.scribd.com,; s,pageParams.contentUrl = \",,; s,\";,,; s,<img class=\".orig=\",,; s,\"/>,,; /^$/d; /html.scribd.com.images.*jpg/p; ' |sed '/http/!d'; }
- nikcub 14y ago
  suggestion: you could make that more useful by dumping it into a gist and including a comment block on how to set it up in an rc file
RShackleford 14y ago
PYTHONNNNNNNN SUCKSSSSSSSSSSSSSSSSSSSSS, FUCK IT
jtchang 14y ago
Sometimes magic is necessary but these slides right here contain some serious voodoo. That said I've used monkey patching before which can be quite useful when you don't want to change the underlying library too much but don't want to make a complete fork. Specifically I've monkey patched methods onto the User model in Django.
- mfieldhouse 14y ago
  What is meant by Python magic? What is and isn't magic? Why would you want to use it and why might it be considered a bad idea?
  - marcus 14y ago
    My definition of magic: Magic is anything which makes other code behave in a way that would require reading the magic to understand, despite not being apparent in the magicked code. Good because it makes many things a lot easier, bad because it makes code a lot less explicit and simple and increases cognitive load (you need to always consider the magic not just the current code you're looking at).
freyrs3 14y ago
Code is here: https://github.com/mitsuhiko/badideas https://github.com/mitsuhiko/badideas

130 points by jnazario 14y ago

This presentation is by Armin Ronacher, the author (amongst many other things) of Flask, Werkzeug and Jinja2. If people find this presentation interesting he catalogues and provides slides (and recordings where appropriate) of all his previous presentations: http://lucumr.pocoo.org/talks/ http://lucumr.pocoo.org/talks/ For those who prefer straight PDF over Scribd, Armin has a PDF provided for this presentation[1]. There's also a video but it seems you need to be a member of EuroPython[2]. [PDF]: http://dev.pocoo.org/~mitsuhiko/badideas.pdf http://dev.pocoo.org/~mitsuhiko/badideas.pdf [EuroPython recording if you're a member]: https://ep2012.europython.eu/conference/talks/5-years-of-bad-ideas https://ep2012.europython.eu/conference/talks/5-years-of-bad...

jnazario 14y ago
i started using flask this past winter. i'm enjoying it as a lightweight framework. armin's got some good code and the reason i posted this link is that the topics and ideas are real and presently simply, useful (in some situations), and great to understand.

sirclueless 14y ago

Oh man, there is so much black magic in those slides. I hope to high heaven that as few people as possible read that. Especially not any future employers. Judging by the number of projects I have seen recently that are predicated on defining new keywords and recognizing them through magic decorators, it may be too late.

luriel 14y ago
This is one of the reasons I love Go: simple, clean, concise, expressive, and completely free of dark magic. Code does what it says, and says what it does. You don't get to feel so clever doing magic tricks, but you gain a lot of time by not having decipher and debug such magic tricks.
- marcus 14y ago
  I believe in a balance, too little magic, and you're working with one hand tied behind your back, too much magic and you're working with blindfolds.
  - luriel 14y ago
    Seriously, try Go for a while, it wont take long before you feel that you have three hands, and more importantly, an extra third eye.
    - hythloday 14y ago
      Can you give an example in Go of how you'd solve the same problem as any of the python solutions?
      - sirclueless 14y ago
        Well actually, people with bizarre build requirements are starting to use clean Go source code as a build target of their increasingly diverse preprocessors. Can't be healthy. [1]: https://groups.google.com/d/msg/golang-nuts/LQMv7Zsmsi0/8Aax1XSekjAJ https://groups.google.com/d/msg/golang-nuts/LQMv7Zsmsi0/8Aax... [2]: https://github.com/jteeuwen/go-bindata https://github.com/jteeuwen/go-bindata [3]: http://stackoverflow.com/questions/9838304/resource-bundling/9840851#9840851 http://stackoverflow.com/questions/9838304/resource-bundling... [4]: https://github.com/chanwit/kgc https://github.com/chanwit/kgc [5]: https://github.com/droundy/gotgo https://github.com/droundy/gotgo
    - batista 14y ago
      I use Go and I don't see any particular expressivity or ability to code easier/faster, compared to, say Python. The benefits are: type checking (if you need it), more speed (but not as much as advertised), and easy concurrency (if you need it). I think one of the main "marketing" points of Go, in a grassroots way, is that some people coming from Python/Ruby/js etc, that have never (or only very lghtly) used C/C++ before, feel suddenly like they are using a "hard, grown up's language", what with pointers and everything. It's mostly an illusion, though.
jnazario 14y ago
here's where i disagree with you, and here's why. there comes times - hopefully rarely - where tricks like metaclasses and monkey patching actually REDUCE overall code complexity, if done right and in a limited way. it takes a lot of experience, education, and skill to do this, but when you see it done right it greatly reduces overall code complexity by isolating the "odd" bits into small, confined spaces, and this can lead to more readable code overall. the other day a link was posted here on HN about metaclasses in python, and dpkt was one of the examples (see the _metapacket class). i think that's a good example of the above: the scary, hard to grok stuff is isolated, and overall the code becomes more readable for it. i've seen - and written - code that could benefit from these things, and it's less maintainable than code with them used appropriately. also, i think it's great if people read this and learn. i want more talented programmers on my team, people who know how to do these things, when to do them (e.g. limited), and where to do them. i've seen the opposite, and it's not pretty. that said i'm unqualified to use the "bad ideas" in this presentation appropriately, and so i wont. one facet of wisdom is knowing your limits.

seigenblues 14y ago

yikes. These are impressive (and terrifying). I particularly like the "see if you use my return value and dynamically change my behavior" trick, I bet debugging a library that used that would be fuuuun.

deleted 14y ago

[deleted]

tomrod 14y ago

Wow. I'm a numpy aficionado and connoisseur.. I'm sad to say I really don't understand most of whats going on in this code...

irahul 14y ago
> I'm sad to say I really don't understand most of whats going on in this code.. Don't be:) Most of the dark voodoo there is not used(macros, implicit self), some of it is used only by toolmakers(finding request, changing traceback messages, import hooks), and then a small part is used here and there by regular applications(monkey patching; "names of variables" sounds useful but it won't work with shared pool. `a = 10; b = 10; show_names(a)` will show both "a" and "b")
- the_mitsuhiko 14y ago
  > but it won't work with shared pool. `a = 10; b = 10; show_names(a)` will show both "a" and "b") That will actually show nothing because strings are not tracked by the cyclic garbage collector because they can never be part of cycles.
  - irahul 14y ago
    My bad. I was talking about this alternate implementation: https://gist.github.com/2688055 https://gist.github.com/2688055

vosper 14y ago

Without the (assumed) accompanying presentation these slides aren't all that illuminating

notaddicted 14y ago

Everything is a first class object? What about the if-statement? Anyway, my personal gripes aside, I've been looking for something like this: http://www.scribd.com/doc/58306088/Bad-Ideas#outer_page_54 http://www.scribd.com/doc/58306088/Bad-Ideas#outer_page_54 ... a way to print "name=value" for variables for debugging.

MaxGabriel 14y ago
Your link just goes to the first page. Is this presumably the code you're referencing? import gc, sys def find_names(obj): frame = sys._getframe(1) while frame is not None: frame.f_locals frame = frame.f_back result = set() for referrer in gc.get_referrers(obj): if isinstance(referrer, dict): for k, v in referrer.iteritems(): if v is obj: result.add(k) return tuple(result)
- notaddicted 14y ago
  yup, thanks for the heads up, it worked for me. I realized that most of that was unnecessary, this is what I was talking about: import gc, sys def show(*args): for obj in args: frame = sys._getframe(1) for k,v in frame.f_locals.iteritems(): if v is obj: print "%s=%r" % (k,v) a, b, c, d, e, = False, 1, [1,2,3], {'x':2 }, map show(a, b, c, d, e) outputs: a=False b=1 c=[1, 2, 3] d={'x': 2} e=<built-in function map> [ http://codepad.org/4JogU069 http://codepad.org/4JogU069 ] [ edit: slight cleanup]
  - irahul 14y ago
    Not so useful when you have shared pool. a = 10 b = 10 show(a) Or a = 'test' b = 'test' show(a)

jay_kyburz 14y ago

Hey Scribd and or Google Chrome team. I was just tricked into clicking an ad that simply said "play now" underneath the first slide. I thought it was going to start a slide show of the presentation. Scribd, if you are going to let just anybody advertise on your site you need to clearly mark what is an ad and what isn't. Google team, clicking the add opened a tab that when attempting to close opened a custom dialog with two buttons at the bottom. "leave now" and "cancel" (or something like that). The page had populated that dialog with a bunch of text and ascii art pointing to the cancel button. I didn't feel confident enough to click either buttons on this dialog because I thought be may have been some co-oped permissions dialog.I was unable to close the tab. I ended up using the task manager to kill chrome. I took the time to make a note here because it's been ages (5 years perhaps) since I had to deal with this kind of crap. Has it always been around and I just haven't seen it or is there a resurgence these days.

packetslave 14y ago
What made you believe this is the support site for either Scribd or Google Chrome?
- Danieru 14y ago
  The guy who founded Scridb is here as well as tons of googlers.
  - oinksoft 14y ago
    As well as a ton of AdBlock users who'd prefer discussion of the presentation's content, and have no idea about the obtrusive adverts others mention ...
    - batista 14y ago
      So what are they doing following this particular sub-thread and giving other people condescending advice? Why aren't they busy reading other sub-threads where the presentation's content is discussed?

Mizza 14y ago

Just got a full page ad with sound from this page. Fuck Scribd. Any links to a video *or a PDF of these slides?

endgame 14y ago
I assume you mean a "video OR a PDF", but since it's scribd we're dealing with here, a video of the PDF would still be better.

kroger 14y ago

Here's the video of the presentation and the slides as a pdf: http://www.youtube.com/watch?v=8e0l_Dt28MQ http://www.youtube.com/watch?v=8e0l_Dt28MQ https://ep2012.europython.eu/media/conference/slides/5-years-of-bad-ideas.pdf https://ep2012.europython.eu/media/conference/slides/5-years...

downvoteme 14y ago

Here's a shell function. Pipe the scribd html page into it and, voila!, out pops .jpg urls for each page of the PDF document. Feed them to curl or whatever you program you use. Open them with your browser or your image viewer. Pipe them to ghostscript, remake the PDF to your own specs. Whatever. But for the love of God, stay away from scribd.com That site is an annoyance and I doubt it will ever improve. scrib(){ sed -n ' s,http://html.scribd.com,\ http://html.scribd.com,\ &,g; s,http://html[1-9].scribdassets.com,\ http://html[1-9].scribdassets.com,\ &,g; s,assets,,; s,pages,images,; s,jsonp,jpg, ; s,html[1-9].scribd.com,html.scribd.com,; s,pageParams.contentUrl = \",,; s,\";,,; s,<img class=\".orig=\",,; s,\"/>,,; /^$/d; /html.scribd.com.images.*jpg/p; ' |sed '/http/!d'; }

nikcub 14y ago
suggestion: you could make that more useful by dumping it into a gist and including a comment block on how to set it up in an rc file

RShackleford 14y ago

PYTHONNNNNNNN SUCKSSSSSSSSSSSSSSSSSSSSS, FUCK IT

jtchang 14y ago

Sometimes magic is necessary but these slides right here contain some serious voodoo. That said I've used monkey patching before which can be quite useful when you don't want to change the underlying library too much but don't want to make a complete fork. Specifically I've monkey patched methods onto the User model in Django.

mfieldhouse 14y ago
What is meant by Python magic? What is and isn't magic? Why would you want to use it and why might it be considered a bad idea?
- marcus 14y ago
  My definition of magic: Magic is anything which makes other code behave in a way that would require reading the magic to understand, despite not being apparent in the magicked code. Good because it makes many things a lot easier, bad because it makes code a lot less explicit and simple and increases cognitive load (you need to always consider the magic not just the current code you're looking at).

freyrs3 14y ago

Code is here: https://github.com/mitsuhiko/badideas https://github.com/mitsuhiko/badideas