5 ms·

Ask HN: Anyone using Python 3 in production? (2011 Edition)

30 points by SandB0x 15y ago

This question was asked nearly a year ago here: http://news.ycombinator.com/item?id=1464199 Is anyone now using Python 3 in production? Would you use it if you were starting over? This was discussed briefly at Google I/O this week: http://www.youtube.com/watch?v=KKQS8EDG1P4&feature=player_detailpage#t=2554s key slide: Python 3: next generation Do this if completely new Python 2: current version Most books/tutorials still here Do this if have existing software Bottom line: either is fine

kwantam 15y ago
If you're not using anyone P3k-incompatible libraries, and if you don't anticipate doing so in the future, and if you think you need something that is offered in 3.x that 2.7 doesn't offer, then it seems reasonable to go with 3.x. For example, the 3.2+ GIL is far improved (and will not be backported, according to the bug tracker), so if you're using threads in Python, that's a good reason to go that way. My impression is that most people can't really say with certainty that they'll never want to use an incompatible library in the future, so they'll stick with 2.7.x. I just started a new project, went through this decision tree, and ended up at 2.7. Game theory probably has something to say about the individual decision to defect (stay with 2.7 because it's more work to use 3.x) versus the public good that would come of using 3.x (i.e., more people use it, so library writers have more reason to support it, so more people can use it because it's better supported). Perhaps someone with more knowledge in this area can lend some insight into rearranging the incentives to make 3.x happen faster, too.
- SandB0x 15y ago
  Thanks for the reply. I'm kind of torn. It would be nice to switch and forget about it, but Python is the language that works with absolutely everything. That's the biggest attraction. Libraries for everything, bindings for everything, APIs for everything. There are always purer and/or faster languages⁺, so why Python 3 over those if compatibly is poor? ⁺"There are only two kinds of languages: the kind everybody bitches about, and the kind nobody uses."
  - lvh 15y ago
    The dichtomy is false. Write good 2.x code and you will only miss a few very specific (like nonlocal) or recent features. You get the benefit of using all the existing 2.x libraries. Run 2to3 on your codebase and run your unit test suite, and see what happens. That way, once your dependencies are ported, you can be fairly sure that porting your library will be a walk in the park.
- d0m 15y ago
  The ideal would be to be able to mix both versions. Use python3 by default and, by specifying an option somewhere, import from an older version. I.e. import-2.7 whatever-lib I know about features that aren't compatible (print as a function for instance), but what about python's bytecode.. ?
  - hartror 15y ago
    I would hope/expect that to be even more incompatible than the syntax given the architectural changes made!
dgallagher 15y ago
The only thing that's keeping me from switching is Django (currently 1.3), which is on Python 2.5-2.7. It might be some time before they migrate to Python 3. Alex Gaynor mentioned that he'd like to work on porting Django to Python 3 this summer, noting that an end-of-summer port is not an official position of the Django developers (2nd link): http://alexgaynor.net/2009/nov/28/django-and-python-3/ http://alexgaynor.net/2009/nov/28/django-and-python-3/ http://alexgaynor.net/2011/feb/17/django-and-python-3-take-2/ http://alexgaynor.net/2011/feb/17/django-and-python-3-take-2... I don't believe Django has officially announced when they will move to Python 3, or what version of Django that'll occur with. Someone please correct me if I'm wrong. Currently though, I'm not sure if Django "needs" Python 3 all that badly. Eventually yes, but not necessarily right now; Python 2 is still quite popular.
jnoller 15y ago
This still applies: http://wiki.python.org/moin/Python2orPython3 http://wiki.python.org/moin/Python2orPython3 Same story: Porting is proceeding as planned, numpy, scipy, matplotlib, etc are all coming. Django is coming, etc.
hartror 15y ago
We're like a bunch of penguins crowded at the edge of the ice waiting to see if the first few to dive in get eaten by a sea lion or not. I for one am happy on my 2.7 perch as while I would love to get my teeth suck into 3.x I have to choose the productivity of full library support over engineering joy.
lvh 15y ago
I regularly test my code that doesn't have 2.x-specific dependencies using tox+2to3, a great tool written by Holger Krekel that lets me test code in a wide variety of contexts. Everything I run in production is 2.x. Two specific comments to things said in the thread: 1) NewGIL is an improvement, but it really optimizes a situation you probably shouldn't have been in, given gevent/twisted/eventlet, or hell, even Tornado. If you're doing a bunch of _computation_ in threads, they are only affected in a very small way. If you're doing IO, it can have a very noticeable effect on some workloads. However, if you've followed any of my previous rants on why you should avoid threads as an IO primitive, you'll notice that performance and the GIL are but a footnote. I don't mean to downplay NewGIL, it's an interesting piece of software engineering. Just please don't see it as the panacea that magically fixed threads, because it isn't. 2) In the opening slide: "do this if completely new" translates to me as "don't do this". You're already using Python, which is a massive layer of abstraction in so many ways. Everyone using Python appreciates this as mostly a good thing, I would assume. So yes -- if no 2.x-only libraries exist that make your life easier and/or you're not willing to spend time porting them, go for it. I have just never found that to be true.

30 points by SandB0x 15y ago

kwantam 15y ago

If you're not using anyone P3k-incompatible libraries, and if you don't anticipate doing so in the future, and if you think you need something that is offered in 3.x that 2.7 doesn't offer, then it seems reasonable to go with 3.x. For example, the 3.2+ GIL is far improved (and will not be backported, according to the bug tracker), so if you're using threads in Python, that's a good reason to go that way. My impression is that most people can't really say with certainty that they'll never want to use an incompatible library in the future, so they'll stick with 2.7.x. I just started a new project, went through this decision tree, and ended up at 2.7. Game theory probably has something to say about the individual decision to defect (stay with 2.7 because it's more work to use 3.x) versus the public good that would come of using 3.x (i.e., more people use it, so library writers have more reason to support it, so more people can use it because it's better supported). Perhaps someone with more knowledge in this area can lend some insight into rearranging the incentives to make 3.x happen faster, too.

SandB0x 15y ago
Thanks for the reply. I'm kind of torn. It would be nice to switch and forget about it, but Python is the language that works with absolutely everything. That's the biggest attraction. Libraries for everything, bindings for everything, APIs for everything. There are always purer and/or faster languages⁺, so why Python 3 over those if compatibly is poor? ⁺"There are only two kinds of languages: the kind everybody bitches about, and the kind nobody uses."
- lvh 15y ago
  The dichtomy is false. Write good 2.x code and you will only miss a few very specific (like nonlocal) or recent features. You get the benefit of using all the existing 2.x libraries. Run 2to3 on your codebase and run your unit test suite, and see what happens. That way, once your dependencies are ported, you can be fairly sure that porting your library will be a walk in the park.
d0m 15y ago
The ideal would be to be able to mix both versions. Use python3 by default and, by specifying an option somewhere, import from an older version. I.e. import-2.7 whatever-lib I know about features that aren't compatible (print as a function for instance), but what about python's bytecode.. ?
- hartror 15y ago
  I would hope/expect that to be even more incompatible than the syntax given the architectural changes made!

dgallagher 15y ago

The only thing that's keeping me from switching is Django (currently 1.3), which is on Python 2.5-2.7. It might be some time before they migrate to Python 3. Alex Gaynor mentioned that he'd like to work on porting Django to Python 3 this summer, noting that an end-of-summer port is not an official position of the Django developers (2nd link): http://alexgaynor.net/2009/nov/28/django-and-python-3/ http://alexgaynor.net/2009/nov/28/django-and-python-3/ http://alexgaynor.net/2011/feb/17/django-and-python-3-take-2/ http://alexgaynor.net/2011/feb/17/django-and-python-3-take-2... I don't believe Django has officially announced when they will move to Python 3, or what version of Django that'll occur with. Someone please correct me if I'm wrong. Currently though, I'm not sure if Django "needs" Python 3 all that badly. Eventually yes, but not necessarily right now; Python 2 is still quite popular.

jnoller 15y ago

This still applies: http://wiki.python.org/moin/Python2orPython3 http://wiki.python.org/moin/Python2orPython3 Same story: Porting is proceeding as planned, numpy, scipy, matplotlib, etc are all coming. Django is coming, etc.

hartror 15y ago

We're like a bunch of penguins crowded at the edge of the ice waiting to see if the first few to dive in get eaten by a sea lion or not. I for one am happy on my 2.7 perch as while I would love to get my teeth suck into 3.x I have to choose the productivity of full library support over engineering joy.

lvh 15y ago

I regularly test my code that doesn't have 2.x-specific dependencies using tox+2to3, a great tool written by Holger Krekel that lets me test code in a wide variety of contexts. Everything I run in production is 2.x. Two specific comments to things said in the thread: 1) NewGIL is an improvement, but it really optimizes a situation you probably shouldn't have been in, given gevent/twisted/eventlet, or hell, even Tornado. If you're doing a bunch of _computation_ in threads, they are only affected in a very small way. If you're doing IO, it can have a very noticeable effect on some workloads. However, if you've followed any of my previous rants on why you should avoid threads as an IO primitive, you'll notice that performance and the GIL are but a footnote. I don't mean to downplay NewGIL, it's an interesting piece of software engineering. Just please don't see it as the panacea that magically fixed threads, because it isn't. 2) In the opening slide: "do this if completely new" translates to me as "don't do this". You're already using Python, which is a massive layer of abstraction in so many ways. Everyone using Python appreciates this as mostly a good thing, I would assume. So yes -- if no 2.x-only libraries exist that make your life easier and/or you're not willing to spend time porting them, go for it. I have just never found that to be true.