13 ms·

Where are we in the Python 3 transition?

110 points by joeyespo 11y ago

TheCowboy 11y ago
One frustrating part of attempting to transition is that not all libraries make it clear which Python versions are supported. Usually it's Python 2-only code that isn't clearly labeled as such, but it happens with Python 3 code as well. Sometimes a library doesn't work under specific versions of Python 3. The result is that library research takes longer than it would. Please include this in the README for any project, even if it seems obvious.
sitkack 11y ago
I moved last month and I have been using Python since 1.5.2 What prevents me from recommending it to everyone I pass on the street is that the PyPy 3.x line is back on 3.2.5 which makes it not compatible with many libs which have a 3.3 minimum, which is most. Yes, I give to pypy.org http://pypy.org/py3donate.html http://pypy.org/py3donate.html Recommendations to folks * switch your daily interpreter to Python3 * use tox, https://pypi.python.org/pypi/tox * use six, https://pypi.python.org/pypi/six * when you go native, go cffi, https://pypi.python.org/pypi/cffi
- PeCaN 11y ago
  Why's this downvoted, it seems like useful advice to me. Also cffi is definitely awesome.
- eiopa 11y ago
  This kills it for me too. I rely on pypy, and I just can't see myself sacrificing my productivity by using Six. Also, Python 3 neglected to fix one of the most annoying things about the language - default arg value def foo(x=[]): x.append(1) print x foo() # 1 foo() # 1 1 Why is this still busted??
  - rspeer 11y ago
    Because nobody's made a Python Enhancement Proposal for what should happen instead.
  - makecheck 11y ago
    It may be an easy mistake to make but using None is more consistent and efficient to detect. You can use "if x is None:" for any type; whereas, a function may or may not want to allow empty strings (or even empty lists) and it is easy to use a None check to decide when to apply the desired default.
  - nbadg 11y ago
    Are you referring persistence of mutable default args? Which then potentially leads to bugs when you treat it as a new variable on each function call? This is intentional behavior. It's the result of one-time evaluation of default args, which is important for memoization. It's also really useful in combination with late-binding closures. For example, using a lambda as a generator function: def create_multipliers(): return [lambda x : i * x for i in range(5)] This doesn't work, you'll get all 8's. Instead you need to: def create_multipliers(): return [lambda x, i=i : i * x for i in range(5)] Late-binding closures and memoization strategies are pretty core language features; I wouldn't expect them to change (and many python devs would be pretty pissed if they did). Yes, this can be confusing with mutable default objects, but the alternative is to have disparate behavior depending on the mutability of the defaults, which would be an utter catastrophe. http://docs.python-guide.org/en/latest/writing/gotchas/ http://docs.python-guide.org/en/latest/writing/gotchas/
    - alextgordon 11y ago
      Other languages seem to get it right. Even C++! #include <vector> static int foo(std::vector<int> x = {}) { x.push_back(10); return x.size(); } int main (int argc, char const *argv[]) { printf("%d\n", foo()); // 1 printf("%d\n", foo()); // still 1 return 0; } def foo(x=[]): x.append(10) return len(x) print(foo()) # 1 print(foo()) # 2? wtf?
      - eiopa 11y ago
        ES6 got this right too. This design decision makes me die a little every time I have to do: def foo(x=None): if not x: x = [] ...
        dajonker 11y ago
        That will get you into trouble as well, please use: if x is None: x = [] ... etc.
  - throwaway91919 11y ago
    Can you (or someone) clarify what problem you're talking about? This seems to work as expected for me: def foo(x=[]): sum = 0 for a in x: sum += a return sum print foo([3,5]) # 8 print foo(x=[3]) # 3 print foo([]) # 0 print foo() # 0 Is it the fact that kwargs are required to have defaults? Or what?
    - raverbashing 11y ago
      Nah, do it like this def foo(i,x=[]): x.append(i) return x foo(1) foo(2) See what happens
    - saltylicorice 11y ago
      The problem is that the default argument is mutable. def f(a=[]): a.append('v') print a f(['ok']) # ['ok', 'v'] f() # ['v'] f() # ['v', 'v']
      - throwaway91919 11y ago
        Ah, thanks.
  - deleted 11y ago
    [deleted]
wh-uws 11y ago
As much flak as ruby has gotten for its performance issues and "magic" (read: heavy preference for use of meta programming) ... One thing I think the ruby core team has done a great job of in my opinion is always having some awesome new features in the next major release that make you say "hey thats awesome" and make you want to upgrade despite any headache it might cause and the headaches are generally minimal. Its been interesting to see python so publicly and painfully fail at that. A whole new language version release that as far as I can tell only changed internal plumbing and went so far as to make you have rewrite print statements? I just don't understand how they could think people would want to move to that.
- lyschoening 11y ago
  Maybe Python 2 to Python 3.0 didn't introduce many "awesome" features, but 3.4 and 3.5 have been great.
  - AnkhMorporkian 11y ago
    asyncio and pathlib are absolutely killer features for me. I never much cared about Python 3 until asyncio came on the field. As soon as I saw some example code Guido had written, I started using tulip, and have been a willing participant on the wild ride it has taken.
    - rspeer 11y ago
      Pathlib lets me down a bit -- I find myself having to str() the Path objects and lose their benefits, because the rest of the standard library doesn't use pathlib. But asyncio is great. It shows how much better asynchronous programming can be when you have support from the language itself. I can't imagine going back to a byzantine system like Twisted even if they did manage to finish their Python 3 port.
x1024 11y ago
Wait, Python 3 has existed since 2008 and the transition is at 20%? How is that any form of success?
- MatthewWilkes 11y ago
  Backwards incompatible language changes having not killed the community and gaining traction after 7 years is pretty good going.
- craigds 11y ago
  He said that 20% have reached "Acceptance", the last of the 5 stages, and said that most people are in stage 4 (Depression). That'd put overall progress at more like 80%. This article also wasn't actually talking about adoption of python 3, just applying the stages of grief to how people feel about it. IOW, adoption of python 3 might still be worse than 20%, but 20% have accepted that they will need to adopt it soon. Anecdotally as a full time python guy, I'd agree with the OP. The number of people interested in Python 3 has been growing a bit every year, and it seems opposition to adopting it at all has finally mostly dwindled :)
  - kzrdude 11y ago
    I still find reasons to be pessimistic. OpenCV was really really late to support Py 3. Every pypy release focuses on Py 2 only (and Py 3 releases are an afterthought). Really weird prioritization for me (I'm not interested in legacy-focused projects).
  - vacri 11y ago
    If you read the wiki link in the article, the 'stages of grief' aren't stages that you progress through. They're different ways of dealing with loss, and you won't necessarily hit all of them or do them in any particular order.
    - claystu 11y ago
      More importantly, these stages have nothing to do with choosing either Python 2 or Python 3. Python is just a tool. If one doesn't work, you choose another, and if the developers of that tool decide to screw it up, you just move on.
captainmuon 11y ago
If someone would make a Python 2.8 (or call it Python 5=2+3 if you like), I would pay for it. It must run basically every python 2.7 (or 2.6) program without changes. It should backport as many features as possible. The meaning of "str" and string literals (bytes or unicode) should be configurable on a per-file basis. It should have a print statement. Finally, it should be able to use all the binary modules from 2.7 (maybe after recompilation). Currently at our work (in physics research), I have the feeling that people are giving up on Python. It's not worth the trouble to port to python 3 - especially because we can't switch all code at once; we'd have to support both versions for years. C++ which is also heavily used here is getting much nicer, with proper tooling it is even more pleasant to work in (code completion etc). We are mostly using Python for a) configuration files and b) quick and dirty scripts. For these cases, Python 2.7 will be good enough forever. If someone does something more complex, they'd probably use Python 3 - but they treat it like it's a completely different language rather than a new version of Python 2.
- SpaceManNabs 11y ago
  Likewise, one of my previous advisors has stopped doing any quick and dirty math in python. I am seeing Julia a lot more (for astrophysics and ML) and more C++ (physics) for the more important simulations.
  - rspeer 11y ago
    Julia seems like an odd thing to bring up on the topic of "avoiding backward-incompatible language changes".
- jamesdutc 11y ago
  You'd have to pay a lot. Jokes of mine like https://github.com/dutc/rwatch https://github.com/dutc/rwatch and https://github.com/dutc/didyoumean https://github.com/dutc/didyoumean and https://bitbucket.org/dutc/astexpr https://bitbucket.org/dutc/astexpr are examples of adding major features to the CPython interpreter at run-time. It works fairly well in practice. It ends up looking like a `__future__`-import pragma that has interpreter lifetime scope rather than file scope. `__future__` imports cannot be undone within the same file -- after all, they're pragmas scoped at the file level, not actual imports -- and it would make sense for these backported features to behave similarly, though at the scope of the interpreter's lifetime. That might not do what you want. As you'd expect, the most superficial of Python 3's new features (syntax changes, `raise from`, &c.) would be easiest to backport. The bulk of the difficulty would be in merging those changes into Python 2.7. Making the result hot-loadable would be just as easy as in `dutc-rwatch`, though being able to toggle features on-and-off independently would require combinatoric effort with the naïve approach. You'd need to do something smarter. All in all, it's wholly possible to do what you want, and you probably have the skill to even do it yourself. I'd happily do the work myself, but only for lots of money. Unfortunately, the burden of both certifying the resulting interpreter as correct and of maintaining this fork would be large. And then when you're on-boarding some fresh new post-grad you've just hired into your research group: "Oh, we use Python 2.8 here. That's what we call Python 2.7 with an unsupported collection of modules that mutate the currently running interpreter to backport features from Python 3. Don't worry; it only breaks in ways that no one outside of this office will ever have a chance of ever understanding or helping you debug."
craigds 11y ago
Personally, I can't wait to switch to python 3. I'm glad most of the community is finally getting behind it. The cogs move slowly. We've only just upgraded our huge codebase to 2.7, but I expect we'll be on some release of 3.x in about two years. Almost all of our big list of dependencies now support 3.3+, so there is little reason to not upgrade.
foobar2357 11y ago
The Python team is blind to the huge number of users that will never switch to Python 3. My industry (a large one) is currently standardized on RHEL 6 (and the CentOS equivalent), which uses Python 2.6! I'm not generally able to install arbitrary software or demand upgrades without a lot of beurocracy. I was looking forward to the day when we move to RHEL 7 with Python 2.7. I've started to view the lack of changes in 2.7 as a feature. I would value the stability and lack of breaking changes. Knowing that Python 2.7 is on a limited timeline, and that the Python 3 folks (users and developers) are trying to kill off 2.7, I can't think of any reason not to switch to another language. In short, my field is nowhere near "acceptance" of Python 3, and the "anger" stage will just make us (me at least) leave Python behind. I will never want Python 3.x, and I don't even want Python 2.8. I wish people would recognize that stability is a virtue for some of us...
- mkesper 11y ago
  The investment in porting your code from Python2 to Python3 should be many times smaller than porting to an entirely new language ecosystem.
  - dimva 11y ago
    Yea, but what's to stop Python 4 from doing the same thing as Python 3? It's clear from the tone of this article that the python core devs haven't learned the value of backwards compatibility even after their huge python 3 failure. The author makes it seem like the problem preventing python 3 adoption was python users' unwillingness to accept the inevitable, rather than the poor migration path provided (especially initially). In a massive, old production codebase, upgrading your platform is incredibly risky, even if no code changes are necessary. But when you have to make code changes in a dynamic language... The thing is, not all systems have tests. Not all systems even have known, defined behavior. Upgrading these things can be a multi-month, very risky project. Given that python 3's backwards incompatible changes were made for purely cosmetic reasons and there are few compelling reasons to upgrade, I totally get the anger people feel. I work in startups with relatively new codebases and good test coverage, but I can empathize with other types of organizations. I'm shocked that python devs don't, even after 7 years.
rdtsc 11y ago
> While some people feel stuck in Python 2 at work and are "depressed" over it, others have reached the point of having transitioned their projects and accepted Python 3, both at work and in personal projects. It is not as simple. I would be simpler if it was 2006 again. Python would have less competition on being an easy to use language, batteries included, with good enough performance (SMP wasn't as popular then). Java had a bad perception -- too bloated, enterprise-y, Javascript on the server wasn't happening much. Go wasn't there. Scala not as popular. In that context, people would eventually have switched to Python 3 even if it took some pain (security maintenance, some newer features etc), because GIL wasn't a big deal and it sure wasn't going to be Perl again (maybe Ruby for webdev?). The situation is more difficult today -- Python has more serious competition. Once a team has found the time/money/excitement to switch to Python 3, it is a high chance they'll start looking and evaluating other languages/platforms/libraries (and I speak as a person who loves Python and used it professionally for 12+ years). It is now competing with Go, Elixir, Javascript, Scala, Julia, Clojure, even Rust, Java, and C++14. No matter how cool Python 3 is and what new features it brings, it will have a harder battle to fight just because of the current environment.
- criddell 11y ago
  > It is now competing with Go, Elixir, Javascript, Scala, Julia, Clojure, even Rust, Java, and C++14. That would have happened no matter what. It's not really a bad thing, is it? I can't see how something better coming along is anything but great news.
upofadown 11y ago
>With five years left until people will need to pay for Python 2 support, ... Is this some sort of thing?
- kemayo 11y ago
  It's just the author pointing out that the EOL for official support for Python 2 is 2020. After then, the Python dev team won't be providing any (free) support, so people who want to keep using it will have to pay for it in some way -- whether by investing time in maintaining a fork, or by paying someone else to do so. The absence of a "Python 2.8" fork suggests that the idea of forking hasn't been popular enough to happen organically without someone putting money in, after all.
  - someguydave 11y ago
    >The absence of a "Python 2.8" fork suggests that the idea of forking hasn't been popular enough to happen organically without someone putting money in, after all. Or it's evidence that python 2.7 works just fine for many people.
- Ded7xSEoPKYNsDd 11y ago
  Cent OS 7 will support Python 2 for free until 2024. I would assume most patches will work without changes on other OSes as well. But if you want someone to do the QA work (and compiling) for you and no independent Python 2 fork will arise, you'd presumably have to pay someone for that. Beyond that, apparently you can pay Red Hat to big bucks to provide you with bug fixes for really old Red Hat releases in a similar way you can pay Microsoft big bucks to get patches for Windows XP even today. I'm sure other companies will do the same if they can find enough customers that find porting costs more than such a support contract.
PhantomGremlin 11y ago
There was a Simpsons episode[1] back in 1991 in which Homer, after eating a poisonous fugu fish, must deal with the five stages. It's an all time classic: Dr. Julius Hibbert: Now, a little death anxiety is normal. You can expect to go through five stages. The first is denial. Homer Simpson: No way, because I'm not dying! Dr. Julius Hibbert: Second is anger. Homer Simpson: [furiously] Why you little... ! Dr. Julius Hibbert: After that comes fear. Homer Simpson: [worried] What's after fear? What's after fear? Dr. Julius Hibbert: Bargaining. Homer Simpson: Doc, you gotta get me outta this. I'll make it worth your while. Dr. Julius Hibbert: Finally acceptance. Homer Simpson: Well, we all gotta go sometime. Dr. Julius Hibbert: Mr. Simpson, your progress astounds me. If only this Python transition were so simple. [1] https://en.wikipedia.org/wiki/One_Fish,_Two_Fish,_Blowfish,_Blue_Fish https://en.wikipedia.org/wiki/One_Fish,_Two_Fish,_Blowfish,_...
Walkman 11y ago
Extended the maintenance timeline from 5 to 10 years is not helping the transition at all.
- someguydave 11y ago
  The extended maintenance timeline is an implicit admission that nobody wants to hop on their wagon.
tonetheman 11y ago
I am on CentOS and honestly for most of our applications there is just no compelling reason to change. We can do everything with 2 that we need. I have seen some async stuff that looks interesting in 3 but not enough to make me change.
cdnsteve 11y ago
Can someone explain aiohttp server vs wsgi? It seems wsgi is the old standard but now you can run a server right in python, kind of like node. Are people running aiohttp instead of wsgi? It seems like aiohttp can do everything that Flask can do. Are more people using this?
- RodericDay 11y ago
  I want to know too. I did a little toy project with websockets in aiohttp and it was extremely enjoyable. It'd be reassuring to know that other people see it as "the future" too, so I could invest into it more heavily.
- et1337 11y ago
  I've used aiohttp for a few things. Works fine, small, lightweight, easy to use.
jacquesm 11y ago
The python 2.x to 3.x change will be studied for many years. How to completely rob an ecosystem of its momentum in one fell swoop. I still don't understand why 3 couldn't have had a mode switch defaulting to 2.x behavior. The whole debacle would have been avoided and all the ugly details would have been hidden behind the language implementation. It would have definitely been more work and you might even call it an ugly hack (and Ed DeCastro would likely have thrown a fit if one had asked his opinion) but it would have at least made the transition instant and seamless. The present mess will - if not resolved in a much more drastic fashion - cause more and more teams to switch away from python. It's a real waste of the potential that python had (and still has). I never even bothered porting my code, I'm 'stuck' on 2.7 and anything new gets written in another language. Even bloody PHP, the language everybody loves to hate on, gets deprecation and backwards compatibility better than python.
- gravypod 11y ago
  Sadly I have to agree with you. Some things that changed were not even necessary. Why would the development team make replacement packages rather than fixing underlying implementation, for example imp and importlib. There was also no sugar added to support this mangled jungle gym of package replacements. Backwards compatibility is a thing of dreams for the python developer. This is only one of many of the problems with the change.
- msellout 11y ago
  > gets ... backwards compatibility That should be past tense, "got" as the Python community has certainly learned its lesson. Luckily it wasn't as bad a mistake as the Perl folks made.
  - jacquesm 11y ago
    > That should be past tense, "got" as the Python community has certainly learned its lesson. Evidence of that to me would be a proposed fix and maybe I'm too out of touch with what's happening in the python world but from what I see the two main version still co-exist rather than that any attempt at unification has been made. Am I missing something major here? My understanding is that you migrate your project to 3.x or you're stuck in the past. > Luckily it wasn't as bad a mistake as the Perl folks made. That's up for debate.
Walkman 11y ago
> I have yet to hear from someone who has used Python 3 that they think it is a worse language than Python 2 the maker of the most succesful Python projects: Armin Ronacher
- mianos 11y ago
  Just ask him about unicode for LOLs.
joseph8th 11y ago
I was in denial stage until we picked Python 3 for a project at work. Then I went into anger and bargaining with the other dev on the project. Heh. blush Being forced to come to terms with Python 3 has made me see the light, though. I still have a host of personal projects in Python 2, but anything new I write is in 3. For me, it was concurrent.futures that pushed me into acceptance stage.
- macavity23 11y ago
  My experience was similar. An even bigger push in my case was the async stuff in 3.5. It's going to be a little while before pypy supports 3.5, but once that happens, Python will be very well placed. Something like Tornado + pypy + py3.5 looks a lot like the app server stack of the future IMHO. It has been a massive PITA, and probably could have been better handled, but hindsight is easy. The future looks bright.
AstroChimpHam 11y ago
I'm a long time Python user and I still tend to go for 2.7 over 3. When writing packages, I'll add support for 3 as an afterthought, but that's it. The main reason for this and the main reason I prefer Python to other languages is the excellent and complete ecosystem of libraries. I've never seen a single Python library I wanted to use in the last 8 years that didn't have support for Python 2.7. I still run into those that don't support Python 3 well. So, using Python 3 means being terrified that I run into a case later on where I need to use some package that only supports 2.7 and I get stuck. With all of the other potential problems to run into while coding, why would I want to add that extra one, especially for something work-related? I imagine this will continue being an issue until some major packages stop supporting Python 2.7 entirely and force people to make a choice.
51109 11y ago
From LearnPythonTheHardWay.org: > A programmer may try to get you to install Python 3 and learn that. Say, "When all of the Python code on your computer is Python 3, then I'll try to learn it." That should keep them busy for about 10 years. I repeat, do not use Python 3. Python 3 is not used very much, and if you learn Python 2 you can easily learn Python 3 when you need it. If you learn Python 3 then you'll still have to learn Python 2 to get anything done. Just learn Python 2 and ignore people saying Python 3 is the future. Besides, judging from the number of DARPA-sponsored projects that require Python 2.7, the future is not going anywhere soon.
- techdragon 11y ago
  All of my code on my machine is Python 3 and soon the python my OS is using will be Python 3. Do we need to keep flogging this dead horse. Python 3 is here, it's big it's bold it's bloody working unless your doing the most arcanely version specific Python code. If your extremely lazy, just use 'futurize -0 -w' and boom Python 2 and 3 compatible code 95% if the time.
- swozey 11y ago
  I'm so tired of seeing people act like LPTHW is some gospel. I tried using it to learn Python and absolutely hated it. It scared me away from Python. Especially the "don't ever use 3, use 2" thing. And if you're going to assume that government entities use of a language is what is going to drive our market than you better stop using Python and hop back on to the ColdFusion and Fortran train.
  - 51109 11y ago
    It's not an act. LPTHW is my Bible. I retake the courses every Sunday. Saying that I should hop on the Fortran train for stating that even advanced fearless Python users opt for 2.7 is heretics!
- japhyr 11y ago
  My advice to new programmers is completely opposite this. I recommend that people learn Python 3 and use it until they have a specific reason not to. Most new programmers can work with Python 3 and never run into difficulty. If someone needs to use 2.7 for a specific project, it's easy to adapt if you're well grounded in Python 3. There's no reason these days to tell beginners to start with Python 2.
BuckRogers 11y ago
What I think people figured out is you can port Python2 code to PyPy4. Then for all the testing involved, you remove any potential CPU performance worries and the GIL in one fell swoop. Those two perks are worth more than any piddly features of Python3, which were already resolved in the ecosystem. I ported to PyPy instead of Python3 and gained those advantages, but can always port to Python3 when/if it ever gains traction. But I'm just as likely to migrate new projects to something else than Python3. I view this blog as a long-form denial letter to himself.
mrfusion 11y ago
I've always wondered why they couldn't package 2 and 3 in the same binary and let the user have a switch for which to use?
- takeda 11y ago
  They don't need to. You can install all versions (at least tested 2.4 - 3.5) side by side without any conflicts, which makes the whole dispute which language version to use so silly.
JustSomeNobody 11y ago
Use 3. Unless you have a good reason to use 2, then use 2. The world doesn't have to be perfect. Go make something.
beej71 11y ago
Those of us running Arch Linux have to check the rearview mirror to see the transition... back in 2010 when "python" started defaulting to Python3. It wasn't the smoothest thing, but the world didn't end, either. Lots can be learned from it if you're thinking of making the move.
jeffdavis 11y ago
I think one fundamental miscalculation went like this: early adopters will start writing new code in python3, and it will gradually become more normal for new code to be python3, and eventually it will snowball and python2 will become legacy. The problem is that early adopters have a choice: python3, or a completely different language. And there are a lot of good languages to choose from, and early adopters are a more radical by nature. That's why breaking backwards compatibility is almost always worse than it looks like on paper: the early adopters you are counting on to get you through the transition are also the most likely to leave your platform completely.
someguydave 11y ago
I like the fact that the 'python core' devs have moved on to doing something other than pointlessly fiddling with something that's reasonably good, ie python 2.7 As for 'lack of support', what are folks expecting support for? If there's a known security issue, somebody will write a patch.

110 points by joeyespo 11y ago

One frustrating part of attempting to transition is that not all libraries make it clear which Python versions are supported. Usually it's Python 2-only code that isn't clearly labeled as such, but it happens with Python 3 code as well. Sometimes a library doesn't work under specific versions of Python 3. The result is that library research takes longer than it would. Please include this in the README for any project, even if it seems obvious.

sitkack 11y ago

I moved last month and I have been using Python since 1.5.2 What prevents me from recommending it to everyone I pass on the street is that the PyPy 3.x line is back on 3.2.5 which makes it not compatible with many libs which have a 3.3 minimum, which is most. Yes, I give to pypy.org http://pypy.org/py3donate.html http://pypy.org/py3donate.html Recommendations to folks * switch your daily interpreter to Python3 * use tox, https://pypi.python.org/pypi/tox * use six, https://pypi.python.org/pypi/six * when you go native, go cffi, https://pypi.python.org/pypi/cffi

PeCaN 11y ago
Why's this downvoted, it seems like useful advice to me. Also cffi is definitely awesome.
eiopa 11y ago
This kills it for me too. I rely on pypy, and I just can't see myself sacrificing my productivity by using Six. Also, Python 3 neglected to fix one of the most annoying things about the language - default arg value def foo(x=[]): x.append(1) print x foo() # 1 foo() # 1 1 Why is this still busted??
- rspeer 11y ago
  Because nobody's made a Python Enhancement Proposal for what should happen instead.
- makecheck 11y ago
  It may be an easy mistake to make but using None is more consistent and efficient to detect. You can use "if x is None:" for any type; whereas, a function may or may not want to allow empty strings (or even empty lists) and it is easy to use a None check to decide when to apply the desired default.
- nbadg 11y ago
  Are you referring persistence of mutable default args? Which then potentially leads to bugs when you treat it as a new variable on each function call? This is intentional behavior. It's the result of one-time evaluation of default args, which is important for memoization. It's also really useful in combination with late-binding closures. For example, using a lambda as a generator function: def create_multipliers(): return [lambda x : i * x for i in range(5)] This doesn't work, you'll get all 8's. Instead you need to: def create_multipliers(): return [lambda x, i=i : i * x for i in range(5)] Late-binding closures and memoization strategies are pretty core language features; I wouldn't expect them to change (and many python devs would be pretty pissed if they did). Yes, this can be confusing with mutable default objects, but the alternative is to have disparate behavior depending on the mutability of the defaults, which would be an utter catastrophe. http://docs.python-guide.org/en/latest/writing/gotchas/ http://docs.python-guide.org/en/latest/writing/gotchas/
  - alextgordon 11y ago
    Other languages seem to get it right. Even C++! #include <vector> static int foo(std::vector<int> x = {}) { x.push_back(10); return x.size(); } int main (int argc, char const *argv[]) { printf("%d\n", foo()); // 1 printf("%d\n", foo()); // still 1 return 0; } def foo(x=[]): x.append(10) return len(x) print(foo()) # 1 print(foo()) # 2? wtf?
    - eiopa 11y ago
      ES6 got this right too. This design decision makes me die a little every time I have to do: def foo(x=None): if not x: x = [] ...
      - dajonker 11y ago
        That will get you into trouble as well, please use: if x is None: x = [] ... etc.
- throwaway91919 11y ago
  Can you (or someone) clarify what problem you're talking about? This seems to work as expected for me: def foo(x=[]): sum = 0 for a in x: sum += a return sum print foo([3,5]) # 8 print foo(x=[3]) # 3 print foo([]) # 0 print foo() # 0 Is it the fact that kwargs are required to have defaults? Or what?
  - raverbashing 11y ago
    Nah, do it like this def foo(i,x=[]): x.append(i) return x foo(1) foo(2) See what happens
  - saltylicorice 11y ago
    The problem is that the default argument is mutable. def f(a=[]): a.append('v') print a f(['ok']) # ['ok', 'v'] f() # ['v'] f() # ['v', 'v']
    - throwaway91919 11y ago
      Ah, thanks.
- deleted 11y ago
  [deleted]

wh-uws 11y ago

As much flak as ruby has gotten for its performance issues and "magic" (read: heavy preference for use of meta programming) ... One thing I think the ruby core team has done a great job of in my opinion is always having some awesome new features in the next major release that make you say "hey thats awesome" and make you want to upgrade despite any headache it might cause and the headaches are generally minimal. Its been interesting to see python so publicly and painfully fail at that. A whole new language version release that as far as I can tell only changed internal plumbing and went so far as to make you have rewrite print statements? I just don't understand how they could think people would want to move to that.

lyschoening 11y ago
Maybe Python 2 to Python 3.0 didn't introduce many "awesome" features, but 3.4 and 3.5 have been great.
- AnkhMorporkian 11y ago
  asyncio and pathlib are absolutely killer features for me. I never much cared about Python 3 until asyncio came on the field. As soon as I saw some example code Guido had written, I started using tulip, and have been a willing participant on the wild ride it has taken.
  - rspeer 11y ago
    Pathlib lets me down a bit -- I find myself having to str() the Path objects and lose their benefits, because the rest of the standard library doesn't use pathlib. But asyncio is great. It shows how much better asynchronous programming can be when you have support from the language itself. I can't imagine going back to a byzantine system like Twisted even if they did manage to finish their Python 3 port.

x1024 11y ago

Wait, Python 3 has existed since 2008 and the transition is at 20%? How is that any form of success?

MatthewWilkes 11y ago
Backwards incompatible language changes having not killed the community and gaining traction after 7 years is pretty good going.
craigds 11y ago
He said that 20% have reached "Acceptance", the last of the 5 stages, and said that most people are in stage 4 (Depression). That'd put overall progress at more like 80%. This article also wasn't actually talking about adoption of python 3, just applying the stages of grief to how people feel about it. IOW, adoption of python 3 might still be worse than 20%, but 20% have accepted that they will need to adopt it soon. Anecdotally as a full time python guy, I'd agree with the OP. The number of people interested in Python 3 has been growing a bit every year, and it seems opposition to adopting it at all has finally mostly dwindled :)
- kzrdude 11y ago
  I still find reasons to be pessimistic. OpenCV was really really late to support Py 3. Every pypy release focuses on Py 2 only (and Py 3 releases are an afterthought). Really weird prioritization for me (I'm not interested in legacy-focused projects).
- vacri 11y ago
  If you read the wiki link in the article, the 'stages of grief' aren't stages that you progress through. They're different ways of dealing with loss, and you won't necessarily hit all of them or do them in any particular order.
  - claystu 11y ago
    More importantly, these stages have nothing to do with choosing either Python 2 or Python 3. Python is just a tool. If one doesn't work, you choose another, and if the developers of that tool decide to screw it up, you just move on.

captainmuon 11y ago

If someone would make a Python 2.8 (or call it Python 5=2+3 if you like), I would pay for it. It must run basically every python 2.7 (or 2.6) program without changes. It should backport as many features as possible. The meaning of "str" and string literals (bytes or unicode) should be configurable on a per-file basis. It should have a print statement. Finally, it should be able to use all the binary modules from 2.7 (maybe after recompilation). Currently at our work (in physics research), I have the feeling that people are giving up on Python. It's not worth the trouble to port to python 3 - especially because we can't switch all code at once; we'd have to support both versions for years. C++ which is also heavily used here is getting much nicer, with proper tooling it is even more pleasant to work in (code completion etc). We are mostly using Python for a) configuration files and b) quick and dirty scripts. For these cases, Python 2.7 will be good enough forever. If someone does something more complex, they'd probably use Python 3 - but they treat it like it's a completely different language rather than a new version of Python 2.

SpaceManNabs 11y ago
Likewise, one of my previous advisors has stopped doing any quick and dirty math in python. I am seeing Julia a lot more (for astrophysics and ML) and more C++ (physics) for the more important simulations.
- rspeer 11y ago
  Julia seems like an odd thing to bring up on the topic of "avoiding backward-incompatible language changes".
jamesdutc 11y ago
You'd have to pay a lot. Jokes of mine like https://github.com/dutc/rwatch https://github.com/dutc/rwatch and https://github.com/dutc/didyoumean https://github.com/dutc/didyoumean and https://bitbucket.org/dutc/astexpr https://bitbucket.org/dutc/astexpr are examples of adding major features to the CPython interpreter at run-time. It works fairly well in practice. It ends up looking like a `__future__`-import pragma that has interpreter lifetime scope rather than file scope. `__future__` imports cannot be undone within the same file -- after all, they're pragmas scoped at the file level, not actual imports -- and it would make sense for these backported features to behave similarly, though at the scope of the interpreter's lifetime. That might not do what you want. As you'd expect, the most superficial of Python 3's new features (syntax changes, `raise from`, &c.) would be easiest to backport. The bulk of the difficulty would be in merging those changes into Python 2.7. Making the result hot-loadable would be just as easy as in `dutc-rwatch`, though being able to toggle features on-and-off independently would require combinatoric effort with the naïve approach. You'd need to do something smarter. All in all, it's wholly possible to do what you want, and you probably have the skill to even do it yourself. I'd happily do the work myself, but only for lots of money. Unfortunately, the burden of both certifying the resulting interpreter as correct and of maintaining this fork would be large. And then when you're on-boarding some fresh new post-grad you've just hired into your research group: "Oh, we use Python 2.8 here. That's what we call Python 2.7 with an unsupported collection of modules that mutate the currently running interpreter to backport features from Python 3. Don't worry; it only breaks in ways that no one outside of this office will ever have a chance of ever understanding or helping you debug."

craigds 11y ago

Personally, I can't wait to switch to python 3. I'm glad most of the community is finally getting behind it. The cogs move slowly. We've only just upgraded our huge codebase to 2.7, but I expect we'll be on some release of 3.x in about two years. Almost all of our big list of dependencies now support 3.3+, so there is little reason to not upgrade.

foobar2357 11y ago

The Python team is blind to the huge number of users that will never switch to Python 3. My industry (a large one) is currently standardized on RHEL 6 (and the CentOS equivalent), which uses Python 2.6! I'm not generally able to install arbitrary software or demand upgrades without a lot of beurocracy. I was looking forward to the day when we move to RHEL 7 with Python 2.7. I've started to view the lack of changes in 2.7 as a feature. I would value the stability and lack of breaking changes. Knowing that Python 2.7 is on a limited timeline, and that the Python 3 folks (users and developers) are trying to kill off 2.7, I can't think of any reason not to switch to another language. In short, my field is nowhere near "acceptance" of Python 3, and the "anger" stage will just make us (me at least) leave Python behind. I will never want Python 3.x, and I don't even want Python 2.8. I wish people would recognize that stability is a virtue for some of us...

mkesper 11y ago
The investment in porting your code from Python2 to Python3 should be many times smaller than porting to an entirely new language ecosystem.
- dimva 11y ago
  Yea, but what's to stop Python 4 from doing the same thing as Python 3? It's clear from the tone of this article that the python core devs haven't learned the value of backwards compatibility even after their huge python 3 failure. The author makes it seem like the problem preventing python 3 adoption was python users' unwillingness to accept the inevitable, rather than the poor migration path provided (especially initially). In a massive, old production codebase, upgrading your platform is incredibly risky, even if no code changes are necessary. But when you have to make code changes in a dynamic language... The thing is, not all systems have tests. Not all systems even have known, defined behavior. Upgrading these things can be a multi-month, very risky project. Given that python 3's backwards incompatible changes were made for purely cosmetic reasons and there are few compelling reasons to upgrade, I totally get the anger people feel. I work in startups with relatively new codebases and good test coverage, but I can empathize with other types of organizations. I'm shocked that python devs don't, even after 7 years.

rdtsc 11y ago

> While some people feel stuck in Python 2 at work and are "depressed" over it, others have reached the point of having transitioned their projects and accepted Python 3, both at work and in personal projects. It is not as simple. I would be simpler if it was 2006 again. Python would have less competition on being an easy to use language, batteries included, with good enough performance (SMP wasn't as popular then). Java had a bad perception -- too bloated, enterprise-y, Javascript on the server wasn't happening much. Go wasn't there. Scala not as popular. In that context, people would eventually have switched to Python 3 even if it took some pain (security maintenance, some newer features etc), because GIL wasn't a big deal and it sure wasn't going to be Perl again (maybe Ruby for webdev?). The situation is more difficult today -- Python has more serious competition. Once a team has found the time/money/excitement to switch to Python 3, it is a high chance they'll start looking and evaluating other languages/platforms/libraries (and I speak as a person who loves Python and used it professionally for 12+ years). It is now competing with Go, Elixir, Javascript, Scala, Julia, Clojure, even Rust, Java, and C++14. No matter how cool Python 3 is and what new features it brings, it will have a harder battle to fight just because of the current environment.

criddell 11y ago
> It is now competing with Go, Elixir, Javascript, Scala, Julia, Clojure, even Rust, Java, and C++14. That would have happened no matter what. It's not really a bad thing, is it? I can't see how something better coming along is anything but great news.

upofadown 11y ago

>With five years left until people will need to pay for Python 2 support, ... Is this some sort of thing?

kemayo 11y ago
It's just the author pointing out that the EOL for official support for Python 2 is 2020. After then, the Python dev team won't be providing any (free) support, so people who want to keep using it will have to pay for it in some way -- whether by investing time in maintaining a fork, or by paying someone else to do so. The absence of a "Python 2.8" fork suggests that the idea of forking hasn't been popular enough to happen organically without someone putting money in, after all.
- someguydave 11y ago
  >The absence of a "Python 2.8" fork suggests that the idea of forking hasn't been popular enough to happen organically without someone putting money in, after all. Or it's evidence that python 2.7 works just fine for many people.
Ded7xSEoPKYNsDd 11y ago
Cent OS 7 will support Python 2 for free until 2024. I would assume most patches will work without changes on other OSes as well. But if you want someone to do the QA work (and compiling) for you and no independent Python 2 fork will arise, you'd presumably have to pay someone for that. Beyond that, apparently you can pay Red Hat to big bucks to provide you with bug fixes for really old Red Hat releases in a similar way you can pay Microsoft big bucks to get patches for Windows XP even today. I'm sure other companies will do the same if they can find enough customers that find porting costs more than such a support contract.

PhantomGremlin 11y ago

There was a Simpsons episode[1] back in 1991 in which Homer, after eating a poisonous fugu fish, must deal with the five stages. It's an all time classic: Dr. Julius Hibbert: Now, a little death anxiety is normal. You can expect to go through five stages. The first is denial. Homer Simpson: No way, because I'm not dying! Dr. Julius Hibbert: Second is anger. Homer Simpson: [furiously] Why you little... ! Dr. Julius Hibbert: After that comes fear. Homer Simpson: [worried] What's after fear? What's after fear? Dr. Julius Hibbert: Bargaining. Homer Simpson: Doc, you gotta get me outta this. I'll make it worth your while. Dr. Julius Hibbert: Finally acceptance. Homer Simpson: Well, we all gotta go sometime. Dr. Julius Hibbert: Mr. Simpson, your progress astounds me. If only this Python transition were so simple. [1] https://en.wikipedia.org/wiki/One_Fish,_Two_Fish,_Blowfish,_Blue_Fish https://en.wikipedia.org/wiki/One_Fish,_Two_Fish,_Blowfish,_...

Walkman 11y ago

Extended the maintenance timeline from 5 to 10 years is not helping the transition at all.

someguydave 11y ago
The extended maintenance timeline is an implicit admission that nobody wants to hop on their wagon.

tonetheman 11y ago

I am on CentOS and honestly for most of our applications there is just no compelling reason to change. We can do everything with 2 that we need. I have seen some async stuff that looks interesting in 3 but not enough to make me change.

cdnsteve 11y ago

Can someone explain aiohttp server vs wsgi? It seems wsgi is the old standard but now you can run a server right in python, kind of like node. Are people running aiohttp instead of wsgi? It seems like aiohttp can do everything that Flask can do. Are more people using this?

RodericDay 11y ago
I want to know too. I did a little toy project with websockets in aiohttp and it was extremely enjoyable. It'd be reassuring to know that other people see it as "the future" too, so I could invest into it more heavily.
et1337 11y ago
I've used aiohttp for a few things. Works fine, small, lightweight, easy to use.

jacquesm 11y ago

The python 2.x to 3.x change will be studied for many years. How to completely rob an ecosystem of its momentum in one fell swoop. I still don't understand why 3 couldn't have had a mode switch defaulting to 2.x behavior. The whole debacle would have been avoided and all the ugly details would have been hidden behind the language implementation. It would have definitely been more work and you might even call it an ugly hack (and Ed DeCastro would likely have thrown a fit if one had asked his opinion) but it would have at least made the transition instant and seamless. The present mess will - if not resolved in a much more drastic fashion - cause more and more teams to switch away from python. It's a real waste of the potential that python had (and still has). I never even bothered porting my code, I'm 'stuck' on 2.7 and anything new gets written in another language. Even bloody PHP, the language everybody loves to hate on, gets deprecation and backwards compatibility better than python.

gravypod 11y ago
Sadly I have to agree with you. Some things that changed were not even necessary. Why would the development team make replacement packages rather than fixing underlying implementation, for example imp and importlib. There was also no sugar added to support this mangled jungle gym of package replacements. Backwards compatibility is a thing of dreams for the python developer. This is only one of many of the problems with the change.
msellout 11y ago
> gets ... backwards compatibility That should be past tense, "got" as the Python community has certainly learned its lesson. Luckily it wasn't as bad a mistake as the Perl folks made.
- jacquesm 11y ago
  > That should be past tense, "got" as the Python community has certainly learned its lesson. Evidence of that to me would be a proposed fix and maybe I'm too out of touch with what's happening in the python world but from what I see the two main version still co-exist rather than that any attempt at unification has been made. Am I missing something major here? My understanding is that you migrate your project to 3.x or you're stuck in the past. > Luckily it wasn't as bad a mistake as the Perl folks made. That's up for debate.

Walkman 11y ago

> I have yet to hear from someone who has used Python 3 that they think it is a worse language than Python 2 the maker of the most succesful Python projects: Armin Ronacher

mianos 11y ago
Just ask him about unicode for LOLs.

joseph8th 11y ago

I was in denial stage until we picked Python 3 for a project at work. Then I went into anger and bargaining with the other dev on the project. Heh. blush Being forced to come to terms with Python 3 has made me see the light, though. I still have a host of personal projects in Python 2, but anything new I write is in 3. For me, it was concurrent.futures that pushed me into acceptance stage.

macavity23 11y ago
My experience was similar. An even bigger push in my case was the async stuff in 3.5. It's going to be a little while before pypy supports 3.5, but once that happens, Python will be very well placed. Something like Tornado + pypy + py3.5 looks a lot like the app server stack of the future IMHO. It has been a massive PITA, and probably could have been better handled, but hindsight is easy. The future looks bright.

AstroChimpHam 11y ago

I'm a long time Python user and I still tend to go for 2.7 over 3. When writing packages, I'll add support for 3 as an afterthought, but that's it. The main reason for this and the main reason I prefer Python to other languages is the excellent and complete ecosystem of libraries. I've never seen a single Python library I wanted to use in the last 8 years that didn't have support for Python 2.7. I still run into those that don't support Python 3 well. So, using Python 3 means being terrified that I run into a case later on where I need to use some package that only supports 2.7 and I get stuck. With all of the other potential problems to run into while coding, why would I want to add that extra one, especially for something work-related? I imagine this will continue being an issue until some major packages stop supporting Python 2.7 entirely and force people to make a choice.

51109 11y ago

From LearnPythonTheHardWay.org: > A programmer may try to get you to install Python 3 and learn that. Say, "When all of the Python code on your computer is Python 3, then I'll try to learn it." That should keep them busy for about 10 years. I repeat, do not use Python 3. Python 3 is not used very much, and if you learn Python 2 you can easily learn Python 3 when you need it. If you learn Python 3 then you'll still have to learn Python 2 to get anything done. Just learn Python 2 and ignore people saying Python 3 is the future. Besides, judging from the number of DARPA-sponsored projects that require Python 2.7, the future is not going anywhere soon.

techdragon 11y ago
All of my code on my machine is Python 3 and soon the python my OS is using will be Python 3. Do we need to keep flogging this dead horse. Python 3 is here, it's big it's bold it's bloody working unless your doing the most arcanely version specific Python code. If your extremely lazy, just use 'futurize -0 -w' and boom Python 2 and 3 compatible code 95% if the time.
swozey 11y ago
I'm so tired of seeing people act like LPTHW is some gospel. I tried using it to learn Python and absolutely hated it. It scared me away from Python. Especially the "don't ever use 3, use 2" thing. And if you're going to assume that government entities use of a language is what is going to drive our market than you better stop using Python and hop back on to the ColdFusion and Fortran train.
- 51109 11y ago
  It's not an act. LPTHW is my Bible. I retake the courses every Sunday. Saying that I should hop on the Fortran train for stating that even advanced fearless Python users opt for 2.7 is heretics!
japhyr 11y ago
My advice to new programmers is completely opposite this. I recommend that people learn Python 3 and use it until they have a specific reason not to. Most new programmers can work with Python 3 and never run into difficulty. If someone needs to use 2.7 for a specific project, it's easy to adapt if you're well grounded in Python 3. There's no reason these days to tell beginners to start with Python 2.

BuckRogers 11y ago

What I think people figured out is you can port Python2 code to PyPy4. Then for all the testing involved, you remove any potential CPU performance worries and the GIL in one fell swoop. Those two perks are worth more than any piddly features of Python3, which were already resolved in the ecosystem. I ported to PyPy instead of Python3 and gained those advantages, but can always port to Python3 when/if it ever gains traction. But I'm just as likely to migrate new projects to something else than Python3. I view this blog as a long-form denial letter to himself.

mrfusion 11y ago

I've always wondered why they couldn't package 2 and 3 in the same binary and let the user have a switch for which to use?

takeda 11y ago
They don't need to. You can install all versions (at least tested 2.4 - 3.5) side by side without any conflicts, which makes the whole dispute which language version to use so silly.

JustSomeNobody 11y ago

Use 3. Unless you have a good reason to use 2, then use 2. The world doesn't have to be perfect. Go make something.

beej71 11y ago

Those of us running Arch Linux have to check the rearview mirror to see the transition... back in 2010 when "python" started defaulting to Python3. It wasn't the smoothest thing, but the world didn't end, either. Lots can be learned from it if you're thinking of making the move.

jeffdavis 11y ago

I think one fundamental miscalculation went like this: early adopters will start writing new code in python3, and it will gradually become more normal for new code to be python3, and eventually it will snowball and python2 will become legacy. The problem is that early adopters have a choice: python3, or a completely different language. And there are a lot of good languages to choose from, and early adopters are a more radical by nature. That's why breaking backwards compatibility is almost always worse than it looks like on paper: the early adopters you are counting on to get you through the transition are also the most likely to leave your platform completely.

someguydave 11y ago

I like the fact that the 'python core' devs have moved on to doing something other than pointlessly fiddling with something that's reasonably good, ie python 2.7 As for 'lack of support', what are folks expecting support for? If there's a known security issue, somebody will write a patch.