12 ms·

Porting a historic Python2 module to Python3

153 points by luch 9y ago

timcosgrove 9y ago
(100% OT of linked article: "a historic", not "an historic", unless you pronounce the word "istoric".) https://en.oxforddictionaries.com/usage/a-historic-event-or-an-historic-event https://en.oxforddictionaries.com/usage/a-historic-event-or-...
- fnord123 9y ago
  >unless you pronounce the word "istoric" Which we do (in Britain and the rest of the English speaking world outside North America), so "an historic" it is. e.g. from BBC: http://www.bbc.co.uk/programmes/p055vr37 http://www.bbc.co.uk/programmes/p055vr37
  - deleted 9y ago
    [deleted]
  - msla 9y ago
    Just keep in mind the fact that English is a polycentric language means your customs are no more or less correct than any other.
  - craigds 9y ago
    In NZ we definitely pronounce it "historic". But that doesn't stop the news anchors from saying "an historic". It sounds ridiculous
  - robotmay 9y ago
    I'm British and I have never personally pronounced it "istoric", and to my knowledge I have never heard anyone pronounce it like that either. Is it a regional thing? I can only recall it being pronounced like that in US TV shows with comedy fake English accents. "An hour" on the other hand makes more sense, as that is truly a soft H.
    - deleted 9y ago
      [deleted]
- jwilk 9y ago
  Shameless plug: I wrote a spellchecker that finds this kind of errors: https://github.com/jwilk/anorack https://github.com/jwilk/anorack
  - disconnected 9y ago
    $ echo "Porting an historic Python2 module into Python3" > test_file $ ./anorack test_file $ out:1: an historic -> a historic /hIst'0rIk/ Interesting. What if... $ ...hacks phonetics.py to use "en-us" voice... $ ./anorack test_file $ out:1: an historic -> a historic /hIst'0rIk/ Hum... same result. What about: $ ...hacks phonetics.py to use "en-wm" voice... $ ./anorack test_file $ Ah, finally someone gets it right :) (just having a bit of fun, this is a cool work!) Edit: formatting...
    - billforsternz 9y ago
      Sorry for my ignorance, what is en-wm ?
      - jwilk 9y ago
        English as spoken in West Midlands No need to feel ignorant; it's a nonstandard, eSpeak-specific notation.
        kennydude 9y ago
        Is there a list of these? Would be fun to know what the Geordie and Scouse ones are
        jwilk 9y ago
        $ espeak --voices | grep ' en-' 2 en-gb M english en (en-uk 2)(en 2) 5 en-sc M en-scottish other/en-sc (en 4) 5 en-uk-north M english-north other/en-n (en-uk 3)(en 5) 5 en-uk-rp M english_rp other/en-rp (en-uk 4)(en 5) 5 en-uk-wmids M english_wmids other/en-wm (en-uk 9)(en 9) 2 en-us M english-us en-us (en-r 5)(en 3) 5 en-wi M en-westindies other/en-wi (en-uk 4)(en 10)
- nerdponx 9y ago
  "An historic" is not only still acceptable in the USA, but it also (IMO) sounds better. The leading "H" is not a strong-sounding consonant, and is often dropped in both American and British English.
  - stretchwithme 9y ago
    I will look that up in an history book and get back to you.
    - greenshackle2 9y ago
      They're pronounced differently: History: /ˈhist(ə)rē/ Historic: /hiˈstôrik/ The 'h' is stressed in 'history', not in 'historic'. Try pronouncing 'historic' with a stress on the first syllable, it sounds wrong. Since it's softer in historic it's more natural (to me anyway) to use 'an'. (Also, I'm french so I barely pronounce h's to begin with - so in my speech the stressed 'h' in history is kinda soft, and the unstressed h in historic is barely there.) (edited)
      - stretchwithme 9y ago
        I've never heard the Hs pronounced differently here in the US. Perhaps its too subtle for me.
- dec0dedab0de 9y ago
  Drives me nuts when people don't pronounce H
- stretchwithme 9y ago
  There's an history of ignoring how other words that start with the exact same sound are handled.
keenerd 9y ago
I've been doing this for ten years. Why wouldn't you use 2to3? In my experience, 2to3 is all you need 95% of the time. It is really that good. The only times it doesn't work is with python that is bad to an unusual degree. For example, if you make a variable named "list", overriding the built-in list type. 2to3 freaks right out. The other type of "bad" python is stuff that abuses the "open kimono" nature of the language and heavily monkeypatches python internals. Or things that depend on version specific binary data, like pickle files.
- deleted 9y ago
  [deleted]
- jwilk 9y ago
  > 2to3 is all you need 95% of the time This is absolutely not the case in my experience. 2to3 tends to produce something that appears to work at first glance, but it actually doesn't.
  - aaossa 9y ago
    You have any examples? I was planning to use 2to3 in a project, can you explain in which case it doesn't work? Thanks!
    - masklinn 9y ago
      First, I'd recommend python-future rather than 2to3, it has additional fixers and replaces some with better versions. Second, those only do the fairly straightforward changes, fixers are simple AST transformations they don't do complex type analysis or anything, you should use them as a starting point but don't expect the work to be over by then: the 2->3 transitions has a number of API and semantics changes which these tools can't handle, text model changes are the biggest one[0] but they're not the only one by far. If the project is non-trivial, having a good test coverage is absolutely crucial. Basically, 2to3 handles the first 90% of the work which are mostly drudgery and syntactic fixes, you're still on the hook for the second 90% which is more subtle. Source: finalising the conversion of a ~200k SLOC codebase to cross-version compatibility. [0] not just the strict separation of text and bytes, but APIs being set to one or the other so you have places where you want to ensure bytes, others where you want to ensure text and yet others where you need "native strings", some APIs (csv) are also incompatibly altered.
yegle 9y ago
There's a patch from pytype that backported the type annotation syntax from python3 to python27: https://github.com/google/pytype/blob/master/pytype/patches/python_2_7_type_annotations.diff https://github.com/google/pytype/blob/master/pytype/patches/...
- sametmax 9y ago
  You don't really need it, Python 2.7 can already use type comments.
  - luch 9y ago
    (author here) by type comments your mean this : https://www.python.org/dev/peps/pep-0484/#type-comments https://www.python.org/dev/peps/pep-0484/#type-comments ? I haven't seen them used anywhere, but that's an okay solution I guess. NB : by the way, love your blog :p
    - carapace 9y ago
      I was experimenting with them a little while ago, along with MyPy [1] but I immediately ran into an issue where the type system couldn't describe the type I was using, so I had to let it go (reluctantly.) [1] http://mypy-lang.org/ http://mypy-lang.org/
      - sametmax 9y ago
        Same. Mypy was very limited at the beginning, and have been much more improved since. I give it a try regularly. It is indeed way better now, although I'm still structural typing and taking duck typing in consideration.
    - sametmax 9y ago
      Yep. Works great. Less convenient than regular type hints, but when you need a transition tool, it's nice. > NB : by the way, love your blog :p Thanks :)
- 9y ago
jcolella 9y ago
Great article, specially with the sections on coverage on testing. Also, specifying the process so another can easily reproduce it. Very nice!
jwilk 9y ago
https://github.com/Microsoft/microsoft-pdb/pull/27 https://github.com/Microsoft/microsoft-pdb/pull/27 is labelled cla-not-required, so it's probably not the best example for "need to sign NDA/CLA for sending PR". But some other projects are so hardcore about CLA enforcement, they won't even accept one-letter typo-fixes without CLA signed.
jxramos 9y ago
I really like the suggestion about focusing on PR for PR, that is pull request public relations. PR diplomacy, very sound advice. +1 for articulating this bit of funny business in the Python Windows ecosystem... "Moreover on Windows the building process was historically so bad that you usually end up downloading binaries from some random guy on the Internet."
opportune 9y ago
I once spent a week porting some python2 researchware into python3. It was only at the end of that week that I discovered that it was the actual logic of the researchware that was broken, and not the way I was porting it. I now distrust all FOSS except for widely used and tested ones. You would think that a graduate student would debug (or at least fix to the point of being interpretable) the work that they spent months on before publishing it as finished and adding it to pip. And yes, 2to3 fixes 90% of porting problems. But in my experience, it was especially bad at handling, of all things, import statements. Definitely not a silver bullet.
- ofek 9y ago
  I'd be weary to conflate OSS with what you refer to as "researchware". Code coming from academia is notorious for poor reproducibility.
  - opportune 9y ago
    I'm not conflating them, but clearly software can be both researchware and open source. I'm not going to start distrusting tensorflow or scikit-learn, but I'm going to be wary before I invest time in a promising niche software package, regardless of whether it was actually from academia or not.
  - aaronchall 9y ago
    It is tiresome, isn't it.
- sevensor 9y ago
  You have a remarkably rosy view of grad students and their relationship to software. All it has to do is produce output (I hesitate to say work) one time, on one computer, under some conditions that may never be documented, let alone reproduced. This is doubly true if the code just a step in the analysis chain and not the actual subject of the research.
  - opportune 9y ago
    In this case, the software itself was the focus of the research. And this grad student still works at a very prestigious university for CS. I've worked in academia (as a research assistant) before so I'm familiar with shitty grad student software. But going to the effort to document and present on the software without making sure it even worked? That seemed excessively stupid to me.
radarsat1 9y ago
In almost every Python-related project I work on, we're developing in an exciting new language called "Python 2-compatible Python 3 subset". Twice as must testing, twice as much fun!
- crncosta 9y ago
  Yes, this is exactly the issue. Porting to Python 3 was never an issue itself, but keep your code compatible with Python 2 (while introduce 3) is a nightmare :P
- skykooler 9y ago
  I've been working on one project that needs to work in every Python verson from 2.4 to 3.3. There are some pretty fun gymnastics getting some syntax to work.
  - madez 9y ago
    Why wouldn't we build a vehicle that can drive on streets, on rails, can fly, swim and dig through the earth? It would be a marvellous construction. And awful engineering. I'm not blaming the person who builds the machine in the end, but the person that sets up the specification – they are doing a bad job.
- metafunctor 9y ago
  Are all these projects some sort of libraries? Is that why you need to support both Python 2 and Python 3? All the projects I've worked on moved to Python 3 and never looked back.
  - daveFNbuck 9y ago
    If it's a large project, you can't necessarily move everything to Python 3 all at once. You can't break Python 2 compatibility until everything works in Python 3.
    - radarsat1 9y ago
pinpeliponni 9y ago
Just tell the Python 2 users frankly to go fuck themselves.
- melling 9y ago
  Swift is 3 years old. Most people who use it will be on the latest version Swift 4 within a few months. Apple won’t let developers stay behind. While it was unrealistic for Python to be that aggressive given its larger community, not forcing the issue created a lot of unnecessary work for the community. A benevolent dictator should have moved developers along faster. Having the language fragmented for this long is extremely unproductive.
  - gogopuppygogo 9y ago
    Python 3 faced a lot of friction when it was introduced. There was no backwards compatibility with Python 2 and it had relatively little adoption. They added Python 3 features to Python 2 over the years to encourage adoption of Python 3 as the standard. It worked. Now we just need Apple to ship Mac OS with Python 3 instead of 2 as the default and that'll push adoption for developers.
    - mikebenfield 9y ago
      > They added Python 3 features to Python 2 over the years to encourage adoption of Python 3 as the standard. It worked. Did it really? My impression was the opposite: the fact that so much of Python 3 became available in Python 2 made people feel like there was no point in moving to Python 3.
      - viraptor 9y ago
        It worked both ways. Some new features were ported back, some could be ported, but explicitly weren't to encourage migration. On the other side, things like the "u" prefix got re-added to python 3 for compatibility.
  - weberc2 9y ago
carapace 9y ago
Oh dear God why do you hate my eyes. Tiny sans-serif gray body text is of the Devil and should be grounds for irate and overbearing truculent curmudgeons to whine about on online forums.
echion 9y ago
> I’m kinda bummed out that Python devs decided not to backport Python type hints in Python2.7. Can't comment on the site itself, so: PEP 484 - Type hints ( https://www.python.org/dev/peps/pep-0484/#suggested-syntax-for-python-2-7-and-straddling-code https://www.python.org/dev/peps/pep-0484/#suggested-syntax-f... ) does suggest Python 2.7-compatible type hints, and PyCharm, and probably other IDEs, do support them (e.g., https://www.jetbrains.com/help/pycharm/type-hinting-in-pycharm.html https://www.jetbrains.com/help/pycharm/type-hinting-in-pycha... ).
_eht 9y ago
Maybe this is an unpopular opinion coming from someone new to Python (2 years) but the way this versioning was done is definitely a pain point. Working with new code and looking up docs to achieve X, working with existing code and trying to apply the same solution. God it's annoying. Having said that, I know the decisions were not made lightly, and were backed by logical analysis. If nothing else, it's a good case study for the evolution of future languages (looking at you golang 2.0).
breatheoften 9y ago
How about a type checked python variant ala typescript that compiles to python2 or python3? Do: introduce and require the compile step for awhile, flesh out the language and build it up, get by-in and an ecosystem while also leveraging compatibility with python2/3 ecosystems -- then eventually release a python4 that runs this new typed python language directly! Bring us the beauty!
- deleted 9y ago
  [deleted]
- Rotareti 9y ago
  Have you tried the typing [0] module? I now use it for every project I'm working on and I find the experience working with it similar to the experience I have working with TypeScript. At least if you use a good IDE that can handle the type annotations. [0] https://docs.python.org/3/library/typing.html https://docs.python.org/3/library/typing.html
seanwilson 9y ago
> Usually, a dev writing a Python 2.7 library will not discriminate between bytes-like buffers and ASCII strings (since the language doesn’t either) which means to have to code review everything str operations and ask yourself if there is a bytes operation implied or is it really a string one. And...that's exactly why you want static typing. Code review to catch stuff like this is poorly and primitively doing a job a computer can do for you perfectly. Dynamic typing is fine for small projects but when you're wanting to port over all Python 2 code that's ever been written to Python 3 types would have made the task much less daunting.
- syrrim 9y ago
  You're thinking strong typing, not static typing. With stronger types, there would be a difference between a bytes string and unicode string, and the programmer would have to explicitly choose between them. With dynamic typing, this difference can still exists, and it is still obvious from the code. The only difference is that it isn't checked by the compiler at compile time. Look at c. It has static typing, but fairly weak types. Strings are represented as a series of bytes (char *). The exact same work would be required to port c code to semantic c++.
  - seanwilson 9y ago
    Whatever terminology you want to use, I mean at compile time you would get an error because the types are wrong for that specific example. Python is generally referred to as strongly dynamically typed, that's why I only wrote "static".
- tyingq 9y ago
  In this case, a new type was introduced...bytes didn't exist. So I don't know that static typing would have helped much, at least for dealing with existing code. You still have to decide what migrates from type1 to type2. The "bytes" that was put into 2.6 was just an alias to str as hinting for the 2to3 utility.
  - seanwilson 9y ago
moyix 9y ago
Well, this isn't exactly how I expected my PDB parsing library to end up on HN ;) The bytes/string changeover is probably the biggest pain point for RE tools written in Python2. Indeed, despite lucasg's excellent work porting things over, there was still at least one byte/string issue that needed to be fixed: https://github.com/moyix/pdbparse/issues/39 https://github.com/moyix/pdbparse/issues/39 Edit: Also, I have one small correction – pdbparse is actually 10 years old, not 5! I started writing it my first year out of college, which also explains (in part) why the code is a bit crap. As evidence I offer the first article I wrote on the PDB format: http://moyix.blogspot.com/2007/08/pdb-stream-decomposition.html http://moyix.blogspot.com/2007/08/pdb-stream-decomposition.h...

153 points by luch 9y ago

(100% OT of linked article: "a historic", not "an historic", unless you pronounce the word "istoric".) https://en.oxforddictionaries.com/usage/a-historic-event-or-an-historic-event https://en.oxforddictionaries.com/usage/a-historic-event-or-...

fnord123 9y ago
>unless you pronounce the word "istoric" Which we do (in Britain and the rest of the English speaking world outside North America), so "an historic" it is. e.g. from BBC: http://www.bbc.co.uk/programmes/p055vr37 http://www.bbc.co.uk/programmes/p055vr37
- deleted 9y ago
  [deleted]
- msla 9y ago
  Just keep in mind the fact that English is a polycentric language means your customs are no more or less correct than any other.
- craigds 9y ago
  In NZ we definitely pronounce it "historic". But that doesn't stop the news anchors from saying "an historic". It sounds ridiculous
- robotmay 9y ago
  I'm British and I have never personally pronounced it "istoric", and to my knowledge I have never heard anyone pronounce it like that either. Is it a regional thing? I can only recall it being pronounced like that in US TV shows with comedy fake English accents. "An hour" on the other hand makes more sense, as that is truly a soft H.
  - deleted 9y ago
    [deleted]
jwilk 9y ago
Shameless plug: I wrote a spellchecker that finds this kind of errors: https://github.com/jwilk/anorack https://github.com/jwilk/anorack
- disconnected 9y ago
  $ echo "Porting an historic Python2 module into Python3" > test_file $ ./anorack test_file $ out:1: an historic -> a historic /hIst'0rIk/ Interesting. What if... $ ...hacks phonetics.py to use "en-us" voice... $ ./anorack test_file $ out:1: an historic -> a historic /hIst'0rIk/ Hum... same result. What about: $ ...hacks phonetics.py to use "en-wm" voice... $ ./anorack test_file $ Ah, finally someone gets it right :) (just having a bit of fun, this is a cool work!) Edit: formatting...
  - billforsternz 9y ago
    Sorry for my ignorance, what is en-wm ?
    - jwilk 9y ago
      English as spoken in West Midlands No need to feel ignorant; it's a nonstandard, eSpeak-specific notation.
      - kennydude 9y ago
        Is there a list of these? Would be fun to know what the Geordie and Scouse ones are
        jwilk 9y ago
        $ espeak --voices | grep ' en-' 2 en-gb M english en (en-uk 2)(en 2) 5 en-sc M en-scottish other/en-sc (en 4) 5 en-uk-north M english-north other/en-n (en-uk 3)(en 5) 5 en-uk-rp M english_rp other/en-rp (en-uk 4)(en 5) 5 en-uk-wmids M english_wmids other/en-wm (en-uk 9)(en 9) 2 en-us M english-us en-us (en-r 5)(en 3) 5 en-wi M en-westindies other/en-wi (en-uk 4)(en 10)
nerdponx 9y ago
"An historic" is not only still acceptable in the USA, but it also (IMO) sounds better. The leading "H" is not a strong-sounding consonant, and is often dropped in both American and British English.
- stretchwithme 9y ago
  I will look that up in an history book and get back to you.
  - greenshackle2 9y ago
    They're pronounced differently: History: /ˈhist(ə)rē/ Historic: /hiˈstôrik/ The 'h' is stressed in 'history', not in 'historic'. Try pronouncing 'historic' with a stress on the first syllable, it sounds wrong. Since it's softer in historic it's more natural (to me anyway) to use 'an'. (Also, I'm french so I barely pronounce h's to begin with - so in my speech the stressed 'h' in history is kinda soft, and the unstressed h in historic is barely there.) (edited)
    - stretchwithme 9y ago
      I've never heard the Hs pronounced differently here in the US. Perhaps its too subtle for me.
dec0dedab0de 9y ago
Drives me nuts when people don't pronounce H
stretchwithme 9y ago
There's an history of ignoring how other words that start with the exact same sound are handled.

keenerd 9y ago

I've been doing this for ten years. Why wouldn't you use 2to3? In my experience, 2to3 is all you need 95% of the time. It is really that good. The only times it doesn't work is with python that is bad to an unusual degree. For example, if you make a variable named "list", overriding the built-in list type. 2to3 freaks right out. The other type of "bad" python is stuff that abuses the "open kimono" nature of the language and heavily monkeypatches python internals. Or things that depend on version specific binary data, like pickle files.

deleted 9y ago
[deleted]
jwilk 9y ago
> 2to3 is all you need 95% of the time This is absolutely not the case in my experience. 2to3 tends to produce something that appears to work at first glance, but it actually doesn't.
- aaossa 9y ago
  You have any examples? I was planning to use 2to3 in a project, can you explain in which case it doesn't work? Thanks!
  - masklinn 9y ago
    First, I'd recommend python-future rather than 2to3, it has additional fixers and replaces some with better versions. Second, those only do the fairly straightforward changes, fixers are simple AST transformations they don't do complex type analysis or anything, you should use them as a starting point but don't expect the work to be over by then: the 2->3 transitions has a number of API and semantics changes which these tools can't handle, text model changes are the biggest one[0] but they're not the only one by far. If the project is non-trivial, having a good test coverage is absolutely crucial. Basically, 2to3 handles the first 90% of the work which are mostly drudgery and syntactic fixes, you're still on the hook for the second 90% which is more subtle. Source: finalising the conversion of a ~200k SLOC codebase to cross-version compatibility. [0] not just the strict separation of text and bytes, but APIs being set to one or the other so you have places where you want to ensure bytes, others where you want to ensure text and yet others where you need "native strings", some APIs (csv) are also incompatibly altered.

yegle 9y ago

There's a patch from pytype that backported the type annotation syntax from python3 to python27: https://github.com/google/pytype/blob/master/pytype/patches/python_2_7_type_annotations.diff https://github.com/google/pytype/blob/master/pytype/patches/...

sametmax 9y ago
You don't really need it, Python 2.7 can already use type comments.
- luch 9y ago
  (author here) by type comments your mean this : https://www.python.org/dev/peps/pep-0484/#type-comments https://www.python.org/dev/peps/pep-0484/#type-comments ? I haven't seen them used anywhere, but that's an okay solution I guess. NB : by the way, love your blog :p
  - carapace 9y ago
    I was experimenting with them a little while ago, along with MyPy [1] but I immediately ran into an issue where the type system couldn't describe the type I was using, so I had to let it go (reluctantly.) [1] http://mypy-lang.org/ http://mypy-lang.org/
    - sametmax 9y ago
      Same. Mypy was very limited at the beginning, and have been much more improved since. I give it a try regularly. It is indeed way better now, although I'm still structural typing and taking duck typing in consideration.
  - sametmax 9y ago
    Yep. Works great. Less convenient than regular type hints, but when you need a transition tool, it's nice. > NB : by the way, love your blog :p Thanks :)
9y ago

jcolella 9y ago

Great article, specially with the sections on coverage on testing. Also, specifying the process so another can easily reproduce it. Very nice!

jwilk 9y ago

https://github.com/Microsoft/microsoft-pdb/pull/27 https://github.com/Microsoft/microsoft-pdb/pull/27 is labelled cla-not-required, so it's probably not the best example for "need to sign NDA/CLA for sending PR". But some other projects are so hardcore about CLA enforcement, they won't even accept one-letter typo-fixes without CLA signed.

jxramos 9y ago

I really like the suggestion about focusing on PR for PR, that is pull request public relations. PR diplomacy, very sound advice. +1 for articulating this bit of funny business in the Python Windows ecosystem... "Moreover on Windows the building process was historically so bad that you usually end up downloading binaries from some random guy on the Internet."

opportune 9y ago

I once spent a week porting some python2 researchware into python3. It was only at the end of that week that I discovered that it was the actual logic of the researchware that was broken, and not the way I was porting it. I now distrust all FOSS except for widely used and tested ones. You would think that a graduate student would debug (or at least fix to the point of being interpretable) the work that they spent months on before publishing it as finished and adding it to pip. And yes, 2to3 fixes 90% of porting problems. But in my experience, it was especially bad at handling, of all things, import statements. Definitely not a silver bullet.

ofek 9y ago
I'd be weary to conflate OSS with what you refer to as "researchware". Code coming from academia is notorious for poor reproducibility.
- opportune 9y ago
  I'm not conflating them, but clearly software can be both researchware and open source. I'm not going to start distrusting tensorflow or scikit-learn, but I'm going to be wary before I invest time in a promising niche software package, regardless of whether it was actually from academia or not.
- aaronchall 9y ago
  It is tiresome, isn't it.
sevensor 9y ago
You have a remarkably rosy view of grad students and their relationship to software. All it has to do is produce output (I hesitate to say work) one time, on one computer, under some conditions that may never be documented, let alone reproduced. This is doubly true if the code just a step in the analysis chain and not the actual subject of the research.
- opportune 9y ago
  In this case, the software itself was the focus of the research. And this grad student still works at a very prestigious university for CS. I've worked in academia (as a research assistant) before so I'm familiar with shitty grad student software. But going to the effort to document and present on the software without making sure it even worked? That seemed excessively stupid to me.

radarsat1 9y ago

In almost every Python-related project I work on, we're developing in an exciting new language called "Python 2-compatible Python 3 subset". Twice as must testing, twice as much fun!

crncosta 9y ago
Yes, this is exactly the issue. Porting to Python 3 was never an issue itself, but keep your code compatible with Python 2 (while introduce 3) is a nightmare :P
skykooler 9y ago
I've been working on one project that needs to work in every Python verson from 2.4 to 3.3. There are some pretty fun gymnastics getting some syntax to work.
- madez 9y ago
  Why wouldn't we build a vehicle that can drive on streets, on rails, can fly, swim and dig through the earth? It would be a marvellous construction. And awful engineering. I'm not blaming the person who builds the machine in the end, but the person that sets up the specification – they are doing a bad job.
metafunctor 9y ago
Are all these projects some sort of libraries? Is that why you need to support both Python 2 and Python 3? All the projects I've worked on moved to Python 3 and never looked back.
- daveFNbuck 9y ago
  If it's a large project, you can't necessarily move everything to Python 3 all at once. You can't break Python 2 compatibility until everything works in Python 3.
  - radarsat1 9y ago

pinpeliponni 9y ago

Just tell the Python 2 users frankly to go fuck themselves.

melling 9y ago
Swift is 3 years old. Most people who use it will be on the latest version Swift 4 within a few months. Apple won’t let developers stay behind. While it was unrealistic for Python to be that aggressive given its larger community, not forcing the issue created a lot of unnecessary work for the community. A benevolent dictator should have moved developers along faster. Having the language fragmented for this long is extremely unproductive.
- gogopuppygogo 9y ago
  Python 3 faced a lot of friction when it was introduced. There was no backwards compatibility with Python 2 and it had relatively little adoption. They added Python 3 features to Python 2 over the years to encourage adoption of Python 3 as the standard. It worked. Now we just need Apple to ship Mac OS with Python 3 instead of 2 as the default and that'll push adoption for developers.
  - mikebenfield 9y ago
    > They added Python 3 features to Python 2 over the years to encourage adoption of Python 3 as the standard. It worked. Did it really? My impression was the opposite: the fact that so much of Python 3 became available in Python 2 made people feel like there was no point in moving to Python 3.
    - viraptor 9y ago
      It worked both ways. Some new features were ported back, some could be ported, but explicitly weren't to encourage migration. On the other side, things like the "u" prefix got re-added to python 3 for compatibility.
- weberc2 9y ago

carapace 9y ago

Oh dear God why do you hate my eyes. Tiny sans-serif gray body text is of the Devil and should be grounds for irate and overbearing truculent curmudgeons to whine about on online forums.

echion 9y ago

> I’m kinda bummed out that Python devs decided not to backport Python type hints in Python2.7. Can't comment on the site itself, so: PEP 484 - Type hints ( https://www.python.org/dev/peps/pep-0484/#suggested-syntax-for-python-2-7-and-straddling-code https://www.python.org/dev/peps/pep-0484/#suggested-syntax-f... ) does suggest Python 2.7-compatible type hints, and PyCharm, and probably other IDEs, do support them (e.g., https://www.jetbrains.com/help/pycharm/type-hinting-in-pycharm.html https://www.jetbrains.com/help/pycharm/type-hinting-in-pycha... ).

_eht 9y ago

Maybe this is an unpopular opinion coming from someone new to Python (2 years) but the way this versioning was done is definitely a pain point. Working with new code and looking up docs to achieve X, working with existing code and trying to apply the same solution. God it's annoying. Having said that, I know the decisions were not made lightly, and were backed by logical analysis. If nothing else, it's a good case study for the evolution of future languages (looking at you golang 2.0).

breatheoften 9y ago

How about a type checked python variant ala typescript that compiles to python2 or python3? Do: introduce and require the compile step for awhile, flesh out the language and build it up, get by-in and an ecosystem while also leveraging compatibility with python2/3 ecosystems -- then eventually release a python4 that runs this new typed python language directly! Bring us the beauty!

deleted 9y ago
[deleted]
Rotareti 9y ago
Have you tried the typing [0] module? I now use it for every project I'm working on and I find the experience working with it similar to the experience I have working with TypeScript. At least if you use a good IDE that can handle the type annotations. [0] https://docs.python.org/3/library/typing.html https://docs.python.org/3/library/typing.html

seanwilson 9y ago

> Usually, a dev writing a Python 2.7 library will not discriminate between bytes-like buffers and ASCII strings (since the language doesn’t either) which means to have to code review everything str operations and ask yourself if there is a bytes operation implied or is it really a string one. And...that's exactly why you want static typing. Code review to catch stuff like this is poorly and primitively doing a job a computer can do for you perfectly. Dynamic typing is fine for small projects but when you're wanting to port over all Python 2 code that's ever been written to Python 3 types would have made the task much less daunting.

syrrim 9y ago
You're thinking strong typing, not static typing. With stronger types, there would be a difference between a bytes string and unicode string, and the programmer would have to explicitly choose between them. With dynamic typing, this difference can still exists, and it is still obvious from the code. The only difference is that it isn't checked by the compiler at compile time. Look at c. It has static typing, but fairly weak types. Strings are represented as a series of bytes (char *). The exact same work would be required to port c code to semantic c++.
- seanwilson 9y ago
  Whatever terminology you want to use, I mean at compile time you would get an error because the types are wrong for that specific example. Python is generally referred to as strongly dynamically typed, that's why I only wrote "static".
tyingq 9y ago
In this case, a new type was introduced...bytes didn't exist. So I don't know that static typing would have helped much, at least for dealing with existing code. You still have to decide what migrates from type1 to type2. The "bytes" that was put into 2.6 was just an alias to str as hinting for the 2to3 utility.
- seanwilson 9y ago

moyix 9y ago

Well, this isn't exactly how I expected my PDB parsing library to end up on HN ;) The bytes/string changeover is probably the biggest pain point for RE tools written in Python2. Indeed, despite lucasg's excellent work porting things over, there was still at least one byte/string issue that needed to be fixed: https://github.com/moyix/pdbparse/issues/39 https://github.com/moyix/pdbparse/issues/39 Edit: Also, I have one small correction – pdbparse is actually 10 years old, not 5! I started writing it my first year out of college, which also explains (in part) why the code is a bit crap. As evidence I offer the first article I wrote on the PDB format: http://moyix.blogspot.com/2007/08/pdb-stream-decomposition.html http://moyix.blogspot.com/2007/08/pdb-stream-decomposition.h...