4 ms·

> The only somewhat problematic code was this: for line in iter( handle.stdout.readline, '' ): self.output.put( ( handle, line ) ) > This no longer term

> The only somewhat problematic code was this: for line in iter( handle.stdout.readline, '' ): self.output.put( ( handle, line ) ) > This no longer terminated an emitted lots of odd values (one number repeatedly). It turns out it returns an empty b'' now when empty, as opposed to a ''. This could have been avoided had this API sensibly returned None instead of an empty string. For any observers, this is much more idiomatically written: for line in handle.stdout: self.output.put((handle, line)) That is, the default iteration over a file-like object is line by line. Frankly, I wonder how many Python programmers are aware that iter() has a two-arg form; this is the first time I've ever seen it used (and I had to look it up myself). > [PyPI] His solution of maintaining separate code bases is … unusual. I feel like a cursory glance around the library landscape shows that most of the projects out there are maintaining a single codebase that supports 2.7+. six greatly simplifies most of this. I'm not aware of any projects that are supporting Python 2 and 3 concurrently by separating out their code completely. > I presumed that I’d just open a “Migration from Python 2 to 3 Guide” and follow the instructions. Alas, there is no such guide. There appears to be no central resource whatsoever on the porting of code from version 2 to version 3. An I'm Feeling Lucky search on that exact quote pulls up the migration guide in Python's official documentation.[1] [1]: https://docs.python.org/3/howto/pyporting.html https://docs.python.org/3/howto/pyporting.html

agf 10y ago
In addition to what you've pointed out, he claims that 2to3 didn't do anything to his code, and yet he had to fix print statements. 2to3 definitely fixes print statements; he likely didn't read the usage or docs at all. By default it prints a diff, you have to use 2to3 -w if you want it to modify files on disk -- I certainly don't want tools modifying files in-place by default, so that seems like the right API to me.
- andybak 10y ago
  That was the red flag to me. That smacks of someone saying "I haven't read the docs. It didn't work. It sucks." Heck. I haven't done any 2 to 3 porting. I just read a few articles out of idle curiosity once and even I know 2to3 doesn't work like that.
- dr_zoidberg 10y ago
  > By default it prints a diff, you have to use 2to3 -w if you want it to modify files on disk -- I certainly don't want tools modifying files in-place by default, so that seems like the right API to me. Well, running 2to3.py -w -n -o [new_py3_path] [old_py2_path] got me a copy of a project with every fix already applied. Another alternative I've been thinking is doing a "last before 2to3 commit" (and leaving no pending changes), and then run 2to3 inplace and check those from the IDE (which may or may not be easier to a few folks that work with me). I have to check a few more things, but it seems 2017 is the year I'm finally porting then two biggest projects I work on to Python 3. Luckily both projects aren't heavy offenders and should still remain Py2 compatible, but we may have the push with the clients to ask for exlusively Py3 support.
  - brianshaler 10y ago
    You could also test the waters with a `2to3` branch. If you have uncommitted changes, instead of dropping them in as "last before..." you can git-stash them before branching and git-stash-pop them when you come back.
    - throwanem 10y ago
      That would prevent 2to3 from seeing them, though.
      - brianshaler 10y ago
        Yes, because they're pending changes. When they're finished, they can be committed and the branch can be rebased to include them.
        throwanem 10y ago
        At which point you need to run 2to3 again, and maybe update callers of the code you stashed, depending on what it actually does. If you know with certainty it isn't going to be affected in any way by the upgrade tool, of course, then either works as well. But otherwise I'd argue strongly for finishing the changes first, committing them, and only then running the upgrader. It's an argument of preference in practice, and therefore all but guaranteed to go nowhere. I tend to think that updating a whole codebase like this is best done atomically rather than piecemeal, especially when the code belongs to a team rather than an individual. But I can see where opinions might differ on such a point.
- mortoray 10y ago
  The simple syntax changes I did prior to running 2to3 as they came up on the first run I tried. The semantic changes that didn't come up in 2to3 is what I was more bothered about.
  - agf 10y ago
    Then you should update your post to say that, rather than saying 2to3 didn't do anything to your code.
  - mkesper 10y ago
    Try using some of the 2to3 fixers https://docs.python.org/2/library/2to3.html#fixers https://docs.python.org/2/library/2to3.html#fixers like the idioms fixer.
zzleeper 10y ago
About PyPI, it is a bit messy. I backported a py3 project to use py2 (on behalf of a few users) and it was a real pain in the ass to have a wheel that allowed for both versions. I ended up having a py3 build on github, and using a TravisCI build that ran pasteurize and pushed to PyPI that version. At this point, I don't even know what the travis.yml file is doing (100 lines), and regret a bit backporting the project.
- slavik81 10y ago
  What makes it difficult? My projects are all quite simple, but I do keep them compatible with both Python 2 and Python 3. All I had to do was create a setup.cfg that said: [wheel] universal = 1 To build, I use: python setup.py sdist bdist_wheel twine upload -s dist/* The .travis.yml is just 7 lines. https://github.com/cgmb/guardonce/blob/v2.0.0/.travis.yml https://github.com/cgmb/guardonce/blob/v2.0.0/.travis.yml
  - zzleeper 10y ago
    Are you using py3 exclusive features? If you are not, then you are not really using python 3 (tuple unpacking, etc.) If you want to do both, then you need pasteurize, and end up with a travis mess: https://github.com/sergiocorreia/panflute/blob/master/.travis.yml https://github.com/sergiocorreia/panflute/blob/master/.travi... To be honest, for me the morale of the story is that supporting py2 is just not worth it. I am not going to use it, so why support it at all?
    - slavik81 10y ago
      Interesting. I do stick to Python 2 features, but put in some work to ensure my programs are valid Python 3. I'm not a very skilled Python developer, so the simplicity of that strategy is attractive. One thing that I notice is that your project is a library, while mine is a program. I wonder if that also plays a role.
mortoray 10y ago
The `iter` form of the `readline` doesn't load the entire stream first, whereas the second form you posted does. This makes a significant difference when reading data from another program. I saw that migration guide, but it has very few details about what is actually involved in migration. It doesn't include API change details.
- agf 10y ago
  Why do you think the `for` loop form loads the whole stream? That shouldn't be true, since `for` just calls `iter` on the object internally, and normal file-like objects by default are iterated line-by-line. If you saw the migration guide but found it incomplete, you should edit your post to say that rather than saying there isn't one.
- orf 10y ago
  I'm sorry but that is fundamentally incorrect (and shows a big knowledge gap). Interesting article though, thank you for posting it.

agf 10y ago

In addition to what you've pointed out, he claims that 2to3 didn't do anything to his code, and yet he had to fix print statements. 2to3 definitely fixes print statements; he likely didn't read the usage or docs at all. By default it prints a diff, you have to use 2to3 -w if you want it to modify files on disk -- I certainly don't want tools modifying files in-place by default, so that seems like the right API to me.

andybak 10y ago
That was the red flag to me. That smacks of someone saying "I haven't read the docs. It didn't work. It sucks." Heck. I haven't done any 2 to 3 porting. I just read a few articles out of idle curiosity once and even I know 2to3 doesn't work like that.
dr_zoidberg 10y ago
> By default it prints a diff, you have to use 2to3 -w if you want it to modify files on disk -- I certainly don't want tools modifying files in-place by default, so that seems like the right API to me. Well, running 2to3.py -w -n -o [new_py3_path] [old_py2_path] got me a copy of a project with every fix already applied. Another alternative I've been thinking is doing a "last before 2to3 commit" (and leaving no pending changes), and then run 2to3 inplace and check those from the IDE (which may or may not be easier to a few folks that work with me). I have to check a few more things, but it seems 2017 is the year I'm finally porting then two biggest projects I work on to Python 3. Luckily both projects aren't heavy offenders and should still remain Py2 compatible, but we may have the push with the clients to ask for exlusively Py3 support.
- brianshaler 10y ago
  You could also test the waters with a `2to3` branch. If you have uncommitted changes, instead of dropping them in as "last before..." you can git-stash them before branching and git-stash-pop them when you come back.
  - throwanem 10y ago
    That would prevent 2to3 from seeing them, though.
    - brianshaler 10y ago
      Yes, because they're pending changes. When they're finished, they can be committed and the branch can be rebased to include them.
      - throwanem 10y ago
        At which point you need to run 2to3 again, and maybe update callers of the code you stashed, depending on what it actually does. If you know with certainty it isn't going to be affected in any way by the upgrade tool, of course, then either works as well. But otherwise I'd argue strongly for finishing the changes first, committing them, and only then running the upgrader. It's an argument of preference in practice, and therefore all but guaranteed to go nowhere. I tend to think that updating a whole codebase like this is best done atomically rather than piecemeal, especially when the code belongs to a team rather than an individual. But I can see where opinions might differ on such a point.
mortoray 10y ago
The simple syntax changes I did prior to running 2to3 as they came up on the first run I tried. The semantic changes that didn't come up in 2to3 is what I was more bothered about.
- agf 10y ago
  Then you should update your post to say that, rather than saying 2to3 didn't do anything to your code.
- mkesper 10y ago
  Try using some of the 2to3 fixers https://docs.python.org/2/library/2to3.html#fixers https://docs.python.org/2/library/2to3.html#fixers like the idioms fixer.

zzleeper 10y ago

About PyPI, it is a bit messy. I backported a py3 project to use py2 (on behalf of a few users) and it was a real pain in the ass to have a wheel that allowed for both versions. I ended up having a py3 build on github, and using a TravisCI build that ran pasteurize and pushed to PyPI that version. At this point, I don't even know what the travis.yml file is doing (100 lines), and regret a bit backporting the project.

slavik81 10y ago
What makes it difficult? My projects are all quite simple, but I do keep them compatible with both Python 2 and Python 3. All I had to do was create a setup.cfg that said: [wheel] universal = 1 To build, I use: python setup.py sdist bdist_wheel twine upload -s dist/* The .travis.yml is just 7 lines. https://github.com/cgmb/guardonce/blob/v2.0.0/.travis.yml https://github.com/cgmb/guardonce/blob/v2.0.0/.travis.yml
- zzleeper 10y ago
  Are you using py3 exclusive features? If you are not, then you are not really using python 3 (tuple unpacking, etc.) If you want to do both, then you need pasteurize, and end up with a travis mess: https://github.com/sergiocorreia/panflute/blob/master/.travis.yml https://github.com/sergiocorreia/panflute/blob/master/.travi... To be honest, for me the morale of the story is that supporting py2 is just not worth it. I am not going to use it, so why support it at all?
  - slavik81 10y ago
    Interesting. I do stick to Python 2 features, but put in some work to ensure my programs are valid Python 3. I'm not a very skilled Python developer, so the simplicity of that strategy is attractive. One thing that I notice is that your project is a library, while mine is a program. I wonder if that also plays a role.

mortoray 10y ago

The `iter` form of the `readline` doesn't load the entire stream first, whereas the second form you posted does. This makes a significant difference when reading data from another program. I saw that migration guide, but it has very few details about what is actually involved in migration. It doesn't include API change details.

agf 10y ago
Why do you think the `for` loop form loads the whole stream? That shouldn't be true, since `for` just calls `iter` on the object internally, and normal file-like objects by default are iterated line-by-line. If you saw the migration guide but found it incomplete, you should edit your post to say that rather than saying there isn't one.
orf 10y ago
I'm sorry but that is fundamentally incorrect (and shows a big knowledge gap). Interesting article though, thank you for posting it.