11 ms·

Python 2, Python 3, Debian and Porting

195 points by neokya 12y ago

stefantalpalaru 12y ago
> Booo, Python 2! It would be better for everybody if we'd treat Python 2 and Python 3 like separate languages. Then we can reason about the sanity of spending development time to port existing and working code from one language to another.
- isaacdl 12y ago
  I concur. In this case, I also think the development time to port the code to Python 3 is worth it, simple because Python 2 will be EOLed in 2020, which isn't that far away!
- Walkman 12y ago
  The problem is they are really really similar and you can write code that runs on Python2 and Python 3 at the same time.
- eeZi 12y ago
  Treating them as seperate languages would be silly, since they aren't. There are some gotchas, but it's far from porting code to a different language.
diminoten 12y ago
Am I correct to assume this would be a fantastic place to start contributing to open source projects by helping convert these Debian tools over to Python 3?
- k2enemy 12y ago
  Yes, as noted in the message: If you're interested in this effort, please email me. This is a really good new contributor task, so if anyone's asked you how they could get involved with Debian, you should send them to us!
  - paultag 12y ago
    Yep! My email is paultag@debian.org - sign up for the porting list @ https://lists.alioth.debian.org/mailman/listinfo/py3porters-devel https://lists.alioth.debian.org/mailman/listinfo/py3porters-... Yes, it's hilarious that version of Mailmain needs to be ported to Python 3.
    - aidos 12y ago
      This actually sounds really interesting. I'll send you an email. Is there a list of projects that need porting? Would be good to see what sort of size they were.
      - rspeer 12y ago
        http://py3readiness.org/ http://py3readiness.org/ Major things that would be useful to port for Debian's sake, because they're system utilities: supervisor, Fabric, carbon Major things that would be useful to port because Python projects depend on them: Twisted, thrift, protobuf Don't bother with these because replacements already exist: MySQL-python, oauth2
    - kerkeslager 12y ago
      Paul, I already emailed you but I'm responding here because I have a feeling your response will be useful to other people interested in helping: could you give some direction for people like me who know Python 2 and/or 3 and want to help, but don't know the first thing about the structure of Debian packages or how they are developed? For example: I'm looking at ./main/f/flask-wtf/flask-wtf_0.10.2-1.json and I'm not even sure what I'm looking at. The contents of that file are: {"has_python3_module": false, "canidate": true, "trove_python3": true, "has_python2_module": true} What's all that mean? On my machine `apt-cache search flask-wtf` finds a package called `python-flaskext-wtf`. Digging deeper with `apt-cache showpkg` I find the homepage http://packages.python.org/Flask-WTF/ http://packages.python.org/Flask-WTF/, which in turn gets me to the code https://github.com/lepture/flask-wtf https://github.com/lepture/flask-wtf. I haven't updated the code to Python 3, but let's assume for a second that I have: now what? How do I tie this all together? Do I just send you a patch for this code? Is this even the right code? If you point me in the right direction I'll be happy to write up a tutorial to help other folks contribute. And as a side note: I really like Hy.
- patricksanders 12y ago
  Absolutely. I've been looking for a good place to start contributing, so this is very opportune.
FreakyT 12y ago
Nice, I'm glad to see that Python 3 adoption is finally picking up steam! Python 2 is nice, but it has really started to overstay its welcome, I think.
chimeracoder 12y ago
> Python 2 is scheduled to be EOL'd upstream officially and for good in 2020. We're in 2015 now (wow, that went quickly), and keeping our release cadence up (3 years a pop) puts Stretch up in 2018, and Buster in 2021. Am I missing something? Did Debian decide to extend their release cycles? Jessie is due to be released next week, which is 2 years since Wheezy (May 2013). In fact, the last several years have all been 2-year cycles, not 3. The only three-year cycle was Woody → Sarge (2002-2005), and AFAIK, that delay was one of the reasons that Ubuntu had room to enter the market. Since this put pressure on Debian to make their development more rapid, I'd be surprised if they decided to move in the opposite direction.
- paultag 12y ago
  Yes, everyone's making fun of me for not saying 2 years rather then 3. Also, our releases aren't timed, they're based on bug count, so the timing of the releases wasn't even the main point (and doesn't really matter) :) I also typoed Python 3 as Python 2 in one of the lines, so, uh, just switch those two numbers. I'm pretty sure it works like that. Either way, getting Buster off Python 2 means the transition will be more bug free, since we'll have a bit of slack time.
  - deleted 12y ago
    [deleted]
- deleted 12y ago
  [deleted]
krylon 12y ago
Oh man. I've been meaning to check out Python 3 for a couple of years now, but so far I did not have a compelling reason. Maybe it's time to start anyway.
- Estragon 12y ago
  Yeah, reading through this thread, I haven't seen a single reason why I should care about Python 3.
  - task_queue 12y ago
    Because Python 2 will be phased out in 5 years.
    - BuckRogers 12y ago
      By who? The core developer team? Or by PyPy and Pyston? The people that matter (the users) aren't phasing out 2.x in 5 years. That much is clear.
  - krylon 12y ago
    I looked into the changes between Python 2 and 3 a few years back (like, 2011, 2012). There lots of minor changes that were all nice, but no single big change that was a compelling reason to switch. If, e.g. they had ditched the GIL and allowed pure Python code to exploit multicore systems, that would have been compelling to me. Instead, at the time, Wikipedia claimed that Python 3 ran somewhat slower on CPython than Python 2. :( Maybe all of those minor changes taken together constitute a big improvement, once I give it a try. The Unicode handling is said to be much better, and Unicode handling does suck really, really hard on Python 2. So there's that.
- bayesianhorse 12y ago
  Two compelling reasons: Blender and Asyncio. Blender has been on Py3K since 2008 and is an underrated software platform. Asyncio is a revolutionary concurrency framework based on coroutines. The "yield from" feature is useful both for coroutines and for other generator functions.
afarrell 12y ago
If you want to switch back and forth between python2 and python3, one option is to install miniconda (like Anaconda, but with just the package manager) as your Python distribution and create a python2 environment (conda create --name=myproject-py2 python=2.7 && source activate myproject-py2) that you can then pip install the python2-only libraries into. Then, when your libraries support Python3, create a new Python3 environment and install them into that (conda env export > environment.yml && conda create --name=myproject-py3 python=3.4 && conda env update --name=myproject-py3 --file=environment.yml && source activate myproject-py3) See http://conda.io http://conda.io Disclosure: conda is open-source, but I work for company that sponsors most of the development.
- Foxboron 12y ago
  I believe virtualenv (with/without virtualenvwrapper) is most widely used. Not sure if there are any advantages/disadvantages compared to conda tho. https://virtualenv.pypa.io/en/latest/ https://virtualenv.pypa.io/en/latest/ https://virtualenvwrapper.readthedocs.org/en/latest/ https://virtualenvwrapper.readthedocs.org/en/latest/
  - a3n 12y ago
    One disadvantage of conda/anaconda is that you can't, or are discouraged from using virtualenv with conda/anaconda. I use Anaconda on Win7, like it a lot, but I haven't figured out how to use, say, the python 2 that comes with anaconda alongside the python 2 that comes from python.org. I've looked around, haven't really understood what I should be looking at for that. But whatever it is, it isn't virtualenv if you use Anaconda.
    - bkcooper 12y ago
      One disadvantage of conda/anaconda is that you can't, or are discouraged from using virtualenv with conda/anaconda. True, but conda has its own virtual environments that work quite nicely. You can make a new one with conda create and switch between them with activate/deactivate. You even get the same sort of interaction with pip (for the packages you want that aren't in conda or binstar), and I've been able to use e.g. Emacs packages that expect to work with virtualenv and have (mostly) had success using conda's envs instead.
ak217 12y ago
Shameless plug of a modern Python 2/3 porting tool that I wrote: https://github.com/kislyuk/eight https://github.com/kislyuk/eight, which builds upon the excellent work in https://pypi.python.org/pypi/future https://pypi.python.org/pypi/future.
- eeZi 12y ago
  This should be higher up. Not cluttering the code (as with six) sounds great.
- aidos 12y ago
  That looks like a really good solution - and an ideal approach as we get to keep writing in Python 3. Are there any large caveats?
  - ak217 12y ago
    There is still a delta between how unicode behaves on 2 and how str behaves on 3, for example. In general, you'll be fine as long as you have a test suite that runs in CI for both 2 and 3 - to catch incompatibilities early.
grumbel 12y ago
> We can all soon look forward to the day where we no longer have to play Unicode whack-a-mole I don't expect Unicode trouble to disappear with Python3, quite the opposite, it's trivially to insert ticking Unicode time bombs into Python3 code, especially in things surrounding shell scripting. Something as trivial as this: print(os.listdir()) Will explode when it runs into filenames that aren't encodable by Unicode (i.e. perfectly legal filenames in Linux). Linux allows raw binary in a lot of places (filenames, arguments, environment variables) where Python expects Unicode.
- eliben 12y ago
  Are you implying that by using Python 3 incorrectly you can run into unicode problems? I agree with that. However, there are ways to do this safely. At the boundaries of your code you have to deal with the unicode/bytes question, in any language out there. The good thing about Python 3 is that once that boundary is crossed, unicode and bytes are nicely separated.
  - grumbel 12y ago
    > Are you implying that by using Python 3 incorrectly you can run into unicode problems? The problem is that `print(some_string)` is now incorrect, even so it looks perfectly harmless. So far I haven't seen any "best practice" guide on how to fix that. `sys.stdout.write(os.fsdecode(some_string)` works, but is pretty ugly. Handling all filenames as bytes instead of strings works as well, but again, that gets ugly quick. Cleanest way seems to be: PYTHONIOENCODING=utf-8:surrogateescape python3 somescript.py But should I depend on users setting `PYTHONIOENCODING`? Should I do that in my scripts? It feels all very rough and causes a lot of issues that you really shouldn't have to deal with in the first place. > The good thing about Python 3 is that once that boundary is crossed, unicode and bytes are nicely separated. Not exactly "nicely separated", the unencodable bytes get squished as surrogates into the Unicode strings waiting to cause trouble later on.
    - the_mitsuhiko 12y ago
      JFTR. In Python 3.4 and later I think, stdout has a surrogateescape error handler by default. Not that it helps much, because this will fall apart if you write to other things. The story with this apparently is now so confusing that I am not sure what the correct way to do it on Python 3 is. From my experience most people don't care anyways. In some of my libraries the Python 3 version just refuses to work unless the encoding is utf-8 to avoid accidental failures down the line which nobody can debug.
winter_blue 12y ago
This seems like a good time and opportunity for Debian to try their hand at one of the recent languages that have popped up like Rust, Elixir, etc.
- task_queue 12y ago
  You're being downvoted because writing Debian's system infrastructure in a language whose spec isn't finalized or is not well known is a Bad Idea when other options exist. Python 2 -> Python 3 is a trivial port compared to adopting a new language and the technical/cultural baggage that come with it.
- nkuttler 12y ago
  Sounds like a good time for you to volunteer.
upofadown 12y ago
I am getting really tired of this "if Python3 is to live then Python2 must die" stuff... Perhaps Debian should avoid the war entirely and switch to another scripting language.
- Karunamon 12y ago
  Python2 is a ticking time bomb - it's officialy EOL'ed in 2020, all new development is happening on 3.x It's like the jump between Ruby 1.8 and 2.0 - there is no reason to be doing anything in 1.8 in 2015.
  - Yuioup 12y ago
    I predict that Python 2 will be forked by a drop-in replacement by 2020.
    - lmm 12y ago
      I'll bet you $50 that no non-maintenance fork of python2 will be shipped by default in any of the top 5 linux distributions before 2020.
      - BuckRogers 12y ago
        Only because the core dev team is making it their priority to move distros over to 3. It won't make any difference, the strength of Python is in the library support, not the distro support. 2.x support by libraries will last a very long time, nothing worth its salt will be dropping it. By the time they do, Python3 and wherever it is by that point will be unrecognizable to the original release of Python 3.0.
rspeer 12y ago
When it comes to finding which packages need to be ported: I've been disappointed by the "Python 3 Wall of Superpowers" and the fact that they've basically never updated their package list. (They update the Python 3 readiness, but you'd never find out if a popular package created in the last 4 years was incompatible, or if packages on the list have become unpopular.) Today I found py3readiness.org, which is much more relevant and up-to-date. It has a better call to action, too. http://py3readiness.org/ http://py3readiness.org/
- sontek 12y ago
  I find https://caniusepython3.com/ https://caniusepython3.com/ very helpful
- neokya 12y ago
  OP and maintainer of the above site here. Thanks for noticing py3readiness.org Yes, it's up-to-date and takes advantage of https://github.com/brettcannon/caniusepython3 https://github.com/brettcannon/caniusepython3 Also, you can help to keep it updated https://github.com/chhantyal/py3readiness https://github.com/chhantyal/py3readiness
  - jordanb 12y ago
    Both lists should replace python-ldap with ldap3. The former is abandon-ware and will never (probably) be upgraded. The latter already works great with python 3 (and is a better library all-around).
    - rspeer 12y ago
      There are more packages like that, in fact: * Python-MySQL is abandoned, but it's been superseded by python-mysql-connector. * oauth2 is kind of superseded by oauthlib. * pathtools can probably be replaced with Python 3's pathlib. * ipaddr can be replaced with iptools, which has the additional advantage that you can contribute to it without approval from Google's lawyers.
      - 12y ago
deckiedan 12y ago
As someone who uses a lot of Python scripting for sys-admin automation stuff, is there anyone here on HN who could ask apple to start distributing a python3 binary with the OS by default? Otherwise we're going to end up with a similar situation as BASH and other utils in a few years time, where OSX comes with Python2.7.x, and everything else in the world is Python3... (Although, by 2020, apple will probably have deprecated OSX in favour of iOS, and have javascript as the only blessed scripting language...)
- SEJeff 12y ago
  s/javascript/swift/ :)
  - deckiedan 12y ago
    Ha, yes. I kind of forgot about swift. But, remember Atwood's Law and all. Apple have invested quite a bit into JS recently, with their new LLVM JIT for Safari, I believe? (OK, by Apple standards, not much investment, but the fact that they did it at all says quite a lot...). Since they did all that work, and are committed to maintaining it now, would it not (in some way) make sense to then use the same pluggable scripting language VM into other parts of the OS, and deprecate applescript, etc?
- bkeroack 12y ago
  $ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" $ brew install python3 pypy3 (to be less pithy, it's super easy to write code that will run on both 2.7 and 3.x -- avoid print-as-statement and use format strings instead of '%' for interpolation and that will get you 95% of the way there--probably 99-100% of the way for simple automation scripts)
  - Thrymr 12y ago
    > use format strings instead of '%' for interpolation To be clear, %-formatting has not gone away in Python 3: https://docs.python.org/3/library/stdtypes.html#printf-style-string-formatting https://docs.python.org/3/library/stdtypes.html#printf-style...
kristjanvalur 12y ago
Or just fork python 2 and keep on trucking there in happy land.
bayesianhorse 12y ago
Interesting side node: Blender (an open source graphics suite) switched to Python 3 in like 2008 (so far as I can determine). The reasoning probably was "Hey, the rest of the Python community would probably leave us in the dust, if we don't..."
syllogism 12y ago
My biggest peeve writing Python 3 is pretty trivial... I cannot get used to typing print() Every time I go to debug a line, I get a syntax error, and have to go back and change it. And I type a lot of debug lines! I need to find a vim extension to handle this, but so far it just makes me frustrated whenever I try to type Python 3.
- rquirk 12y ago
  Have you tried using the logging module? Might be overkill for really small scripts as it requires a few lines to set up, but even the tiniest tool ends up requiring print-debugging at some point. With logging it works the same in 2/3, you can leave all the debugging statements in, and if a library also uses it then you get additional debug output from that too. Add a "debug" command line flag to pass when something goes wrong down the road (to change the default log level), and you don't have to add/remove statements for debugging. For a vim extension - I use snipMate http://www.vim.org/scripts/script.php?script_id=2540 http://www.vim.org/scripts/script.php?script_id=2540 It's a bit on the dead side but works great as-is. ifmain followed by the expander key (shift-tab in my case) expands to the if __name__ ... idiom. You could add your own "p" snippet that expanded to print() in python code.
- lmm 12y ago
  Yeah that really is trivial. print now works like any other function, like it does in most languages. The new way really is objectively better. Short of allowing all functions to be called as "function argument" à la Haskell (which I would support, but would be a massive change) I don't see any alternative.
rkuska 12y ago
We (Fedora) are step ahead. Currently we are waiting for a last few packages to be ported (anaconda is already running on python3 and ready for Fedora 23 release). Ironically the hardest part isn't porting (we've sent a huge amount of patches to upstream projects) but getting upstreams to accept the python3 as supported interpreter ("what is this python3 thing?" yes, this is a real quote). I am happy to see a new distros joinning a party, gl!
NeoAbdn 12y ago
Someone asked why not swap to Elixir or Rust, but why not actually swap to the sysadmins dream of Ruby?
- Johnny_Brahms 12y ago
  I would love that. Ruby has become my go-to language for quick scripts, even in places where bash does the job. Bash is for things where I can keep everything in my head and there aren't too many lines. Ruby for everything else.
- task_queue 12y ago
  I don't know a sysadmin on this planet that dreams of using Ruby as systems scripting language. That sounds like hell. Again, its the trade off between a relatively simple port and adopting a new language with different standards, technical difficulties and culture.

195 points by neokya 12y ago

> Booo, Python 2! It would be better for everybody if we'd treat Python 2 and Python 3 like separate languages. Then we can reason about the sanity of spending development time to port existing and working code from one language to another.

isaacdl 12y ago
I concur. In this case, I also think the development time to port the code to Python 3 is worth it, simple because Python 2 will be EOLed in 2020, which isn't that far away!
Walkman 12y ago
The problem is they are really really similar and you can write code that runs on Python2 and Python 3 at the same time.
eeZi 12y ago
Treating them as seperate languages would be silly, since they aren't. There are some gotchas, but it's far from porting code to a different language.

diminoten 12y ago

Am I correct to assume this would be a fantastic place to start contributing to open source projects by helping convert these Debian tools over to Python 3?

k2enemy 12y ago
Yes, as noted in the message: If you're interested in this effort, please email me. This is a really good new contributor task, so if anyone's asked you how they could get involved with Debian, you should send them to us!
- paultag 12y ago
  Yep! My email is paultag@debian.org - sign up for the porting list @ https://lists.alioth.debian.org/mailman/listinfo/py3porters-devel https://lists.alioth.debian.org/mailman/listinfo/py3porters-... Yes, it's hilarious that version of Mailmain needs to be ported to Python 3.
  - aidos 12y ago
    This actually sounds really interesting. I'll send you an email. Is there a list of projects that need porting? Would be good to see what sort of size they were.
    - rspeer 12y ago
      http://py3readiness.org/ http://py3readiness.org/ Major things that would be useful to port for Debian's sake, because they're system utilities: supervisor, Fabric, carbon Major things that would be useful to port because Python projects depend on them: Twisted, thrift, protobuf Don't bother with these because replacements already exist: MySQL-python, oauth2
  - kerkeslager 12y ago
    Paul, I already emailed you but I'm responding here because I have a feeling your response will be useful to other people interested in helping: could you give some direction for people like me who know Python 2 and/or 3 and want to help, but don't know the first thing about the structure of Debian packages or how they are developed? For example: I'm looking at ./main/f/flask-wtf/flask-wtf_0.10.2-1.json and I'm not even sure what I'm looking at. The contents of that file are: {"has_python3_module": false, "canidate": true, "trove_python3": true, "has_python2_module": true} What's all that mean? On my machine `apt-cache search flask-wtf` finds a package called `python-flaskext-wtf`. Digging deeper with `apt-cache showpkg` I find the homepage http://packages.python.org/Flask-WTF/ http://packages.python.org/Flask-WTF/, which in turn gets me to the code https://github.com/lepture/flask-wtf https://github.com/lepture/flask-wtf. I haven't updated the code to Python 3, but let's assume for a second that I have: now what? How do I tie this all together? Do I just send you a patch for this code? Is this even the right code? If you point me in the right direction I'll be happy to write up a tutorial to help other folks contribute. And as a side note: I really like Hy.
patricksanders 12y ago
Absolutely. I've been looking for a good place to start contributing, so this is very opportune.

FreakyT 12y ago

Nice, I'm glad to see that Python 3 adoption is finally picking up steam! Python 2 is nice, but it has really started to overstay its welcome, I think.

chimeracoder 12y ago

> Python 2 is scheduled to be EOL'd upstream officially and for good in 2020. We're in 2015 now (wow, that went quickly), and keeping our release cadence up (3 years a pop) puts Stretch up in 2018, and Buster in 2021. Am I missing something? Did Debian decide to extend their release cycles? Jessie is due to be released next week, which is 2 years since Wheezy (May 2013). In fact, the last several years have all been 2-year cycles, not 3. The only three-year cycle was Woody → Sarge (2002-2005), and AFAIK, that delay was one of the reasons that Ubuntu had room to enter the market. Since this put pressure on Debian to make their development more rapid, I'd be surprised if they decided to move in the opposite direction.

paultag 12y ago
Yes, everyone's making fun of me for not saying 2 years rather then 3. Also, our releases aren't timed, they're based on bug count, so the timing of the releases wasn't even the main point (and doesn't really matter) :) I also typoed Python 3 as Python 2 in one of the lines, so, uh, just switch those two numbers. I'm pretty sure it works like that. Either way, getting Buster off Python 2 means the transition will be more bug free, since we'll have a bit of slack time.
- deleted 12y ago
  [deleted]
deleted 12y ago
[deleted]

krylon 12y ago

Oh man. I've been meaning to check out Python 3 for a couple of years now, but so far I did not have a compelling reason. Maybe it's time to start anyway.

Estragon 12y ago
Yeah, reading through this thread, I haven't seen a single reason why I should care about Python 3.
- task_queue 12y ago
  Because Python 2 will be phased out in 5 years.
  - BuckRogers 12y ago
    By who? The core developer team? Or by PyPy and Pyston? The people that matter (the users) aren't phasing out 2.x in 5 years. That much is clear.
- krylon 12y ago
  I looked into the changes between Python 2 and 3 a few years back (like, 2011, 2012). There lots of minor changes that were all nice, but no single big change that was a compelling reason to switch. If, e.g. they had ditched the GIL and allowed pure Python code to exploit multicore systems, that would have been compelling to me. Instead, at the time, Wikipedia claimed that Python 3 ran somewhat slower on CPython than Python 2. :( Maybe all of those minor changes taken together constitute a big improvement, once I give it a try. The Unicode handling is said to be much better, and Unicode handling does suck really, really hard on Python 2. So there's that.
bayesianhorse 12y ago
Two compelling reasons: Blender and Asyncio. Blender has been on Py3K since 2008 and is an underrated software platform. Asyncio is a revolutionary concurrency framework based on coroutines. The "yield from" feature is useful both for coroutines and for other generator functions.

afarrell 12y ago

If you want to switch back and forth between python2 and python3, one option is to install miniconda (like Anaconda, but with just the package manager) as your Python distribution and create a python2 environment (conda create --name=myproject-py2 python=2.7 && source activate myproject-py2) that you can then pip install the python2-only libraries into. Then, when your libraries support Python3, create a new Python3 environment and install them into that (conda env export > environment.yml && conda create --name=myproject-py3 python=3.4 && conda env update --name=myproject-py3 --file=environment.yml && source activate myproject-py3) See http://conda.io http://conda.io Disclosure: conda is open-source, but I work for company that sponsors most of the development.

Foxboron 12y ago
I believe virtualenv (with/without virtualenvwrapper) is most widely used. Not sure if there are any advantages/disadvantages compared to conda tho. https://virtualenv.pypa.io/en/latest/ https://virtualenv.pypa.io/en/latest/ https://virtualenvwrapper.readthedocs.org/en/latest/ https://virtualenvwrapper.readthedocs.org/en/latest/
- a3n 12y ago
  One disadvantage of conda/anaconda is that you can't, or are discouraged from using virtualenv with conda/anaconda. I use Anaconda on Win7, like it a lot, but I haven't figured out how to use, say, the python 2 that comes with anaconda alongside the python 2 that comes from python.org. I've looked around, haven't really understood what I should be looking at for that. But whatever it is, it isn't virtualenv if you use Anaconda.
  - bkcooper 12y ago
    One disadvantage of conda/anaconda is that you can't, or are discouraged from using virtualenv with conda/anaconda. True, but conda has its own virtual environments that work quite nicely. You can make a new one with conda create and switch between them with activate/deactivate. You even get the same sort of interaction with pip (for the packages you want that aren't in conda or binstar), and I've been able to use e.g. Emacs packages that expect to work with virtualenv and have (mostly) had success using conda's envs instead.

ak217 12y ago

Shameless plug of a modern Python 2/3 porting tool that I wrote: https://github.com/kislyuk/eight https://github.com/kislyuk/eight, which builds upon the excellent work in https://pypi.python.org/pypi/future https://pypi.python.org/pypi/future.

eeZi 12y ago
This should be higher up. Not cluttering the code (as with six) sounds great.
aidos 12y ago
That looks like a really good solution - and an ideal approach as we get to keep writing in Python 3. Are there any large caveats?
- ak217 12y ago
  There is still a delta between how unicode behaves on 2 and how str behaves on 3, for example. In general, you'll be fine as long as you have a test suite that runs in CI for both 2 and 3 - to catch incompatibilities early.

grumbel 12y ago

> We can all soon look forward to the day where we no longer have to play Unicode whack-a-mole I don't expect Unicode trouble to disappear with Python3, quite the opposite, it's trivially to insert ticking Unicode time bombs into Python3 code, especially in things surrounding shell scripting. Something as trivial as this: print(os.listdir()) Will explode when it runs into filenames that aren't encodable by Unicode (i.e. perfectly legal filenames in Linux). Linux allows raw binary in a lot of places (filenames, arguments, environment variables) where Python expects Unicode.

eliben 12y ago
Are you implying that by using Python 3 incorrectly you can run into unicode problems? I agree with that. However, there are ways to do this safely. At the boundaries of your code you have to deal with the unicode/bytes question, in any language out there. The good thing about Python 3 is that once that boundary is crossed, unicode and bytes are nicely separated.
- grumbel 12y ago
  > Are you implying that by using Python 3 incorrectly you can run into unicode problems? The problem is that `print(some_string)` is now incorrect, even so it looks perfectly harmless. So far I haven't seen any "best practice" guide on how to fix that. `sys.stdout.write(os.fsdecode(some_string)` works, but is pretty ugly. Handling all filenames as bytes instead of strings works as well, but again, that gets ugly quick. Cleanest way seems to be: PYTHONIOENCODING=utf-8:surrogateescape python3 somescript.py But should I depend on users setting `PYTHONIOENCODING`? Should I do that in my scripts? It feels all very rough and causes a lot of issues that you really shouldn't have to deal with in the first place. > The good thing about Python 3 is that once that boundary is crossed, unicode and bytes are nicely separated. Not exactly "nicely separated", the unencodable bytes get squished as surrogates into the Unicode strings waiting to cause trouble later on.
  - the_mitsuhiko 12y ago
    JFTR. In Python 3.4 and later I think, stdout has a surrogateescape error handler by default. Not that it helps much, because this will fall apart if you write to other things. The story with this apparently is now so confusing that I am not sure what the correct way to do it on Python 3 is. From my experience most people don't care anyways. In some of my libraries the Python 3 version just refuses to work unless the encoding is utf-8 to avoid accidental failures down the line which nobody can debug.

winter_blue 12y ago

This seems like a good time and opportunity for Debian to try their hand at one of the recent languages that have popped up like Rust, Elixir, etc.

task_queue 12y ago
You're being downvoted because writing Debian's system infrastructure in a language whose spec isn't finalized or is not well known is a Bad Idea when other options exist. Python 2 -> Python 3 is a trivial port compared to adopting a new language and the technical/cultural baggage that come with it.
nkuttler 12y ago
Sounds like a good time for you to volunteer.

upofadown 12y ago

I am getting really tired of this "if Python3 is to live then Python2 must die" stuff... Perhaps Debian should avoid the war entirely and switch to another scripting language.

Karunamon 12y ago
Python2 is a ticking time bomb - it's officialy EOL'ed in 2020, all new development is happening on 3.x It's like the jump between Ruby 1.8 and 2.0 - there is no reason to be doing anything in 1.8 in 2015.
- Yuioup 12y ago
  I predict that Python 2 will be forked by a drop-in replacement by 2020.
  - lmm 12y ago
    I'll bet you $50 that no non-maintenance fork of python2 will be shipped by default in any of the top 5 linux distributions before 2020.
    - BuckRogers 12y ago
      Only because the core dev team is making it their priority to move distros over to 3. It won't make any difference, the strength of Python is in the library support, not the distro support. 2.x support by libraries will last a very long time, nothing worth its salt will be dropping it. By the time they do, Python3 and wherever it is by that point will be unrecognizable to the original release of Python 3.0.

rspeer 12y ago

When it comes to finding which packages need to be ported: I've been disappointed by the "Python 3 Wall of Superpowers" and the fact that they've basically never updated their package list. (They update the Python 3 readiness, but you'd never find out if a popular package created in the last 4 years was incompatible, or if packages on the list have become unpopular.) Today I found py3readiness.org, which is much more relevant and up-to-date. It has a better call to action, too. http://py3readiness.org/ http://py3readiness.org/

sontek 12y ago
I find https://caniusepython3.com/ https://caniusepython3.com/ very helpful
neokya 12y ago
OP and maintainer of the above site here. Thanks for noticing py3readiness.org Yes, it's up-to-date and takes advantage of https://github.com/brettcannon/caniusepython3 https://github.com/brettcannon/caniusepython3 Also, you can help to keep it updated https://github.com/chhantyal/py3readiness https://github.com/chhantyal/py3readiness
- jordanb 12y ago
  Both lists should replace python-ldap with ldap3. The former is abandon-ware and will never (probably) be upgraded. The latter already works great with python 3 (and is a better library all-around).
  - rspeer 12y ago
    There are more packages like that, in fact: * Python-MySQL is abandoned, but it's been superseded by python-mysql-connector. * oauth2 is kind of superseded by oauthlib. * pathtools can probably be replaced with Python 3's pathlib. * ipaddr can be replaced with iptools, which has the additional advantage that you can contribute to it without approval from Google's lawyers.
    - 12y ago

deckiedan 12y ago

As someone who uses a lot of Python scripting for sys-admin automation stuff, is there anyone here on HN who could ask apple to start distributing a python3 binary with the OS by default? Otherwise we're going to end up with a similar situation as BASH and other utils in a few years time, where OSX comes with Python2.7.x, and everything else in the world is Python3... (Although, by 2020, apple will probably have deprecated OSX in favour of iOS, and have javascript as the only blessed scripting language...)

SEJeff 12y ago
s/javascript/swift/ :)
- deckiedan 12y ago
  Ha, yes. I kind of forgot about swift. But, remember Atwood's Law and all. Apple have invested quite a bit into JS recently, with their new LLVM JIT for Safari, I believe? (OK, by Apple standards, not much investment, but the fact that they did it at all says quite a lot...). Since they did all that work, and are committed to maintaining it now, would it not (in some way) make sense to then use the same pluggable scripting language VM into other parts of the OS, and deprecate applescript, etc?
bkeroack 12y ago
$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" $ brew install python3 pypy3 (to be less pithy, it's super easy to write code that will run on both 2.7 and 3.x -- avoid print-as-statement and use format strings instead of '%' for interpolation and that will get you 95% of the way there--probably 99-100% of the way for simple automation scripts)
- Thrymr 12y ago
  > use format strings instead of '%' for interpolation To be clear, %-formatting has not gone away in Python 3: https://docs.python.org/3/library/stdtypes.html#printf-style-string-formatting https://docs.python.org/3/library/stdtypes.html#printf-style...

kristjanvalur 12y ago

Or just fork python 2 and keep on trucking there in happy land.

bayesianhorse 12y ago

Interesting side node: Blender (an open source graphics suite) switched to Python 3 in like 2008 (so far as I can determine). The reasoning probably was "Hey, the rest of the Python community would probably leave us in the dust, if we don't..."

syllogism 12y ago

My biggest peeve writing Python 3 is pretty trivial... I cannot get used to typing print() Every time I go to debug a line, I get a syntax error, and have to go back and change it. And I type a lot of debug lines! I need to find a vim extension to handle this, but so far it just makes me frustrated whenever I try to type Python 3.

rquirk 12y ago
Have you tried using the logging module? Might be overkill for really small scripts as it requires a few lines to set up, but even the tiniest tool ends up requiring print-debugging at some point. With logging it works the same in 2/3, you can leave all the debugging statements in, and if a library also uses it then you get additional debug output from that too. Add a "debug" command line flag to pass when something goes wrong down the road (to change the default log level), and you don't have to add/remove statements for debugging. For a vim extension - I use snipMate http://www.vim.org/scripts/script.php?script_id=2540 http://www.vim.org/scripts/script.php?script_id=2540 It's a bit on the dead side but works great as-is. ifmain followed by the expander key (shift-tab in my case) expands to the if __name__ ... idiom. You could add your own "p" snippet that expanded to print() in python code.
lmm 12y ago
Yeah that really is trivial. print now works like any other function, like it does in most languages. The new way really is objectively better. Short of allowing all functions to be called as "function argument" à la Haskell (which I would support, but would be a massive change) I don't see any alternative.

rkuska 12y ago

We (Fedora) are step ahead. Currently we are waiting for a last few packages to be ported (anaconda is already running on python3 and ready for Fedora 23 release). Ironically the hardest part isn't porting (we've sent a huge amount of patches to upstream projects) but getting upstreams to accept the python3 as supported interpreter ("what is this python3 thing?" yes, this is a real quote). I am happy to see a new distros joinning a party, gl!

NeoAbdn 12y ago

Someone asked why not swap to Elixir or Rust, but why not actually swap to the sysadmins dream of Ruby?

Johnny_Brahms 12y ago
I would love that. Ruby has become my go-to language for quick scripts, even in places where bash does the job. Bash is for things where I can keep everything in my head and there aren't too many lines. Ruby for everything else.
task_queue 12y ago
I don't know a sysadmin on this planet that dreams of using Ruby as systems scripting language. That sounds like hell. Again, its the trade off between a relatively simple port and adopting a new language with different standards, technical difficulties and culture.