8 ms·

From the outside, Python 3 seems like a much better language. I don't have strong views of its object system (I avoid OOP as much as I can) but it seems like th

From the outside, Python 3 seems like a much better language. I don't have strong views of its object system (I avoid OOP as much as I can) but it seems like the string/bytes handling is much better, and I'm also a fan of map and filter returning generators rather than being hard-coded to a list implementation (stream fusion is a good thing). Also, I fail to see any value in print being a special keyword instead of a regular function (as in 3). What I don't get is: why has Python 3 adoption been so slow? Is it just backward compatibility, or are there deeper problems with it that I'm not aware of?

harryf 11y ago
What makes me snarky is the replacing of '%s %s' % ('one', 'two') With '%s %s'.format('one', 'two') The latter is just more annoying to type. Stupid argument I know but I find myself grumbling to myself every time...
- davidism 11y ago
  It's not replaced, both are perfectly valid. And this addition also exists in Python 2.
- RussianCow 11y ago
  The second example should be: '{} {}'.format('one', 'two') Either way, the former example still works in Python 3.5, so that syntax hasn't gone away. `format` is preferred, though. This Stack Overflow question has some good answers as to why: http://stackoverflow.com/questions/5082452/python-string-formatting-vs-format http://stackoverflow.com/questions/5082452/python-string-for...
- dietrichepp 11y ago
  You mean, '{} {}'.format('one', 'two') Your experience may vary, but in my experience, when I switched to .format(), I found a number of bugs in code that used % instead. As mentioned, you can continue using %.
  - teach 11y ago
    I had the same experience. I love love love printf() and it's ilk, so switching to something else (no matter how well designed) seemed asinine at first. But .format() has really started to grow on me and did uncover some subtle bugs in old code.
- gluelogic 11y ago
  The thing about the latter form is that you can do something like '{0} {1} {0} {2}'.format('apple', 'banana', 'orange') which results in 'apple banana apple orange'
  - squeaky-clean 11y ago
    You can also use keyword arguments, which is really helpful in certain situation. Example from the docs: 'Coordinates: {latitude}, {longitude}'.format(latitude='37.24N', longitude='-115.81W')
    - wyldfire 11y ago
      You could do that before with % substitution too. But I prefer .format() because (1) it's the new idiom and (2) will coerce to string without me binding the type in the format string.
- mixmastamyk 11y ago
  Simplified to this in Py 3.6: f'{one} {two}'
  - teddyh 11y ago
    Do you have a reference for that? Wouldn’t that be introducing a huge security hole in all programs?
    - mixmastamyk 11y ago
      PEP 498. Literals only, does not add any additional security problems.
      - teddyh 11y ago
        Ah, right; I missed the ‘f’ prefix. And since it’s only done when parsing the expression, it is not a security problem. Thanks!
  - kazagistar 11y ago
    Is it actually happening? Proper string interpolation? In all its extendable glory?
    - mixmastamyk 11y ago
      Pep was accepted, and I believe can be tried in a nightly or alpha release. It is not extensible though, GvR didn't see much utility, yet at least.
- frewsxcv 11y ago
  Let's not also forget string.Template ;) https://docs.python.org/2/library/string.html#string.Template https://docs.python.org/2/library/string.html#string.Templat...
  - sevensor 11y ago
    I actually found a good use for string.Template, but that doesn't mean I understand why it exists in addition to the other two formatting sub-languages.
    - mixmastamyk 11y ago
      It was created long before .format and is better in some cases for i18n where simplicity is best.
- andreasvc 11y ago
  No need to get snarky, because the % formatting is still in Python 3. It is also my preferred formatting method.
- RodericDay 11y ago
  The super, super controversial automatic interpolation feature is what I'm really looking forward to.
RussianCow 11y ago
It's lack of backward compatibility, and not enough of an upside for most developers to go through the effort to upgrade their existing code. It's especially tedious for open source projects, because making a codebase compatible with both 2 and 3 can be a lot of work.
andrewstuart 11y ago
>> why has Python 3 adoption been so slow Really this topic is coming to an end. Most libraries support Python 3 and if they don't there's better alternatives. For new users and new projects there's no reason now except personal preference to choose Python 2 and in fact beginners who start with Python 2 are just instantly incurring a learning debt upon themselves to be paid down the track when they have to move to Python 3. The community has some extremely vocal Python2 diehards but their arguments no longer hold water. >>why has Python 3 adoption been so slow? Whatever, it's just history now.
- mwcampbell 11y ago
  What about MySQLdb? That's the main extension module that has been keeping me on Python 2 for server-side software. That, plus a lack of perceived upside to CPython 3, which is still an interpreter with a GIL after all. PyPy 3 might be compelling if it can run MySQLdb or a bug-compatible replacement. True, PyPy still has a GIL, but at least it also has a JIT compiler.
  - andrewstuart 11y ago
    https://dev.mysql.com/downloads/connector/python/2.1.html https://dev.mysql.com/downloads/connector/python/2.1.html http://dev.mysql.com/doc/connector-python/en/connector-python-versions.html http://dev.mysql.com/doc/connector-python/en/connector-pytho...
  - frewsxcv 11y ago
    MySQLdb has other issues besides Python 3 support. It's no longer maintained and hasn't been updated in quite some time. The Django documentation (and I) now recommends mysqlclient-python, which is compatible with systems that rely on MySQLdb https://github.com/PyMySQL/mysqlclient-python https://github.com/PyMySQL/mysqlclient-python
    - kstrauser 11y ago
      Another endorsement from me. We migrated with import pymysql as MySQLdb and everything just worked.
rkrzr 11y ago
The main reason for most bigger projects is that they rely on that one library that is not compatible with Python3. Although there are fewer and fewer of those fortunately. Another reason is that the advantage of switching is just not that big, if you already have everything working in Python2.
aidos 11y ago
It feels like there are a whole bunch of factors (though I'm no expert). It took a few versions of 3 to hit a sweet spot (in some cases features that were removed in the initial version of 3 have slowly been re-added in subsequent versions). There were a lot of crucial libs that needed to be ported. Just general inertia.
Scarbutt 11y ago
one of many reasons could also be devs wanting a faster implementation or a better multi-core story, so Python3 has probably lose ground to languages like Go, Clojure, Javascript.
e1ven 11y ago
I can only speak for my personal experience - I write all my new code in Py3, and almost every still-developed library works great with Python 3.. The authors usually ported it several years ago. But sometimes you'll need something that's not still supported - Perhaps you want to parse an old obscure file format, and the only code you find for it is from a usenet post in 1996. That code isn't being updated, and no one has ported it. That means you need to do the work to update it, and that can be hard when you aren't familiar with what it's supposed to do. The other place I've seen people sticking with Py2 is when they've got a huge chunk of internal code. Some companies have been writing python for 20 years, and the original authors have long since left the company. It can be hard to write a business case for having someone spend several weeks updating all the old code, particularly if it's purely internal, and doesn't touch the internet.
- dcosson 11y ago
  I think you're exaggerating the obscurity of libraries that only run on python 2. The list I've gone off of is https://python3wos.appspot.com/ https://python3wos.appspot.com/. I use python a lot, and only now in late 2015 I might finally use python3 if starting a new project. When I started a new project last year, we used python2. The biggest hold-out for me was gevent, which was only released 5 months ago. Gunicorn with gevent workers is my preferred stack for running python apps. If you use protobufs or thrift, those both aren't yet on python3. The wall of shame currently lists requests as not working on python3, though I think that might be a fluke. These ones might not be a deal-breaker since you can have a separate environment for infrastructure & app code, but for some reason a lot of the infrastructure tools still haven't updated to python 3 (supervisor, ansible, fabric, graphite). All together, it adds up to a not-insignificant number of things that aren't yet on python3. And even if nothing you use when you first start a project is python2 only, you have no idea what libraries you might need or want in the future and if those might be python2 only.
  - zardeh 11y ago
    protobuff supports 3.0 (though apparently the pip3 version still requires you to manually run 2to3, but then it works) and there's already a port of fabric (ish) by the original author to py3. And Ansible should be python2 until distros start shipping py3 by default, (which they have so I assume Ansible3 will come out soonTM). If you're willing to do a bit more than "pip install x", well I guess it doesn't work, back to py2, you can use almost everything on py3. (and yes requests is ported too)
rgacote 11y ago
I think one thing that stalled adoption was the difficulty of migrating to Unicode string constants. In Python 2, you could make a unicode string as u"Entré", but in Python 3, the 'u' was not permitted. Allowing the superfluous u" notation in Python 3 was a big aid in writing 2 and 3 compatible software. Don't remember when that was introduced, Python 3.2? Within the last six months we've moved to writing all new code in Python 3 and migrating a fair bit of legacy code as well. Been fairly smooth on Linux -- a bit rockier on Windows.
- gsnedders 11y ago
  > I think one thing that stalled adoption was the difficulty of migrating to Unicode string constants. This wasn't just down to the fact that Python 3 didn't support u"" (it was 3.3 that was added, for reference), but also down to the fact that much of the eco-system still supported RHEL5's default of Python 2.4 which meant `from __future__ import unicode_literals` wasn't an option (it is, almost certainly, a less good option, but it's in many ways good enough).
Spiritus 11y ago
I think the reason is that it's basically only now that Linux distros are starting to ship Python 3 as the default. When RHEL, CentOS and Debian moves completely to Python 3, the rest of the world will follow.
- tshtf 11y ago
  RHEL/CentOS 7 was released in June of 2014 with Python 2.7. Following their glacial release schedule, maybe we'll see Python 3 by 2019 in RHEL 8.
  - jessaustin 11y ago
    Those who use value a more rapid package update schedule shouldn't use those distros. One of their most salient features is the ancient software packages.
- detaro 11y ago
  Is the shipped default really so important, esp. for third-party software? E.g. for RHEL you get python3 packages through Red Hat Software Collections, with support and "intended for production use". (It of course limits software that is to be shipped with the distro itself)
groner 11y ago
PEP3003 was a moratorium on language changes in order to allow alternate implementations time to catch up. This meant that Python 3.1 and Python 3.2 didn't include any new language features. Releases since the moratorium ended have been much more compelling, IMO.
jstimpfle 11y ago
I'm coding exclusively in python3, and I agree the "least small" changes from python2 made it cleaner BUT > I'm also a fan of map and filter returning generators rather than being hard-coded to a list implementation I find myself often having to wrap expressions in list(...) to force the lists. (which is annoying) Generators make things much more complicated. They are basically a way to make (interacting, by means of side-effects) coroutines, which are difficult to control. In most use cases (scripting) lists are much easier to use (no interleaving of side effects) and there is plenty of memory available to force them. Generators also go against the "explicit is better than implicit" mantra. It's hard to tell them apart from lists. And often it's just not clear if the code at hand works for lists, or generators, or both. So IMHO generators by default is a bad choice. > stream fusion is a good thing I don't think generators qualify for "stream fusion". I think stream fusion is a notion from compiled languages like Haskell where multiple individual per-item actions can be compiled and optimized to a combined action. Python instead, I guess, just interleaves actions, which might even be less efficient for complicated actions.
- pdonis 11y ago
  > I find myself often having to wrap expressions in list(...) to force the lists. Out of curiosity, why do you need to force the lists? > Generators make things much more complicated. They are basically a way to make (interacting, by means of side-effects) coroutines Huh? Generators are a way to not make expensive computations until you have to--as well as to not use memory that you don't need. Basically, if all you're doing with a collection of items is iterating over it (which covers a lot of use cases--but perhaps not yours), you should use a generator, not a list--your code will run faster and use less memory. > In most use cases (scripting) lists are much easier to use (no interleaving of side effects) and there is plenty of memory available to force them. Generators don't have to have side effects. And there are plenty of use cases for which you do not have "plenty of memory available" (again, perhaps not yours). > IMHO generators by default is a bad choice. I think lists by default was a bad choice, because it forces everyone to incur the memory and performance overhead of constructing a list whether they need to or not. The default should be the leaner of the two alternatives; people who need or prefer the extra overhead can then get it by using list() (or a list comprehension instead of a generator expression, which is just a matter of typing brackets instead of parentheses).
davvid 11y ago
why has Python 3 adoption been so slow? I can tell you about our situation. We are an animation studio with decades of legacy Python 2 code. We sponser pycon and are one of the poster children for python. We have absolutely no plans to switch to Python3. Here are the various reasons: - Performance is a big deal, and moving to a version of python that is slower is a no-go off the bat. - Python3 has no compelling features that matter to us. The GIL was the one thing that should have been tackled in Python3. - Since the GIL is here to stay, our long-term plan will likely involve removing more python from the pipeline rather than putting a huge effort into a python3 port. - We have dependencies on 3rd-party applications (Houdini, Maya, Nuke) that do not support Python3 - We have no desire to port code "just because". Each production has the choice of either spending effort on Real Features that get pixels on the screen, or on porting code for No Observable Benefit. Real Features always win. - Python3 has a Windows-centric "everything is unicode" view of the world that we do not care about. In our use case, the original behavior where "everything is a byte" is closer to UNIX. A lot of the motivation behind Python3 was to fix its Windows implementation. We are a Linux house, and we do not care about Windows. - Armin's discussion about unicode in Python3 hits many of the points spot on. Why has adoption been so slow? Simply because we have no desire to adopt the new version whatsoever. We'll be using Python2.7 for _at least_ the next 5 years, if not more. It's far more likely that we'll adopt Lua as a scripting language before adopting Python3.
- Keyframe 11y ago
  Same industry, smaller scope here. - We have dependencies on 3rd-party applications (Houdini, Maya, Nuke) that do not support Python3 This is our reason. It's a bit of a conundrum meets catch-22 situation. On one hand we would start using python3 if there would be a support for it, on the other hand no one wants to because of porting legacy code seems bothersome if everything works as it should. To be honest though, all of our python code is to augment those 3rd party applications. Everything we have that's not tied to those applications is C(99 more or less).
- ubernostrum 11y ago
  It's far more likely that we'll adopt Lua as a scripting language before adopting Python3. And you're welcome to do that. But "we want a language frozen in time forever so we never have to maintain code" -- which seems to be what you're aiming for -- is not a goal you can achieve short of developing your own in-house language and never letting it make contact with the public (since as soon as it goes public it will change). Meanwhile, the libraries are moving on and sooner or later they'll either move to Python 3, or be replaced by equivalent libraries with active maintenance, and the distros are winding down their support for Python 2. Switching to something else, and probably just rolling an in-house language you can control forever, is likely your only option if this is your genuine technical position.
overgard 11y ago
I think it's because 3.0 and 3.1 were basically "we broke everything, it's slower, and there are no new compelling features." The versions after that have been fine, but I think the first two versions were such clunkers that it created a bad schism, and now that there's a schism nobody wants to support two languages, so people just stick with 2.7. I think if they had just put some new feature that people had to upgrade for in 3.0, they would have avoided this mess, and people would have grumbled for a few months and adapted. I don't think the lesson is "never break compatibility". The lesson is "don't compete with yourself by releasing a product that is actively worse than your current version"

harryf 11y ago

What makes me snarky is the replacing of '%s %s' % ('one', 'two') With '%s %s'.format('one', 'two') The latter is just more annoying to type. Stupid argument I know but I find myself grumbling to myself every time...

davidism 11y ago
It's not replaced, both are perfectly valid. And this addition also exists in Python 2.
RussianCow 11y ago
The second example should be: '{} {}'.format('one', 'two') Either way, the former example still works in Python 3.5, so that syntax hasn't gone away. `format` is preferred, though. This Stack Overflow question has some good answers as to why: http://stackoverflow.com/questions/5082452/python-string-formatting-vs-format http://stackoverflow.com/questions/5082452/python-string-for...
dietrichepp 11y ago
You mean, '{} {}'.format('one', 'two') Your experience may vary, but in my experience, when I switched to .format(), I found a number of bugs in code that used % instead. As mentioned, you can continue using %.
- teach 11y ago
  I had the same experience. I love love love printf() and it's ilk, so switching to something else (no matter how well designed) seemed asinine at first. But .format() has really started to grow on me and did uncover some subtle bugs in old code.
gluelogic 11y ago
The thing about the latter form is that you can do something like '{0} {1} {0} {2}'.format('apple', 'banana', 'orange') which results in 'apple banana apple orange'
- squeaky-clean 11y ago
  You can also use keyword arguments, which is really helpful in certain situation. Example from the docs: 'Coordinates: {latitude}, {longitude}'.format(latitude='37.24N', longitude='-115.81W')
  - wyldfire 11y ago
    You could do that before with % substitution too. But I prefer .format() because (1) it's the new idiom and (2) will coerce to string without me binding the type in the format string.
mixmastamyk 11y ago
Simplified to this in Py 3.6: f'{one} {two}'
- teddyh 11y ago
  Do you have a reference for that? Wouldn’t that be introducing a huge security hole in all programs?
  - mixmastamyk 11y ago
    PEP 498. Literals only, does not add any additional security problems.
    - teddyh 11y ago
      Ah, right; I missed the ‘f’ prefix. And since it’s only done when parsing the expression, it is not a security problem. Thanks!
- kazagistar 11y ago
  Is it actually happening? Proper string interpolation? In all its extendable glory?
  - mixmastamyk 11y ago
    Pep was accepted, and I believe can be tried in a nightly or alpha release. It is not extensible though, GvR didn't see much utility, yet at least.
frewsxcv 11y ago
Let's not also forget string.Template ;) https://docs.python.org/2/library/string.html#string.Template https://docs.python.org/2/library/string.html#string.Templat...
- sevensor 11y ago
  I actually found a good use for string.Template, but that doesn't mean I understand why it exists in addition to the other two formatting sub-languages.
  - mixmastamyk 11y ago
    It was created long before .format and is better in some cases for i18n where simplicity is best.
andreasvc 11y ago
No need to get snarky, because the % formatting is still in Python 3. It is also my preferred formatting method.
RodericDay 11y ago
The super, super controversial automatic interpolation feature is what I'm really looking forward to.

RussianCow 11y ago

It's lack of backward compatibility, and not enough of an upside for most developers to go through the effort to upgrade their existing code. It's especially tedious for open source projects, because making a codebase compatible with both 2 and 3 can be a lot of work.

andrewstuart 11y ago

>> why has Python 3 adoption been so slow Really this topic is coming to an end. Most libraries support Python 3 and if they don't there's better alternatives. For new users and new projects there's no reason now except personal preference to choose Python 2 and in fact beginners who start with Python 2 are just instantly incurring a learning debt upon themselves to be paid down the track when they have to move to Python 3. The community has some extremely vocal Python2 diehards but their arguments no longer hold water. >>why has Python 3 adoption been so slow? Whatever, it's just history now.

mwcampbell 11y ago
What about MySQLdb? That's the main extension module that has been keeping me on Python 2 for server-side software. That, plus a lack of perceived upside to CPython 3, which is still an interpreter with a GIL after all. PyPy 3 might be compelling if it can run MySQLdb or a bug-compatible replacement. True, PyPy still has a GIL, but at least it also has a JIT compiler.
- andrewstuart 11y ago
  https://dev.mysql.com/downloads/connector/python/2.1.html https://dev.mysql.com/downloads/connector/python/2.1.html http://dev.mysql.com/doc/connector-python/en/connector-python-versions.html http://dev.mysql.com/doc/connector-python/en/connector-pytho...
- frewsxcv 11y ago
  MySQLdb has other issues besides Python 3 support. It's no longer maintained and hasn't been updated in quite some time. The Django documentation (and I) now recommends mysqlclient-python, which is compatible with systems that rely on MySQLdb https://github.com/PyMySQL/mysqlclient-python https://github.com/PyMySQL/mysqlclient-python
  - kstrauser 11y ago
    Another endorsement from me. We migrated with import pymysql as MySQLdb and everything just worked.

rkrzr 11y ago

The main reason for most bigger projects is that they rely on that one library that is not compatible with Python3. Although there are fewer and fewer of those fortunately. Another reason is that the advantage of switching is just not that big, if you already have everything working in Python2.

aidos 11y ago

It feels like there are a whole bunch of factors (though I'm no expert). It took a few versions of 3 to hit a sweet spot (in some cases features that were removed in the initial version of 3 have slowly been re-added in subsequent versions). There were a lot of crucial libs that needed to be ported. Just general inertia.

Scarbutt 11y ago

one of many reasons could also be devs wanting a faster implementation or a better multi-core story, so Python3 has probably lose ground to languages like Go, Clojure, Javascript.

e1ven 11y ago

I can only speak for my personal experience - I write all my new code in Py3, and almost every still-developed library works great with Python 3.. The authors usually ported it several years ago. But sometimes you'll need something that's not still supported - Perhaps you want to parse an old obscure file format, and the only code you find for it is from a usenet post in 1996. That code isn't being updated, and no one has ported it. That means you need to do the work to update it, and that can be hard when you aren't familiar with what it's supposed to do. The other place I've seen people sticking with Py2 is when they've got a huge chunk of internal code. Some companies have been writing python for 20 years, and the original authors have long since left the company. It can be hard to write a business case for having someone spend several weeks updating all the old code, particularly if it's purely internal, and doesn't touch the internet.

dcosson 11y ago
I think you're exaggerating the obscurity of libraries that only run on python 2. The list I've gone off of is https://python3wos.appspot.com/ https://python3wos.appspot.com/. I use python a lot, and only now in late 2015 I might finally use python3 if starting a new project. When I started a new project last year, we used python2. The biggest hold-out for me was gevent, which was only released 5 months ago. Gunicorn with gevent workers is my preferred stack for running python apps. If you use protobufs or thrift, those both aren't yet on python3. The wall of shame currently lists requests as not working on python3, though I think that might be a fluke. These ones might not be a deal-breaker since you can have a separate environment for infrastructure & app code, but for some reason a lot of the infrastructure tools still haven't updated to python 3 (supervisor, ansible, fabric, graphite). All together, it adds up to a not-insignificant number of things that aren't yet on python3. And even if nothing you use when you first start a project is python2 only, you have no idea what libraries you might need or want in the future and if those might be python2 only.
- zardeh 11y ago
  protobuff supports 3.0 (though apparently the pip3 version still requires you to manually run 2to3, but then it works) and there's already a port of fabric (ish) by the original author to py3. And Ansible should be python2 until distros start shipping py3 by default, (which they have so I assume Ansible3 will come out soonTM). If you're willing to do a bit more than "pip install x", well I guess it doesn't work, back to py2, you can use almost everything on py3. (and yes requests is ported too)

rgacote 11y ago

I think one thing that stalled adoption was the difficulty of migrating to Unicode string constants. In Python 2, you could make a unicode string as u"Entré", but in Python 3, the 'u' was not permitted. Allowing the superfluous u" notation in Python 3 was a big aid in writing 2 and 3 compatible software. Don't remember when that was introduced, Python 3.2? Within the last six months we've moved to writing all new code in Python 3 and migrating a fair bit of legacy code as well. Been fairly smooth on Linux -- a bit rockier on Windows.

gsnedders 11y ago
> I think one thing that stalled adoption was the difficulty of migrating to Unicode string constants. This wasn't just down to the fact that Python 3 didn't support u"" (it was 3.3 that was added, for reference), but also down to the fact that much of the eco-system still supported RHEL5's default of Python 2.4 which meant `from __future__ import unicode_literals` wasn't an option (it is, almost certainly, a less good option, but it's in many ways good enough).

Spiritus 11y ago

I think the reason is that it's basically only now that Linux distros are starting to ship Python 3 as the default. When RHEL, CentOS and Debian moves completely to Python 3, the rest of the world will follow.

tshtf 11y ago
RHEL/CentOS 7 was released in June of 2014 with Python 2.7. Following their glacial release schedule, maybe we'll see Python 3 by 2019 in RHEL 8.
- jessaustin 11y ago
  Those who use value a more rapid package update schedule shouldn't use those distros. One of their most salient features is the ancient software packages.
detaro 11y ago
Is the shipped default really so important, esp. for third-party software? E.g. for RHEL you get python3 packages through Red Hat Software Collections, with support and "intended for production use". (It of course limits software that is to be shipped with the distro itself)

groner 11y ago

PEP3003 was a moratorium on language changes in order to allow alternate implementations time to catch up. This meant that Python 3.1 and Python 3.2 didn't include any new language features. Releases since the moratorium ended have been much more compelling, IMO.

jstimpfle 11y ago

I'm coding exclusively in python3, and I agree the "least small" changes from python2 made it cleaner BUT > I'm also a fan of map and filter returning generators rather than being hard-coded to a list implementation I find myself often having to wrap expressions in list(...) to force the lists. (which is annoying) Generators make things much more complicated. They are basically a way to make (interacting, by means of side-effects) coroutines, which are difficult to control. In most use cases (scripting) lists are much easier to use (no interleaving of side effects) and there is plenty of memory available to force them. Generators also go against the "explicit is better than implicit" mantra. It's hard to tell them apart from lists. And often it's just not clear if the code at hand works for lists, or generators, or both. So IMHO generators by default is a bad choice. > stream fusion is a good thing I don't think generators qualify for "stream fusion". I think stream fusion is a notion from compiled languages like Haskell where multiple individual per-item actions can be compiled and optimized to a combined action. Python instead, I guess, just interleaves actions, which might even be less efficient for complicated actions.

pdonis 11y ago
> I find myself often having to wrap expressions in list(...) to force the lists. Out of curiosity, why do you need to force the lists? > Generators make things much more complicated. They are basically a way to make (interacting, by means of side-effects) coroutines Huh? Generators are a way to not make expensive computations until you have to--as well as to not use memory that you don't need. Basically, if all you're doing with a collection of items is iterating over it (which covers a lot of use cases--but perhaps not yours), you should use a generator, not a list--your code will run faster and use less memory. > In most use cases (scripting) lists are much easier to use (no interleaving of side effects) and there is plenty of memory available to force them. Generators don't have to have side effects. And there are plenty of use cases for which you do not have "plenty of memory available" (again, perhaps not yours). > IMHO generators by default is a bad choice. I think lists by default was a bad choice, because it forces everyone to incur the memory and performance overhead of constructing a list whether they need to or not. The default should be the leaner of the two alternatives; people who need or prefer the extra overhead can then get it by using list() (or a list comprehension instead of a generator expression, which is just a matter of typing brackets instead of parentheses).

davvid 11y ago

why has Python 3 adoption been so slow? I can tell you about our situation. We are an animation studio with decades of legacy Python 2 code. We sponser pycon and are one of the poster children for python. We have absolutely no plans to switch to Python3. Here are the various reasons: - Performance is a big deal, and moving to a version of python that is slower is a no-go off the bat. - Python3 has no compelling features that matter to us. The GIL was the one thing that should have been tackled in Python3. - Since the GIL is here to stay, our long-term plan will likely involve removing more python from the pipeline rather than putting a huge effort into a python3 port. - We have dependencies on 3rd-party applications (Houdini, Maya, Nuke) that do not support Python3 - We have no desire to port code "just because". Each production has the choice of either spending effort on Real Features that get pixels on the screen, or on porting code for No Observable Benefit. Real Features always win. - Python3 has a Windows-centric "everything is unicode" view of the world that we do not care about. In our use case, the original behavior where "everything is a byte" is closer to UNIX. A lot of the motivation behind Python3 was to fix its Windows implementation. We are a Linux house, and we do not care about Windows. - Armin's discussion about unicode in Python3 hits many of the points spot on. Why has adoption been so slow? Simply because we have no desire to adopt the new version whatsoever. We'll be using Python2.7 for _at least_ the next 5 years, if not more. It's far more likely that we'll adopt Lua as a scripting language before adopting Python3.

Keyframe 11y ago
Same industry, smaller scope here. - We have dependencies on 3rd-party applications (Houdini, Maya, Nuke) that do not support Python3 This is our reason. It's a bit of a conundrum meets catch-22 situation. On one hand we would start using python3 if there would be a support for it, on the other hand no one wants to because of porting legacy code seems bothersome if everything works as it should. To be honest though, all of our python code is to augment those 3rd party applications. Everything we have that's not tied to those applications is C(99 more or less).
ubernostrum 11y ago
It's far more likely that we'll adopt Lua as a scripting language before adopting Python3. And you're welcome to do that. But "we want a language frozen in time forever so we never have to maintain code" -- which seems to be what you're aiming for -- is not a goal you can achieve short of developing your own in-house language and never letting it make contact with the public (since as soon as it goes public it will change). Meanwhile, the libraries are moving on and sooner or later they'll either move to Python 3, or be replaced by equivalent libraries with active maintenance, and the distros are winding down their support for Python 2. Switching to something else, and probably just rolling an in-house language you can control forever, is likely your only option if this is your genuine technical position.

overgard 11y ago

I think it's because 3.0 and 3.1 were basically "we broke everything, it's slower, and there are no new compelling features." The versions after that have been fine, but I think the first two versions were such clunkers that it created a bad schism, and now that there's a schism nobody wants to support two languages, so people just stick with 2.7. I think if they had just put some new feature that people had to upgrade for in 3.0, they would have avoided this mess, and people would have grumbled for a few months and adapted. I don't think the lesson is "never break compatibility". The lesson is "don't compete with yourself by releasing a product that is actively worse than your current version"