4 ms·

Letting "Python 2" zombie around is unacceptable. Python 3 is better in every way, and has been since 3.3 (which Armin deserves a lot of credit for). Consider

Letting "Python 2" zombie around is unacceptable. Python 3 is better in every way, and has been since 3.3 (which Armin deserves a lot of credit for). Consider anyone who wants to build something with Python, whether it's a library, application, or service. What's better, having to build for Python 3 and 2, or just Python 3? Thank God that Guido did this, despite knowing all the blowback he'd get. To me, that's super cool.

eesmith 7y ago
"better in every way" ... except for 1) startup time (according to the linked-to article), 2) support for existing Python 2 code, and 3) support for Python 2 C extensions. For example, https://blog.khinsen.net/posts/2017/11/16/a-plea-for-stability-in-the-scipy-ecosystem/ https://blog.khinsen.net/posts/2017/11/16/a-plea-for-stabili... describes the "Molecular Modelling Toolkit (MMTK), which might well be the oldest domain-specific library of the SciPy ecosystem, will probably go away after 2020. Porting it to Python 3 is possible, of course, but an enormous effort (some details are in this Twitter thread[1]) for which resources (funding plus competent staff) are very difficult to find." [1] The thread at https://twitter.com/khinsen/status/930749714567434240 https://twitter.com/khinsen/status/930749714567434240 includes "Lots of C modules written for Python 1.4 are waiting for enthusiastic code archeologists ;-)". I don't think Hinsen is alone in that situation. I can well believe there are some people who, for example, plan to retire in about 5 years and would rather keep with with a Python 2 zombie than spend time to port working code to Python 3.
- camgunz 7y ago
  Startup time is complex, but base startup only increased about 20ms, and that's being generous. I'll admit Python 3 is still slower at a lot of things. But that feels like saying your new dog is even worse at math than your old one. The C extension thing isn't Python's fault. It's the job of library and app authors to update. Do we complain that Vulkan has bad SunOS support? This is totally backwards. Could Hinsen (and others) not just version their deps? It's not like people are erasing Python 2 off the internet. If his main worry is reproducibility, he should be doing that anyway. --- I don't want to give the impression I like the whole Python 3 thing. I think it was a pretty big mistake and a huge missed opportunity. I'm very sympathetic to people who had to put in a lot of work for basically no good reason--Python 3 didn't really offer anything significantly better than 2 until... 3.5 (3.4 if you think the first pass at async was useful, I personally don't). But I also find the ballyhooing about it really insufferable. Yeah it was a mistake; Armin Ronacher (as usual) was right. It was also over 11 years ago. Time to forget all about this and build cool stuff, please please please.
  - tick_tock_tick 7y ago
    It takes me 37ms to load hackernews a 20ms start time is embarrassing what is it even doing?
    - eesmith 7y ago
      It does a lot of module imports. Most of those probe the filesystem. Try "python -vv -c 'pass'" - I'm only showing the first few dozen lines, and I've trimmed some of the paths for conciseness: % python -vv -c 'pass' import _frozen_importlib # frozen import _imp # builtin import '_thread' # <class '_frozen_importlib.BuiltinImporter'> import '_warnings' # <class '_frozen_importlib.BuiltinImporter'> import '_weakref' # <class '_frozen_importlib.BuiltinImporter'> # installing zipimport hook import 'zipimport' # <class '_frozen_importlib.BuiltinImporter'> # installed zipimport hook import '_frozen_importlib_external' # <class '_frozen_importlib.FrozenImporter'> import '_io' # <class '_frozen_importlib.BuiltinImporter'> import 'marshal' # <class '_frozen_importlib.BuiltinImporter'> import 'posix' # <class '_frozen_importlib.BuiltinImporter'> import _thread # previously loaded ('_thread') import '_thread' # <class '_frozen_importlib.BuiltinImporter'> import _weakref # previously loaded ('_weakref') import '_weakref' # <class '_frozen_importlib.BuiltinImporter'> # miniconda3/lib/python3.7/encodings/__pycache__/__init__.cpython-37.pyc matches miniconda3/lib/python3.7/encodings/__init__.py # code object from 'miniconda3/lib/python3.7/encodings/__pycache__/__init__.cpython-37.pyc' # trying miniconda3/lib/python3.7/codecs.cpython-37m-darwin.so # trying miniconda3/lib/python3.7/codecs.abi3.so # trying miniconda3/lib/python3.7/codecs.so # trying miniconda3/lib/python3.7/codecs.py # miniconda3/lib/python3.7/__pycache__/codecs.cpython-37.pyc matches miniconda3/lib/python3.7/codecs.py # code object from 'miniconda3/lib/python3.7/__pycache__/codecs.cpython-37.pyc' import '_codecs' # <class '_frozen_importlib.BuiltinImporter'> import 'codecs' # <_frozen_importlib_external.SourceFileLoader object at 0x10dd44c90> # trying miniconda3/lib/python3.7/encodings/aliases.cpython-37m-darwin.so # trying miniconda3/lib/python3.7/encodings/aliases.abi3.so # trying miniconda3/lib/python3.7/encodings/aliases.so # trying miniconda3/lib/python3.7/encodings/aliases.py # miniconda3/lib/python3.7/encodings/__pycache__/aliases.cpython-37.pyc matches miniconda3/lib/python3.7/encodings/aliases.py # code object from 'miniconda3/lib/python3.7/encodings/__pycache__/aliases.cpython-37.pyc' import 'encodings.aliases' # <_frozen_importlib_external.SourceFileLoader object at 0x10dd67d10> import 'encodings' # <_frozen_importlib_external.SourceFileLoader object at 0x10dd440d0> # trying miniconda3/lib/python3.7/encodings/utf_8.cpython-37m-darwin.so # trying miniconda3/lib/python3.7/encodings/utf_8.abi3.so # trying miniconda3/lib/python3.7/encodings/utf_8.so # trying miniconda3/lib/python3.7/encodings/utf_8.py # miniconda3/lib/python3.7/encodings/__pycache__/utf_8.cpython-37.pyc matches miniconda3/lib/python3.7/encodings/utf_8.py # code object from 'miniconda3/lib/python3.7/encodings/__pycache__/utf_8.cpython-37.pyc' import 'encodings.utf_8' # <_frozen_importlib_external.SourceFileLoader object at 0x10dd44bd0> import '_signal' # <class '_frozen_importlib.BuiltinImporter'> # trying miniconda3/lib/python3.7/encodings/latin_1.cpython-37m-darwin.so # trying miniconda3/lib/python3.7/encodings/latin_1.abi3.so # trying miniconda3/lib/python3.7/encodings/latin_1.so # trying miniconda3/lib/python3.7/encodings/latin_1.py # miniconda3/lib/python3.7/encodings/__pycache__/latin_1.cpython-37.pyc matches miniconda3/lib/python3.7/encodings/latin_1 ... many, many more lines omitted ... This can be sped up a lot using a zipimport of the Python standard library, https://docs.python.org/3/library/zipimport.html?highlight=zipimport#module-zipimport https://docs.python.org/3/library/zipimport.html?highlight=z... , where all of the standard library is put in a zipfile. Then there's a file access to get the zip metadata. One of the things that bugged me in Python2 was that every startup imported UserDict: # trying python2.7/UserDict.so # trying python2.7/UserDictmodule.so # trying python2.7/UserDict.py # python2.7/UserDict.pyc matches python2.7/UserDict.py import UserDict # precompiled from python2.7/UserDict.pyc This is because os.environ was an instance of UserDict: % python2.7 -c 'import os; print(os.environ.__class__.__bases__)' (<class UserDict.IterableUserDict at 0x1029b14c8>,) Under Python3 this is spelled collections.abc.MutableMapping: % python3 -c 'import os; print(os.environ.__class__.__bases__)' (<class 'collections.abc.MutableMapping'>,) which triggers its own set of imports: # trying python3.6/collections/abc.cpython-36m-darwin.so # trying python3.6/collections/abc.abi3.so # trying python3.6/collections/abc.so # trying python3.6/collections/abc.py # python3.6/collections/__pycache__/abc.cpython-36.pyc matches python3.6/collections/abc.py # code object from 'python3.6/collections/__pycache__/abc.cpython-36.pyc' import 'collections.abc' # <_frozen_importlib_external.SourceFileLoader object at 0x103c2ecf8> There's better performance using SDD than a HDD, which is in turn better than using a networked filesystem.

eesmith 7y ago

"better in every way" ... except for 1) startup time (according to the linked-to article), 2) support for existing Python 2 code, and 3) support for Python 2 C extensions. For example, https://blog.khinsen.net/posts/2017/11/16/a-plea-for-stability-in-the-scipy-ecosystem/ https://blog.khinsen.net/posts/2017/11/16/a-plea-for-stabili... describes the "Molecular Modelling Toolkit (MMTK), which might well be the oldest domain-specific library of the SciPy ecosystem, will probably go away after 2020. Porting it to Python 3 is possible, of course, but an enormous effort (some details are in this Twitter thread[1]) for which resources (funding plus competent staff) are very difficult to find." [1] The thread at https://twitter.com/khinsen/status/930749714567434240 https://twitter.com/khinsen/status/930749714567434240 includes "Lots of C modules written for Python 1.4 are waiting for enthusiastic code archeologists ;-)". I don't think Hinsen is alone in that situation. I can well believe there are some people who, for example, plan to retire in about 5 years and would rather keep with with a Python 2 zombie than spend time to port working code to Python 3.

camgunz 7y ago
Startup time is complex, but base startup only increased about 20ms, and that's being generous. I'll admit Python 3 is still slower at a lot of things. But that feels like saying your new dog is even worse at math than your old one. The C extension thing isn't Python's fault. It's the job of library and app authors to update. Do we complain that Vulkan has bad SunOS support? This is totally backwards. Could Hinsen (and others) not just version their deps? It's not like people are erasing Python 2 off the internet. If his main worry is reproducibility, he should be doing that anyway. --- I don't want to give the impression I like the whole Python 3 thing. I think it was a pretty big mistake and a huge missed opportunity. I'm very sympathetic to people who had to put in a lot of work for basically no good reason--Python 3 didn't really offer anything significantly better than 2 until... 3.5 (3.4 if you think the first pass at async was useful, I personally don't). But I also find the ballyhooing about it really insufferable. Yeah it was a mistake; Armin Ronacher (as usual) was right. It was also over 11 years ago. Time to forget all about this and build cool stuff, please please please.
- tick_tock_tick 7y ago
  It takes me 37ms to load hackernews a 20ms start time is embarrassing what is it even doing?
  - eesmith 7y ago
    It does a lot of module imports. Most of those probe the filesystem. Try "python -vv -c 'pass'" - I'm only showing the first few dozen lines, and I've trimmed some of the paths for conciseness: % python -vv -c 'pass' import _frozen_importlib # frozen import _imp # builtin import '_thread' # <class '_frozen_importlib.BuiltinImporter'> import '_warnings' # <class '_frozen_importlib.BuiltinImporter'> import '_weakref' # <class '_frozen_importlib.BuiltinImporter'> # installing zipimport hook import 'zipimport' # <class '_frozen_importlib.BuiltinImporter'> # installed zipimport hook import '_frozen_importlib_external' # <class '_frozen_importlib.FrozenImporter'> import '_io' # <class '_frozen_importlib.BuiltinImporter'> import 'marshal' # <class '_frozen_importlib.BuiltinImporter'> import 'posix' # <class '_frozen_importlib.BuiltinImporter'> import _thread # previously loaded ('_thread') import '_thread' # <class '_frozen_importlib.BuiltinImporter'> import _weakref # previously loaded ('_weakref') import '_weakref' # <class '_frozen_importlib.BuiltinImporter'> # miniconda3/lib/python3.7/encodings/__pycache__/__init__.cpython-37.pyc matches miniconda3/lib/python3.7/encodings/__init__.py # code object from 'miniconda3/lib/python3.7/encodings/__pycache__/__init__.cpython-37.pyc' # trying miniconda3/lib/python3.7/codecs.cpython-37m-darwin.so # trying miniconda3/lib/python3.7/codecs.abi3.so # trying miniconda3/lib/python3.7/codecs.so # trying miniconda3/lib/python3.7/codecs.py # miniconda3/lib/python3.7/__pycache__/codecs.cpython-37.pyc matches miniconda3/lib/python3.7/codecs.py # code object from 'miniconda3/lib/python3.7/__pycache__/codecs.cpython-37.pyc' import '_codecs' # <class '_frozen_importlib.BuiltinImporter'> import 'codecs' # <_frozen_importlib_external.SourceFileLoader object at 0x10dd44c90> # trying miniconda3/lib/python3.7/encodings/aliases.cpython-37m-darwin.so # trying miniconda3/lib/python3.7/encodings/aliases.abi3.so # trying miniconda3/lib/python3.7/encodings/aliases.so # trying miniconda3/lib/python3.7/encodings/aliases.py # miniconda3/lib/python3.7/encodings/__pycache__/aliases.cpython-37.pyc matches miniconda3/lib/python3.7/encodings/aliases.py # code object from 'miniconda3/lib/python3.7/encodings/__pycache__/aliases.cpython-37.pyc' import 'encodings.aliases' # <_frozen_importlib_external.SourceFileLoader object at 0x10dd67d10> import 'encodings' # <_frozen_importlib_external.SourceFileLoader object at 0x10dd440d0> # trying miniconda3/lib/python3.7/encodings/utf_8.cpython-37m-darwin.so # trying miniconda3/lib/python3.7/encodings/utf_8.abi3.so # trying miniconda3/lib/python3.7/encodings/utf_8.so # trying miniconda3/lib/python3.7/encodings/utf_8.py # miniconda3/lib/python3.7/encodings/__pycache__/utf_8.cpython-37.pyc matches miniconda3/lib/python3.7/encodings/utf_8.py # code object from 'miniconda3/lib/python3.7/encodings/__pycache__/utf_8.cpython-37.pyc' import 'encodings.utf_8' # <_frozen_importlib_external.SourceFileLoader object at 0x10dd44bd0> import '_signal' # <class '_frozen_importlib.BuiltinImporter'> # trying miniconda3/lib/python3.7/encodings/latin_1.cpython-37m-darwin.so # trying miniconda3/lib/python3.7/encodings/latin_1.abi3.so # trying miniconda3/lib/python3.7/encodings/latin_1.so # trying miniconda3/lib/python3.7/encodings/latin_1.py # miniconda3/lib/python3.7/encodings/__pycache__/latin_1.cpython-37.pyc matches miniconda3/lib/python3.7/encodings/latin_1 ... many, many more lines omitted ... This can be sped up a lot using a zipimport of the Python standard library, https://docs.python.org/3/library/zipimport.html?highlight=zipimport#module-zipimport https://docs.python.org/3/library/zipimport.html?highlight=z... , where all of the standard library is put in a zipfile. Then there's a file access to get the zip metadata. One of the things that bugged me in Python2 was that every startup imported UserDict: # trying python2.7/UserDict.so # trying python2.7/UserDictmodule.so # trying python2.7/UserDict.py # python2.7/UserDict.pyc matches python2.7/UserDict.py import UserDict # precompiled from python2.7/UserDict.pyc This is because os.environ was an instance of UserDict: % python2.7 -c 'import os; print(os.environ.__class__.__bases__)' (<class UserDict.IterableUserDict at 0x1029b14c8>,) Under Python3 this is spelled collections.abc.MutableMapping: % python3 -c 'import os; print(os.environ.__class__.__bases__)' (<class 'collections.abc.MutableMapping'>,) which triggers its own set of imports: # trying python3.6/collections/abc.cpython-36m-darwin.so # trying python3.6/collections/abc.abi3.so # trying python3.6/collections/abc.so # trying python3.6/collections/abc.py # python3.6/collections/__pycache__/abc.cpython-36.pyc matches python3.6/collections/abc.py # code object from 'python3.6/collections/__pycache__/abc.cpython-36.pyc' import 'collections.abc' # <_frozen_importlib_external.SourceFileLoader object at 0x103c2ecf8> There's better performance using SDD than a HDD, which is in turn better than using a networked filesystem.