9 ms·

When Python 3 broke compatibility, it would be perfect timing to implement tagged pointers, small integers and other performance improvements. At least it would

When Python 3 broke compatibility, it would be perfect timing to implement tagged pointers, small integers and other performance improvements. At least it would be an easier sell to your boss to rewrite your code base. "We are doing so it runs faster" is much better than "we are doing it for intellectual purity" from a business standpoint.

bramblerose 9y ago
Why would "makes your code run 1% faster" be a better sell than "saves you 1% development time by reducing the number of bugs"?
- egwor 9y ago
  That tends to be a more difficult sell because the counter-argument from further up is 'But we're not seeing those bugs right now, and won't we find bugs when we do this upgrade?'
- snovv_crash 9y ago
  It depends on a) the size of your deployment b) whether you are paying SV salaries to your developers c) how sensitive you are to speed. In many things I'd be happy to double the number of bugs if it meant I could run the code twice as fast. Bugs can be fixed. Slow code is still slow.
- netheril96 9y ago
  Because during the upgrade you would have an explosion of bugs and therefore development time fixing them. The 1% saved by purity wouldn't make a dent.
axaxs 9y ago
Agreed. A 'fast' Python, to me, the perfect language. The changes 2 to 3 broke compatibility, but for 'pureness' sake. If speed and the deployment situation were better for python, that would be all I use. I once thought to make a compiler, but don't have the time and honestly knowhow to begin. Nuitka is trying, but also agreeing to live with compatabilty vs performance or type safety. A type safe compiled python is my dream.
- svennek 9y ago
  Python 3 did not break compatibilty for "pureness" sake. They broke it, because for 6 billion people in the world, for whom English is not the only language, there was a very real pain (bordering on stuff that was impossible). Most of the world is moving to unicode exactly because of this (I know Sillicon Vally companies are probably moving to unicode, so that their users can use cute emojis). Most people also need to handle stuff that are in different encodings (I work daily in both Latin1 (north-western European) and UTF8), translating between that was painful/borderline-impossible in the old days (python 2). The sharp distinction between bytes and strings are essential for any non-english/american/aussie person. Sadly python2 got that wrong, and hence had to die.. Which is probably also why Perl6 choose to go the same route.. If you really need a good reason to go to python 3 other than 6 billion extra potential customers, try "oh look, we have emojis" <sarcasm/>
  - detaro 9y ago
    One might argue that making unicode strings the default instead of a special datatype (which they were in 2.x) is also a "pureness" decision (since you can handle unicode as much in 2.7 as you can in 3.x)
  - prevedmedved 9y ago
    I'm one of those people for whom English is not the only language. Python 3 is objectively harder and more confusing to use in a multilingual environment. Python 2 didn't pretend that only Unicode exists and Python 2 was encoding-agnostic. This is the correct default for the real world. Ironically, the effect of Python 3 was the opposite: it made life really hard for those that aren't exclusive anglophones and made life a lot easier for American programmers who now don't have to bother with the whole character encoding mess. Personally, Python 3's broken support for internationalisation made me drop Python altogether. Python 2 was confusing, but the defaults were at least workable.
    - svennek 9y ago
      Really? I find it so much easier (I am also not an English language programmer), because it is very explicit about what is the codepoints (i.e. string) and what is the bytes (i.e. on the disk or the network). The encode/decode functions is basically the only change, but it gives you the reason to explicitly tell it about the encoding you expect.. But as you left python, the point is moot :)
ubernostrum 9y ago
When Python 3 broke compatibility, the Python team did exactly the thing you're supposed to do with code: first they made it correct (i.e., got all the changes implemented and working as expected), then made it fast. "We're doing a huge rewrite already, let's pile on more scope" is a recipe for disaster.
- bonzini 9y ago
  It depends. It is really surprising that Python does not use tagged integers, which is a 50 year old technique used by pretty much any other dynamic language in existence. Using an extension API breakage as the excuse to introduce them would have made sense, since it also breaks the API. Now they cannot happen until Python 4.
  - netheril96 9y ago
    And Guido says he won't break backwards compat again in Python 4 as he has learned the lesson. No tagged integers for life.
    - coldtea 9y ago
      He learned the wrong lesson then. Which wasn't "don't break backwards compatibility", but: 1) If you break it, break it for compelling reasons. 2) If you provide a tool to convert, don't make it as awful as 2to3
    - bonzini 9y ago
      Breaking language backwards-compatibility is one thing; breaking the extension API is less problematic, and breaking the extension ABI is better still.
  - cdavid 9y ago
    I believe ABC, one of python's influence, used tagged pointers, but I can't find a quote on this. Not using tagged pointers was a conscientious decision from Guido (see e.g. https://github.com/dropbox/pyston/issues/181#issuecomment-62941865 https://github.com/dropbox/pyston/issues/181#issuecomment-62...) [edited] Something that people often overlook when talking about optimizing python is that the "simplicity" of the C Python API allowed the rich ecosystem of C extensions. GC, tagged pointers, etc... are great, and often used in modern VMs, but they tend to make integration w/ C code more difficult. Without rich C extensions, no numpy, no scipy, no scikit learn, no tensorflow/theano/etc... I am not sure this is completely incidental. [edit 2] Ah, found the quote I remembered about tagged pointers in ABC from Guido: https://mail.python.org/pipermail/python-dev/2004-July/046147.html https://mail.python.org/pipermail/python-dev/2004-July/04614...
ForHackernews 9y ago
How about "We're doing it so that subtle encoding errors don't shoot us or our customers in the foot."

bramblerose 9y ago

Why would "makes your code run 1% faster" be a better sell than "saves you 1% development time by reducing the number of bugs"?

egwor 9y ago
That tends to be a more difficult sell because the counter-argument from further up is 'But we're not seeing those bugs right now, and won't we find bugs when we do this upgrade?'
snovv_crash 9y ago
It depends on a) the size of your deployment b) whether you are paying SV salaries to your developers c) how sensitive you are to speed. In many things I'd be happy to double the number of bugs if it meant I could run the code twice as fast. Bugs can be fixed. Slow code is still slow.
netheril96 9y ago
Because during the upgrade you would have an explosion of bugs and therefore development time fixing them. The 1% saved by purity wouldn't make a dent.

axaxs 9y ago

Agreed. A 'fast' Python, to me, the perfect language. The changes 2 to 3 broke compatibility, but for 'pureness' sake. If speed and the deployment situation were better for python, that would be all I use. I once thought to make a compiler, but don't have the time and honestly knowhow to begin. Nuitka is trying, but also agreeing to live with compatabilty vs performance or type safety. A type safe compiled python is my dream.

svennek 9y ago
Python 3 did not break compatibilty for "pureness" sake. They broke it, because for 6 billion people in the world, for whom English is not the only language, there was a very real pain (bordering on stuff that was impossible). Most of the world is moving to unicode exactly because of this (I know Sillicon Vally companies are probably moving to unicode, so that their users can use cute emojis). Most people also need to handle stuff that are in different encodings (I work daily in both Latin1 (north-western European) and UTF8), translating between that was painful/borderline-impossible in the old days (python 2). The sharp distinction between bytes and strings are essential for any non-english/american/aussie person. Sadly python2 got that wrong, and hence had to die.. Which is probably also why Perl6 choose to go the same route.. If you really need a good reason to go to python 3 other than 6 billion extra potential customers, try "oh look, we have emojis" <sarcasm/>
- detaro 9y ago
  One might argue that making unicode strings the default instead of a special datatype (which they were in 2.x) is also a "pureness" decision (since you can handle unicode as much in 2.7 as you can in 3.x)
- prevedmedved 9y ago
  I'm one of those people for whom English is not the only language. Python 3 is objectively harder and more confusing to use in a multilingual environment. Python 2 didn't pretend that only Unicode exists and Python 2 was encoding-agnostic. This is the correct default for the real world. Ironically, the effect of Python 3 was the opposite: it made life really hard for those that aren't exclusive anglophones and made life a lot easier for American programmers who now don't have to bother with the whole character encoding mess. Personally, Python 3's broken support for internationalisation made me drop Python altogether. Python 2 was confusing, but the defaults were at least workable.
  - svennek 9y ago
    Really? I find it so much easier (I am also not an English language programmer), because it is very explicit about what is the codepoints (i.e. string) and what is the bytes (i.e. on the disk or the network). The encode/decode functions is basically the only change, but it gives you the reason to explicitly tell it about the encoding you expect.. But as you left python, the point is moot :)

ubernostrum 9y ago

When Python 3 broke compatibility, the Python team did exactly the thing you're supposed to do with code: first they made it correct (i.e., got all the changes implemented and working as expected), then made it fast. "We're doing a huge rewrite already, let's pile on more scope" is a recipe for disaster.

bonzini 9y ago
It depends. It is really surprising that Python does not use tagged integers, which is a 50 year old technique used by pretty much any other dynamic language in existence. Using an extension API breakage as the excuse to introduce them would have made sense, since it also breaks the API. Now they cannot happen until Python 4.
- netheril96 9y ago
  And Guido says he won't break backwards compat again in Python 4 as he has learned the lesson. No tagged integers for life.
  - coldtea 9y ago
    He learned the wrong lesson then. Which wasn't "don't break backwards compatibility", but: 1) If you break it, break it for compelling reasons. 2) If you provide a tool to convert, don't make it as awful as 2to3
  - bonzini 9y ago
    Breaking language backwards-compatibility is one thing; breaking the extension API is less problematic, and breaking the extension ABI is better still.
- cdavid 9y ago
  I believe ABC, one of python's influence, used tagged pointers, but I can't find a quote on this. Not using tagged pointers was a conscientious decision from Guido (see e.g. https://github.com/dropbox/pyston/issues/181#issuecomment-62941865 https://github.com/dropbox/pyston/issues/181#issuecomment-62...) [edited] Something that people often overlook when talking about optimizing python is that the "simplicity" of the C Python API allowed the rich ecosystem of C extensions. GC, tagged pointers, etc... are great, and often used in modern VMs, but they tend to make integration w/ C code more difficult. Without rich C extensions, no numpy, no scipy, no scikit learn, no tensorflow/theano/etc... I am not sure this is completely incidental. [edit 2] Ah, found the quote I remembered about tagged pointers in ABC from Guido: https://mail.python.org/pipermail/python-dev/2004-July/046147.html https://mail.python.org/pipermail/python-dev/2004-July/04614...

ForHackernews 9y ago

How about "We're doing it so that subtle encoding errors don't shoot us or our customers in the foot."