4 ms·

And how many subtle bugs will be introduced during porting software for Python 2 to Python 3?

by anewhnaccount 11y ago

agentultra 11y ago
The traps and tricks are well known. With due diligence -- few if any.
- lqdc13 11y ago
  The traps are not always well known (to me). For example, I just discovered a few days ago while porting a Python2 library that byte concatenation is an O(N^2) procedure in Python3 while it is an O(N) procedure in Python2. So you have to use bytearrays and really rewrite all the code that deals with bytes. Basically, even if something works, you still have to performance profile.
  - mark-r 11y ago
    Do you have a reference that explains this? I'd like to learn more.
    - lqdc13 11y ago
      I don't have a reference, but bytearray concatenation with some more bytes is amortized O(1) w.r.t. original bytearray size, but for bytes it is O(n) where n is the length of the byte string, because bytes are immutable and they are copied to a new byte string on each concatenation. In python2 strings/bytes are also immutable but concatenation is O(1) w.r.t. original string. Not sure about internal implementation differences. so basically whenever you see code like: some_bstr = b'' for something in get_something(): some_bstr += something And if you have to do it this way, make sure that some_bstr is a bytearray or make it a list that you later join.
m0th87 11y ago
We ported a major production site from python 2 to 3 a year ago. Exactly one bug has come up - something to do with unicode encoding on a relatively minor part of the site. Thanks to 2to3, the concerns for bugs coming up in porting are way overblown.
- brbsix 11y ago
  The only insidious issues I've encountered with porting apps have been related to Unicode as well. A lot of old Python 2 code is (what appears to me) very hackish compared to the elegance that is Python 3.
  - lqdc13 11y ago
    I think it has more to do with that people using Python3 want to write good looking code, and the people still using Python2 are more interested in getting something running quickly and therefore tend to produce hackish code.
  - qfwfq_ 11y ago
    I've seen quite a bit when porting a statistical library that used Python 2's default floor/int division behavior for quite a few operations. Sometimes, though, the 2 version had bugs because of this behavior. So, the bugs were both subtle and bi-directional. I definitely fault Python 2 for allowing that practice, and am glad we're now using Python 3.

And how many subtle bugs will be introduced during porting software for Python 2 to Python 3?

agentultra 11y ago

The traps and tricks are well known. With due diligence -- few if any.

lqdc13 11y ago
The traps are not always well known (to me). For example, I just discovered a few days ago while porting a Python2 library that byte concatenation is an O(N^2) procedure in Python3 while it is an O(N) procedure in Python2. So you have to use bytearrays and really rewrite all the code that deals with bytes. Basically, even if something works, you still have to performance profile.
- mark-r 11y ago
  Do you have a reference that explains this? I'd like to learn more.
  - lqdc13 11y ago
    I don't have a reference, but bytearray concatenation with some more bytes is amortized O(1) w.r.t. original bytearray size, but for bytes it is O(n) where n is the length of the byte string, because bytes are immutable and they are copied to a new byte string on each concatenation. In python2 strings/bytes are also immutable but concatenation is O(1) w.r.t. original string. Not sure about internal implementation differences. so basically whenever you see code like: some_bstr = b'' for something in get_something(): some_bstr += something And if you have to do it this way, make sure that some_bstr is a bytearray or make it a list that you later join.

m0th87 11y ago

We ported a major production site from python 2 to 3 a year ago. Exactly one bug has come up - something to do with unicode encoding on a relatively minor part of the site. Thanks to 2to3, the concerns for bugs coming up in porting are way overblown.

brbsix 11y ago
The only insidious issues I've encountered with porting apps have been related to Unicode as well. A lot of old Python 2 code is (what appears to me) very hackish compared to the elegance that is Python 3.
- lqdc13 11y ago
  I think it has more to do with that people using Python3 want to write good looking code, and the people still using Python2 are more interested in getting something running quickly and therefore tend to produce hackish code.
- qfwfq_ 11y ago
  I've seen quite a bit when porting a statistical library that used Python 2's default floor/int division behavior for quite a few operations. Sometimes, though, the 2 version had bugs because of this behavior. So, the bugs were both subtle and bi-directional. I definitely fault Python 2 for allowing that practice, and am glad we're now using Python 3.