3 ms·
The traps are not always well known (to me). For example, I just discovered a few days ago while porting a Python2 library that byte concatenation is an O(N^2)
by lqdc13 11y ago
The traps are not always well known (to me).
For example, I just discovered a few days ago while porting a Python2 library that byte concatenation is an O(N^2) procedure in Python3 while it is an O(N) procedure in Python2.
So you have to use bytearrays and really rewrite all the code that deals with bytes. Basically, even if something works, you still have to performance profile.
- mark-r 11y agoDo you have a reference that explains this? I'd like to learn more.
- lqdc13 11y agoI don't have a reference, but bytearray concatenation with some more bytes is amortized O(1) w.r.t. original bytearray size, but for bytes it is O(n) where n is the length of the byte string, because bytes are immutable and they are copied to a new byte string on each concatenation. In python2 strings/bytes are also immutable but concatenation is O(1) w.r.t. original string. Not sure about internal implementation differences. so basically whenever you see code like: some_bstr = b'' for something in get_something(): some_bstr += something And if you have to do it this way, make sure that some_bstr is a bytearray or make it a list that you later join.