3 ms·
It's possible to just force everyone into unicode strings in a way that doesn't cause too much trouble. It's impossible to try to do this gradually via a future
by rzwitserloot 13y ago
It's possible to just force everyone into unicode strings in a way that doesn't cause too much trouble. It's impossible to try to do this gradually via a future import, and it's impossible to work out all the potential scenarios where things would behave differently, and particularly the performance characteristics can change and pretty wildly at that, but it sounds like this still, as painful as it would be, sounds like a better approach than P2->P3.
Basically, all strings would internally be represented as unicode (OR as a byte array + an encoding, that might be a little too ambitious though), and it has 2 APIs that it can be accessed by. non-instance related operations (such as making new strings) similarly has 2 APIs. pre-unicode-switch code gets an API that emulates as best as is possible the behaviour as before, and post-unicode-switch code gets the full unicode API as exists now for unicode strings.
Even if encoding is flat out broken, if you put in a stream of chars and store that as <whatever>, and then later you query this thing, the broken usually ends up undoing itself. Yes, this will fail spectacularly once you try to for example concatenate a string with broken encoding to a string with a different encoding, but presumably places that are international proof have switched to unicode strings long ago, and places that aren't really aware of the importance of anything that isn't in ASCII 32-127 will find all their code to magically 'just work'.
Similar things should be possible for the iterator business. Internally things are iterators, but if ever any P2 code touches them, they turn into a (memory hogging, etc) list. yes, of course, this will flat out break if you attempt to pass an infinite iterator to code that isn't used to it, but these are all transitional pains, and the key point is: As long as you don't fork the pre-big-switch version (beyond security updates), sooner rather than later libraries will fix the bugs, or their community of users will just die out as someone else writes a new one to fill the void.
Painful? yes. very. We've seen this in programming land before. Java5 introduced generics and as a result any interaction with pretty much any library written before it resulted in a cavalcade of unsafe/raw warnings. It sucked. Huge communities stuck to 1.4 (IBM WebSphere notably stayed there for almost half a decade before moving on to 1.5).
But, today? Libraries have upgraded or have been replaced. The chance you still run into pre-generics code is tiny. It took long, it sucked, yadayada, but for all intents and purposes that was java's Python2->Python3, and everyone is on Python3 at this point. In the same time span as P2->P3, roughly speaking, and that seems like it's nowhere near complete.