3 ms·

You are correct that there's no real benefit in UTF-32 over UTF-8, which is why Go and Rust (and others) have worked with UTF-8 in memory just fine. However, t

You are correct that there's no real benefit in UTF-32 over UTF-8, which is why Go and Rust (and others) have worked with UTF-8 in memory just fine. However, the actual encoding of a str object is irrelevant, and it's not the point. The whole point of the str/bytes difference in Python is that you make Python keep track of whether you've done the conversion or not. In Python 2, you can be sloppy, and the programs are buggy as a result! You're putting the cart before the horse here in terms of the Python 3 transition. The str/unicode fix was one of the driving factors for Python 3 to exist in the first place, and if you removed it, then what's the point? Again, look at Go or Rust. Both of them have separate types for strings and bytes, even though they have the same representation in memory, and as a result we don't have that kind of bug in our program.

bdarnell 11y ago
> The str/unicode fix was one of the driving factors for Python 3 to exist in the first place, and if you removed it, then what's the point? The reason python 3 exists is that most python 2 code had latent unicode-related bugs that would only manifest when they were exposed to non-ascii data. The backwards-incompatible barrier between str and bytes was the solution the python 3 team chose for this problem; adopting utf-8 as the standard encoding would have been another solution which I claim would have been more backwards-compatible (essentially moving to the go/rust model, which prove that you don't necessarily need separate byte and character string types for correct unicode handling).
- hetman 11y ago
  But not every 8-bit byte string is valid UTF-8 so that could still cause a world of pain.