5 ms·
UTF-8 is the current default.
by heinrich5991 7y ago
UTF-8 is the current default.
- the_mitsuhiko 7y agoIt uses unicode but it doesn’t use UTF-8 by default as external encoding and never as internal encoding except for an optional secondary buffer.
- jwilk 7y agoWhat do you mean by "external encoding", "internal encoding", "secondary buffer"?
- bjoli 7y agoEncoding strings internally as UTF-8 is a bad idea, since you can't do efficient constant time access (utf-8 isn't fixed width).
- heinrich5991 7y agoEncoding strings internally as UTF-8 is a fine idea, usually you don't need constant-time access to individual code points. E.g. PyPy (faster Python, with JIT) does so. Many other languages also save strings as UTF-8.
- bjoli 7y agoAnd then you have to either re-train them to use string cursors or you have lots of people writing inefficient string code.
- heinrich5991 7y agoNot really, no. PyPy demonstrates that.
- bjoli 7y agoSure it does. Where CPython does byte inflation to make sure random access works, PyPy makes you use an extra indexing structure for cases where you can't just advance one char at a time. I must say it is a rather elegant way, but it still fits firmly within "a new way or inefficient code".
- the_mitsuhiko 7y agoExternal encoding: what it reads and writes from and to files Internal encoding: what the strings are encoded in memory (latin1/UCS2/UCS4 in python) Secondary buffer: python strings can be encoded twice. They can hold a secondary encoded version if themselves as utf-8.