3 ms·
> UTF-8/UTF-16 are encodings, used to encode strings for transport/exchange. Internally, strings are stored in UCS-2 or UCS-4 representation. That makes no sen
by gsnedders 10y ago
> UTF-8/UTF-16 are encodings, used to encode strings for transport/exchange. Internally, strings are stored in UCS-2 or UCS-4 representation.
That makes no sense; UCS-2 and UCS-4 are encodings too.
- guitarbill 10y agoIt isn't completely false. It's a simplification (as UCS-* is often used to denote internal encodings), because oh god, Unicode. This does a great job at explaining some intricacies: http://lucumr.pocoo.org/2014/1/9/ucs-vs-utf8/ http://lucumr.pocoo.org/2014/1/9/ucs-vs-utf8/ Also, in Python 3, strings can be UCS-1 (latin1?), UCS-2 (UTF-16?), UCS-4 (UTF-32?) or other: https://www.python.org/dev/peps/pep-0393/ https://www.python.org/dev/peps/pep-0393/ It's complicated. Really complicated. Sorry.
- gsnedders 10y agoI'd lean against referring to them as any Unicode-based terminology, as all of that is ultimately designed to represent all scalar values (e.g., even UTF-16 code units compose in such a way as to give some way to represent scalar values outside of the BMP, though they obviously cannot directly). The other notable thing about Python 3.3+'s representation is their ability to represent surrogates, which UTFs cannot. Realistically, simply referring to it as "flexibly sized codepoint sequences" is probably about as good as we can get, IMO, because that's what they fundamentally are. (And there's very little terminology for a sequence of codepoints!)