5 ms·
The main reason that Python Unicode is hard is because of the disastrously confusingly named "encode" and "decode". It's a constant question which direction you
by forgottenacc57 10y ago
The main reason that Python Unicode is hard is because of the disastrously confusingly named "encode" and "decode". It's a constant question which direction you are going - encoding or decoding, huh, what?
Python 3 Unicode would be MUCH easier if encoding/decoding was more intuitive, memorable and obvious.
Still an unfixed problem. Not even an acknowledged problem.
- icebraining 10y agoHow would you design the API?
- forgottenacc57 10y agoExplicitly, obviously and unambiguously.
- icebraining 10y agoThat's a wishlist, not a design. Do you suppose they tried to make it implicit, non-obvious and ambiguous?
- crdoconnor 10y agoHe's got a point. I'd probably use something like to_bytes and to_str to disambiguate.
- dom0 10y agoencode/decode were likely not changed from Py2 to make porting easier. Otherwise it would likely have been to_bytes and str.from_bytes - much like "int" in Py3 with to_bytes and int.from_bytes.
- baq 10y agoI'd copy directly from Rust.
- deleted 10y ago[deleted]
- icebraining 10y agoSay I get a file and the name of the encoding it's in (like an XML file with a declaration), how I decode it in Rust?
- majewsky 10y agoQt has QString::fromUtf8 which takes a QByteArray, and QString.toUtf8 which gives a QByteArray. (And similar for a few other encodings.) That was always obvious enough for me. http://doc.qt.io/qt-5/qstring.html http://doc.qt.io/qt-5/qstring.html
- icebraining 10y agoLike I asked baq, say I don't know the encoding ahead of time, but get it dynamically (like in the declaration of an XML file). How would I decode the string?
- protomikron 10y agoI would try utf-8, and otherwise just panic ("invalid input: string is not utf-8 encoded"). Obviously it depends on the context, but I think parsers should be strict, otherwise ambiguities leak into your system.
- icebraining 10y agoI'm not saying "try multiple encodings till you get something". I'm asking about the API when the encoding is not static, but provided at runtime, like in XML files, eg: <?xml version="1.0" encoding="ISO-8859-1"?>
- majewsky 10y agoYou would have the QByteArray first (e.g. read from a file or socket), parse the <?xml?>, thus find the encoding, convert the rest to QString with that, and parse the rest from the QString.
- icebraining 10y agoMy question is how does one "convert the rest to QString with that", in terms of actual code?
- minitech 10y ago`encode` converts text to an encoded form. `decode` converts an encoded form to text. This was confusing in Python 2 because you could encode bytes into… more bytes. What? Python 3 only lets you encode text and decode bytes, as it should, and removes the confusion.
- forgottenacc57 10y agoYeah that explanation really ironed out any confusion.
- minitech 10y agoI think you’d probably get used to it after using it a couple of times/being familiar with what the types represent. Text is a sequence of characters, but you can encode it into a sequence of bytes when necessary. Give it a try! Also, it’s probably worth mentioning that you rarely need to use `encode` or `decode` in Python 3. Most text can just stay text. If you make a mistake, the stronger types are there to help you notice it.
- tyilo 10y agoIn python 3 you can use: b1 = b'bytes' s = str(b1, 'utf-8') b2 = bytes(s, 'utf-8') instead of using encode and decode.
- forgottenacc57 10y agoThere should be one obvious way to do it.