3 ms·

In the reply below "str" and "unicode" refer to the python types of those names, not to generic "string" and "unicode". No, in this case it was broken because

by anon4 11y ago

In the reply below "str" and "unicode" refer to the python types of those names, not to generic "string" and "unicode". No, in this case it was broken because things suddenly started working in weird ways, because type conversions between str and unicode were taking place when you weren't expecting it. So it's "broken practically" and not "broken philosophically". You could call X.encode('utf-8') expecting to get the utf-8 byte representation of a string, only to be told "character \xf8 at position whatever cannot be handled by ASCII decoder", because X is a str containing weird bytes and not an unicode, so when you ask to have it encoded, it is first silently decoded to unicode using the default encoding - ASCII in this case. You could do X + Y and get the same error because one of them happens to be an unicode, but the other is a str which python tries to convert to unicode.

masklinn 11y ago
Your comment has very limited relation to the GP in that the issue you outline is not the implicit conversion itself but the fact that said implicit conversion may fault and it is nigh impossible to know where that could happen until it does. For the record I would agree, but I'm guessing it made sense in the context surrounding the introduction of "proper" unicode objects in Python 2 considering all the issues which cropped up after the Python 3 split and the issue of correctly dispatching APIs between str and unicode. Furthermore some of the APIs are decidedly idiotic: `unicode` can take a string and an encoding, one could expect that if an encoding is provided it'll try to decode str inputs but let unicode go through unmolested, and thus could be a convenient way to ensure a possibly-str input is converted to unicode. After all, forced conversion is already available using e.g. `str(obj).decode(encoding)`. Well, >>> unicode(u"straße", 'utf-8') Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: decoding Unicode is not supported

by anon4 11y ago

masklinn 11y ago

Your comment has very limited relation to the GP in that the issue you outline is not the implicit conversion itself but the fact that said implicit conversion may fault and it is nigh impossible to know where that could happen until it does. For the record I would agree, but I'm guessing it made sense in the context surrounding the introduction of "proper" unicode objects in Python 2 considering all the issues which cropped up after the Python 3 split and the issue of correctly dispatching APIs between str and unicode. Furthermore some of the APIs are decidedly idiotic: `unicode` can take a string and an encoding, one could expect that if an encoding is provided it'll try to decode str inputs but let unicode go through unmolested, and thus could be a convenient way to ensure a possibly-str input is converted to unicode. After all, forced conversion is already available using e.g. `str(obj).decode(encoding)`. Well, >>> unicode(u"straße", 'utf-8') Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: decoding Unicode is not supported