3 ms·

I believe the behavior is correct though. Python uses encoding specified through LANG/LC_* which is the encoding that supposed to be used, and all properly beha

by takeda 7y ago

I believe the behavior is correct though. Python uses encoding specified through LANG/LC_* which is the encoding that supposed to be used, and all properly behaved applications use it. If your application works on binary data, you can use sys.stdin/out/err.buffer to get binary version. Most people will use it for text, so the defaults make sense. Personally I would like if there was no automatic conversion when using files/network/pipes etc. but I guess that would make it more confusing for new users, and would be unnecessary boilerplate for most use cases.

pdonis 7y ago
> Python uses encoding specified through LANG/LC_ Yes, that's the best you can do, but it's still not always correct. I agree that it should be, but "should be" and "is" aren't always the same. > If your application works on binary data, you can use sys.stdin/out/err.buffer to get binary version. Yes, but there are still standard library functions that will use the regular streams, and that might conflict with what your application is doing. There is no way to tell Python as a whole "use binary streams everywhere because they are pipes for this application". > Personally I would like if there was no automatic conversion when using files/network/pipes etc. That would work if (a) Python could always detect that condition (it can't) and (b) the entire standard library adjusted itself accordingly. > I guess that would make it more confusing for new users, and would be unnecessary boilerplate for most use cases. Python 2 worked fine with the standard streams being binary, and applications wrapping them to decode to Unicode when necessary. Python 2.7 even back ported the TextIOWrapper and similar classes to make the wrapping as simple as possible. A similar approach could have been taken in Python 3 (binary streams and a simple wrapper class), but it wasn't.

by takeda 7y ago

pdonis 7y ago

> Python uses encoding specified through LANG/LC_ Yes, that's the best you can do, but it's still not always correct. I agree that it should be, but "should be" and "is" aren't always the same. > If your application works on binary data, you can use sys.stdin/out/err.buffer to get binary version. Yes, but there are still standard library functions that will use the regular streams, and that might conflict with what your application is doing. There is no way to tell Python as a whole "use binary streams everywhere because they are pipes for this application". > Personally I would like if there was no automatic conversion when using files/network/pipes etc. That would work if (a) Python could always detect that condition (it can't) and (b) the entire standard library adjusted itself accordingly. > I guess that would make it more confusing for new users, and would be unnecessary boilerplate for most use cases. Python 2 worked fine with the standard streams being binary, and applications wrapping them to decode to Unicode when necessary. Python 2.7 even back ported the TextIOWrapper and similar classes to make the wrapping as simple as possible. A similar approach could have been taken in Python 3 (binary streams and a simple wrapper class), but it wasn't.