3 ms·

I think the issue comes from PHP magic with non-ASCII values. PHP tries to guess and never fails. Python instead doesn't do this for you. Anyway, some non-eng

I think the issue comes from PHP magic with non-ASCII values. PHP tries to guess and never fails. Python instead doesn't do this for you. Anyway, some non-english speaker countries developers feel like programming language developers still have different encodings as second citizens like this was the sixties. I hope this changes some time in the future although 10 years ago I already hoped for today to be that future.

oefrha 7y ago
Let me quote the sys.stdout/sys.stderr encoding rules: > The character encoding is platform-dependent. Non-Windows platforms use the locale encoding (see locale.getpreferredencoding()). > On Windows, UTF-8 is used for the console device. Non-character devices such as disk files and pipes use the system locale encoding (i.e. the ANSI codepage). Non-console character devices such as NUL (i.e. where isatty() returns True) use the value of the console input and output codepages at startup, respectively for stdin and stdout/stderr. This defaults to the system locale encoding if the process is not initially attached to a console. > The special behaviour of the console can be overridden by setting the environment variable PYTHONLEGACYWINDOWSSTDIO before starting Python. In that case, the console codepages are used as for any other character device. > Under all platforms, you can override the character encoding by setting the PYTHONIOENCODING environment variable before starting Python or by using the new -X utf8 command line option and PYTHONUTF8 environment variable. However, for the Windows console, this only applies when PYTHONLEGACYWINDOWSSTDIO is also set. https://docs.python.org/3/library/sys.html#sys.stdout https://docs.python.org/3/library/sys.html#sys.stdout locale.getpreferredencoding doc is vague > Return the encoding used for text data, according to user preferences. User preferences are expressed differently on different systems, and might not be available programmatically on some systems, so this function only returns a guess. https://docs.python.org/3/library/locale.html#locale.getpreferredencoding https://docs.python.org/3/library/locale.html#locale.getpref... But on *nix it's basically based on the second part of your locale. If it's anything.UTF-8 (and your terminal talks UTF-8) there won't be a problem. If you set your locale to C or something.ASCII or whatever you can't really blame Python for trying to encode the output according to what you claim to prefer. I fail to see how you can "guess" the terminal's expected encoding otherwise.

oefrha 7y ago

Let me quote the sys.stdout/sys.stderr encoding rules: > The character encoding is platform-dependent. Non-Windows platforms use the locale encoding (see locale.getpreferredencoding()). > On Windows, UTF-8 is used for the console device. Non-character devices such as disk files and pipes use the system locale encoding (i.e. the ANSI codepage). Non-console character devices such as NUL (i.e. where isatty() returns True) use the value of the console input and output codepages at startup, respectively for stdin and stdout/stderr. This defaults to the system locale encoding if the process is not initially attached to a console. > The special behaviour of the console can be overridden by setting the environment variable PYTHONLEGACYWINDOWSSTDIO before starting Python. In that case, the console codepages are used as for any other character device. > Under all platforms, you can override the character encoding by setting the PYTHONIOENCODING environment variable before starting Python or by using the new -X utf8 command line option and PYTHONUTF8 environment variable. However, for the Windows console, this only applies when PYTHONLEGACYWINDOWSSTDIO is also set. https://docs.python.org/3/library/sys.html#sys.stdout https://docs.python.org/3/library/sys.html#sys.stdout locale.getpreferredencoding doc is vague > Return the encoding used for text data, according to user preferences. User preferences are expressed differently on different systems, and might not be available programmatically on some systems, so this function only returns a guess. https://docs.python.org/3/library/locale.html#locale.getpreferredencoding https://docs.python.org/3/library/locale.html#locale.getpref... But on *nix it's basically based on the second part of your locale. If it's anything.UTF-8 (and your terminal talks UTF-8) there won't be a problem. If you set your locale to C or something.ASCII or whatever you can't really blame Python for trying to encode the output according to what you claim to prefer. I fail to see how you can "guess" the terminal's expected encoding otherwise.