4 ms·

The author is mistaken about fwrite() being exempt from locale conversions on Windows. Here's a snippet to test this: #include <stdio.h> #include <loca

The author is mistaken about fwrite() being exempt from locale conversions on Windows. Here's a snippet to test this: #include <stdio.h> #include <locale.h> #include <stdlib.h> #include <windows.h> #include <io.h> int main() { if (!setlocale(LC_ALL, "Japanese_Japan.932")) { puts("setlocale failed"); exit(1); } // U+00A7 SECTION SIGN encoded in CP932 char cp932_string[] = "\x81\x98\n"; // this converts the encoding: printf("%s", cp932_string); // this too: fwrite(cp932_string, 1, sizeof(cp932_string) - 1, stdout); // this too: _write(1, cp932_string, sizeof(cp932_string) - 1); // this outputs raw bytes: DWORD bytes_written; WriteFile( GetStdHandle(STD_OUTPUT_HANDLE), cp932_string, sizeof(cp932_string) - 1, &bytes_written, NULL ); return 0; } And this is the output (compiled with Visual C++ 2022 on Windows 10): >chcp Active code page: 852 >.\a.exe § § § üś

eesmith 3y ago
https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/fwrite?view=msvc-170 https://learn.microsoft.com/en-us/cpp/c-runtime-library/refe... says: > When stream is opened in Unicode translation mode—for example, if stream is opened by calling fopen and using a mode parameter that includes ccs=UNICODE, ccs=UTF-16LE, or ccs=UTF-8, or if the mode is changed to a Unicode translation mode by using _setmode and a mode parameter that includes _O_WTEXT, _O_U16TEXT, or _O_U8TEXT—buffer is interpreted as a pointer to an array of wchar_t that contains UTF-16 data. An attempt to write an odd number of bytes in this mode causes a parameter validation error. I presume stdout is open in that mode, but can't verify it. Try adding: _setmode(_fileno(stdout), _O_BINARY); https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/setmode?view=msvc-170 https://learn.microsoft.com/en-us/cpp/c-runtime-library/refe... But you are right, that's not mentioned in the linked-to examples. Does setlocale() have a side-effect of modifying the mode?
- xeeeeeeeeeeenu 3y ago
  I think it isn't documented, but the console is a special case. The default mode of stdout is _O_TEXT. _write(), which is used internally by stdio, performs encoding translation if all of the following are true: - The current locale isn't "C". - The _O_TEXT flag is set. - The fd points to the console. When you redirect the stdout (e.g. with > in a shell), it's no longer considered a console handle, so the translation won't be applied. Personally, I think it was a weird design decision to include encoding and CRLF translation in what Microsoft calls "low-level I/O".
  - jlokier 3y ago
    Windows isn't unusual for the CRLF translation part. Unix based operating systems, such as Linux, MacOS and the BSDs, do something similar. When writing to a terminal device, which can be a console device, the terminal driver performs CRLF conversions and other special character handling depending on the terminal settings. If an application wants to disable these character operations on a terminal device it has to explicitly turn them off, which is a bit like using _O_BINARY on Windows. CRLF conversions aren't done in the application's stdio code, but the effect is similar to a Windows console fd.

eesmith 3y ago

https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/fwrite?view=msvc-170 https://learn.microsoft.com/en-us/cpp/c-runtime-library/refe... says: > When stream is opened in Unicode translation mode—for example, if stream is opened by calling fopen and using a mode parameter that includes ccs=UNICODE, ccs=UTF-16LE, or ccs=UTF-8, or if the mode is changed to a Unicode translation mode by using _setmode and a mode parameter that includes _O_WTEXT, _O_U16TEXT, or _O_U8TEXT—buffer is interpreted as a pointer to an array of wchar_t that contains UTF-16 data. An attempt to write an odd number of bytes in this mode causes a parameter validation error. I presume stdout is open in that mode, but can't verify it. Try adding: _setmode(_fileno(stdout), _O_BINARY); https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/setmode?view=msvc-170 https://learn.microsoft.com/en-us/cpp/c-runtime-library/refe... But you are right, that's not mentioned in the linked-to examples. Does setlocale() have a side-effect of modifying the mode?

xeeeeeeeeeeenu 3y ago
I think it isn't documented, but the console is a special case. The default mode of stdout is _O_TEXT. _write(), which is used internally by stdio, performs encoding translation if all of the following are true: - The current locale isn't "C". - The _O_TEXT flag is set. - The fd points to the console. When you redirect the stdout (e.g. with > in a shell), it's no longer considered a console handle, so the translation won't be applied. Personally, I think it was a weird design decision to include encoding and CRLF translation in what Microsoft calls "low-level I/O".
- jlokier 3y ago
  Windows isn't unusual for the CRLF translation part. Unix based operating systems, such as Linux, MacOS and the BSDs, do something similar. When writing to a terminal device, which can be a console device, the terminal driver performs CRLF conversions and other special character handling depending on the terminal settings. If an application wants to disable these character operations on a terminal device it has to explicitly turn them off, which is a bit like using _O_BINARY on Windows. CRLF conversions aren't done in the application's stdio code, but the effect is similar to a Windows console fd.