5 ms·
Like explained in https://utf8everywhere.org/#windows https://utf8everywhere.org/#windows , you can write simple wrapper functions `narrow`/`widen` used when yo
by ormax3 3y ago
Like explained in https://utf8everywhere.org/#windows https://utf8everywhere.org/#windows , you can write simple wrapper functions `narrow`/`widen` used when you are about to call window api functions.
::SetWindowTextW(widen(someStdString).c_str());
Implementation is straightforward, relying on `WideCharToMultiByte`/`MultiByteToWideChar` to do the conversion:
https://github.com/neacsum/utf8/blob/master/src/utf8.cpp https://github.com/neacsum/utf8/blob/master/src/utf8.cpp
- huhtenberg 3y agoYou are missing OP's point - this still costs you 2 extra calls. If this cost really matters (and practically speaking it never does), then, as the other commenter said, the correct solution is to just use OS-native encoding for all file system paths and names used by the program, hidden behind an abstraction layer if needs be. UTF16 for Windows, UTF8 elsewhere.
- ormax3 3y agoThe above manifesto makes the argument to use UTF-8 *everywhere*, even on windows where the internal representation is not native utf8. The conversion overhead is really negligible: https://utf8everywhere.org/#faq.cvt.perf https://utf8everywhere.org/#faq.cvt.perf (note: the two api calls per conversion is because how those specific functions work, first call to get the size to allocate, second to do the actual conversion, but you can always use another library in the implementation for the utf8<->utf16 conversion that might be more optimized than those windows api functions)
- donatj 3y agoEspecially negligible versus the trip to the file system you are setting up for.
- Tempest1981 3y agoAnd AVX-512 can help: https://lemire.me/blog/2023/09/13/transcoding-unicode-strings-at-crazy-speeds-with-avx-512/ https://lemire.me/blog/2023/09/13/transcoding-unicode-string...
- jiggawatts 3y agoNot all API calls are for filesystem access.
- donatj 3y agoSure, but basically everything having to do with file paths on Windows, the topic here, relates to the file system.
- Someone1234 3y agoI thought the OP's point is there are too many considerations when doing this? Someone is suggesting a way of making it less tedious, and your response is "performance?!" even though in both scenarios you're running the same code and it is likely the compiler in release would remove the intermediary.
- huhtenberg 3y ago> your response is "performance?!" No, that's not what I said.
- ynik 3y ago"2 extra calls" is a weird metric here. Some calls are vastly more expensive than others. Syscalls come with a significant cost, encoding conversion of short strings (esp. filenames) does not. Hiding just the syscalls behind an abstraction layer is vastly simpler than doing that and additionally hiding the string representation, so "UTF-8 everywhere" is IMHO the right solution.
- mkoubaa 3y agoI have good experiences with this approach. It isolates the weirdness of windows and let's the rest of your code do things idiomatically
- eco 3y agoMicrosoft added a setting in Windows 10 to switch the code page over to utf-8 and then in Windows 11 they made it on by default. Individual applications can turn it on for themselves, so they don't need to rely on the system setting being checked. I haven't tried it yet but with that you can just use the -A variants of the winapi with utf-8 strings. No conversion necessary.
- adzm 3y agoDo you have any references about it being enabled by default in Windows 11? I've seen conflicting reports and often seems to vary depending on the system locale whether it gets enabled or disabled by default.
- deleted 3y ago[deleted]