3 ms·
PS: found some example code in the github project which seems to take care of the special Windows situation: https://github.com/soasis/text/blob/main/examples/
by flohofwoe 5y ago
PS: found some example code in the github project which seems to take care of the special Windows situation:
https://github.com/soasis/text/blob/main/examples/documentation/source/runtime_locale_encoding.cpp https://github.com/soasis/text/blob/main/examples/documentat...
Unfortunately this doesn't look so simple anymore.
- nly 5y agoThis code looks like a disaster to me. The issue on Windows is simply that code pages are a per-application setting (or state) and not a system setting. Programs are free to run with, and change between, any "active code page" they want and spit bytes out in that code page. This is all fine and dandy when you're talking to a -A Windows API, since Windows knows the ACP of your application, but it's disaster for I/O and making arbitrary programs talk to one another. Nothing you can do in your code can fix this, it's a contract you have to have with the outside world. On Linux calling setlocale() with a non-UTF-8 locale might not even even work because your /etc/locale.gen file will probably only have a couple of enabled entries, and doing so would be mostly pointless because your terminal and all your other programs are likely using UTF-8. The downside to the Linux approach is that if somebody does actually send you a ISO-8859-1 encoded file you will have to do some conversion. #include <boost/locale.hpp> #include <fstream> #include <iostream> #include <string> int main(int argc, char** argv) { std::ifstream ifs ("test.txt"); std::string str; boost::locale::generator gen; // No dependency on what's in /etc/locale.gen auto loc = gen ("en_US.ISO-8859-1"); ifs.imbue (loc); // Should make formatting input functions work std::getline (ifs, str); str = boost::locale::conv::to_utf<char> (str, loc); for (auto wc: str) { std::cout << sizeof(wc) << ": " << static_cast<uint64_t>(static_cast<unsigned char>(wc)) << "\n"; } }