4 ms·
This code looks like a disaster to me. The issue on Windows is simply that code pages are a per-application setting (or state) and not a system setting. Progra
by nly 5y ago
This code looks like a disaster to me.
The issue on Windows is simply that code pages are a per-application setting (or state) and not a system setting. Programs are free to run with, and change between, any "active code page" they want and spit bytes out in that code page.
This is all fine and dandy when you're talking to a -A Windows API, since Windows knows the ACP of your application, but it's disaster for I/O and making arbitrary programs talk to one another.
Nothing you can do in your code can fix this, it's a contract you have to have with the outside world.
On Linux calling setlocale() with a non-UTF-8 locale might not even even work because your /etc/locale.gen file will probably only have a couple of enabled entries, and doing so would be mostly pointless because your terminal and all your other programs are likely using UTF-8.
The downside to the Linux approach is that if somebody does actually send you a ISO-8859-1 encoded file you will have to do some conversion.
#include <boost/locale.hpp>
#include <fstream>
#include <iostream>
#include <string>
int
main(int argc, char** argv) {
std::ifstream ifs ("test.txt");
std::string str;
boost::locale::generator gen; // No dependency on what's in /etc/locale.gen
auto loc = gen ("en_US.ISO-8859-1");
ifs.imbue (loc); // Should make formatting input functions work
std::getline (ifs, str);
str = boost::locale::conv::to_utf<char> (str, loc);
for (auto wc: str) {
std::cout << sizeof(wc) << ": "
<< static_cast<uint64_t>(static_cast<unsigned char>(wc)) << "\n";
}
}