3 ms·
They're all over the place. There are several dozen files, each named after a country. Some contain a single text file that's normal, others have different de
by Natsu 6y ago
They're all over the place. There are several dozen files, each named after a country. Some contain a single text file that's normal, others have different delimiters, encodings and other nonsense. The last record of some of the split files is split across files, etc.
This is not particularly high quality data, I'm sure a lot of people didn't look at the data very much and just ignored a few lines of errors on import or did simple greps as you say because it's a pain in the rear to find editors that even can handle multi-GB files, etc. Even good converters like iconv can bomb out on, I think it was Qatar (which was also little endian UTF-16 with a BOM), etc.
But yes, it's all over the place. I love how one of them decided to number things 1, 2, III, 4, though maybe that was the translator, I dunno, I never learned to read Arabic letters...