6 ms·

Nice work...but this is not a perfect database of countries. * Territories such as Réunion are listed as countries * Countries that ceased to exist are not li

Nice work...but this is not a perfect database of countries. * Territories such as Réunion are listed as countries * Countries that ceased to exist are not listed (Yugoslavia, Soviet Union) although the world's most recent new country (South Sudan) is listed. * Kosovo is listed, although not recognised by all the world. Transnistria is not listed, even though it has de facto sovereignty. * Total land area is a disputable amount in several circumstances. There are several more such examples I could list. Yes I'm being pedantic. But I have a special interest in this area, and I'm aware of just how hard it is. People are born in one country (Soviet Union) and die in another (Ukraine), even though they never moved from their home city (Odessa).

mjn 14y ago
This is a problem even in professional data sources. I recently did an admittedly somewhat silly analysis, to see what happens if you take the "voting with your feet" metaphor for emigration seriously, and interpret each country's emigrants as a ranked-ballot "vote" for most appealing destination country, with all the ballots tallied using a Condorcet ranking. Results: http://www.kmjn.org/notes/voting_with_feet.html http://www.kmjn.org/notes/voting_with_feet.html I used bilateral migration data from the World Bank to do it: http://data.worldbank.org/data-catalog/global-bilateral-migration-database http://data.worldbank.org/data-catalog/global-bilateral-migr... I thought this might produce complexities I'd have to deal with, since over the 50 year period this data set covers, countries have come and gone a number of times. But to my surprise, the data set somehow uses only current countries, even when giving data points for 1960 or 1970. It doesn't really explain how that happened, either. Did they somehow impute past data to current countries? What does the 1970 figure for Germany, or Serbia, actually mean? Something was too complex and didn't fit the database, so it got swept under the rug? Made my job writing this off-the-cuff blog post much easier, but now I'm left wondering what the data I used actually means, despite it coming from a highly respected source, and used in many academic papers. edit: Found a paper describing the construction of the World Bank data. It looks like they use what they describe as an admittedly crude method of attributing based on population shares, so e.g. immigrants to "Yugoslavia" are apportioned to Croatia/Serbia/etc. based on population share, assuming the immigration was uniform throughout. Not horrible as a first approximation, probably, though clearly not "right" either. Source: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=969242 http://papers.ssrn.com/sol3/papers.cfm?abstract_id=969242
- aes256 14y ago
  > Not horrible as a first approximation, probably, though clearly not "right" either. You really do have to leave any hope of perfectionism at the door when it comes to migration data. Even if record-keeping were consistent, a considerable amount of migration goes unrecorded — mass migration due to environmental disasters or conflict, illegal immigration, etc. There are very few countries that can claim to know the exact number of people within their borders at any given time, let alone where they came from.
vhf 14y ago
People are born in one country (Soviet Union) and die in another (Ukraine), even though they never moved from their home city (Odessa). People are even born in one city (Leningrad) in one country (Soviet Union) and die in another city (St-Petersburg) in another country (Russia), although they never moved.
- deleted 14y ago
  [deleted]
- talaketu 14y ago
  "Ever-newer waters flow on those who step into the same rivers."
- webjunkie 14y ago
  Wasn't the country Russia still Russia, just within the Union?
  - guard-of-terra 14y ago
    It was Russian Soviet Federal Socialist Republic, and now it is Russian Federation. You probably still could go with "Russia" every time.
zalew 14y ago
> Transnistria is not listed, even though it has de facto sovereignty. if by 'de facto sovereignty' you mean not being recognized by any country aka their passports being worthless. a database should not make political statements. if they earn recognition, they'll be listed. there are dozens of separatist territories in the world who claim sovereignty. if Transnistria were to be listed as a country - why not Karabakh? not to mention autonomous regions like Iraqi Kurdistan.
- gurkendoktor 14y ago
  Is it even possible to build a database of countries without making political statements? Taiwan is one example where you can either use the ISO database and piss users off (and they will perceive it as a political statement), or you can make a change to the ISO database and make a political statement the other way around.
  - deleted 14y ago
    [deleted]
  - stevoski 14y ago
    Indeed, a database of countries is politically tough. IIRC, the Syrian government banned Facebook from Syria, because Golan Heights (occupied and annexed by Israel, but not recognised by most (or any?) other countries) was listed in Facebook as being in Israel.
    - deleted 14y ago
      [deleted]
- guard-of-terra 14y ago
  Well, Abkhasia is partially recognized (by perhaps a half dozen states), but it is not listed. Taiwan is partially recognized too (by a comparable number of states), but it is listed. There should perhaps be a "disputed" flag - raise, you got all the territories, clean - only solid recognized states.
- 14y ago
karaoke77 14y ago
For those who are pedantic when it comes to countries, the OxJS library comes with a useful database. See https://oxjs.org/#doc/Ox.COUNTRIES https://oxjs.org/#doc/Ox.COUNTRIES -- or open the console and type: Ox.load('Geo', function() { Ox.print(Ox.COUNTRIES); }) Being the author of said library, I can confirm that compiling and maintaining a list of countries is hard, and will never be 100% "objective".
h2s 14y ago
> Countries that ceased to exist are not listed Where I work, we have a very thoroughly compiled list of countries for our user profile "Country" dropdown. It never ceases to amuse me that people signing up for our services are given choices such as: - Neutral Zone - Union of Soviet Socialist Republics - Unknown or Invalid Region I think it's from the framework we use or something. I suppose it is somewhat reassuring that we're prepared if the USSR ever re-emerges or another Gulf War breaks out. We have a similarly thorough list of possible languages for another dropdown which is used to indicate the language of the things our users create. If SignWriting ever gets absorbed into Unicode, we'll be prepared with our "Sign Language" dropdown option.
dasil003 14y ago
In our case we had to simply use two different country tables. One for geolocation and film rights that matches our CDN's database, and one for historical film data such as the production country. This is one case where I think DRY would be worse because the special cases that apply to one do not apply to the other semantically, and therefore the logic would get very subtle and muddy.

mjn 14y ago

This is a problem even in professional data sources. I recently did an admittedly somewhat silly analysis, to see what happens if you take the "voting with your feet" metaphor for emigration seriously, and interpret each country's emigrants as a ranked-ballot "vote" for most appealing destination country, with all the ballots tallied using a Condorcet ranking. Results: http://www.kmjn.org/notes/voting_with_feet.html http://www.kmjn.org/notes/voting_with_feet.html I used bilateral migration data from the World Bank to do it: http://data.worldbank.org/data-catalog/global-bilateral-migration-database http://data.worldbank.org/data-catalog/global-bilateral-migr... I thought this might produce complexities I'd have to deal with, since over the 50 year period this data set covers, countries have come and gone a number of times. But to my surprise, the data set somehow uses only current countries, even when giving data points for 1960 or 1970. It doesn't really explain how that happened, either. Did they somehow impute past data to current countries? What does the 1970 figure for Germany, or Serbia, actually mean? Something was too complex and didn't fit the database, so it got swept under the rug? Made my job writing this off-the-cuff blog post much easier, but now I'm left wondering what the data I used actually means, despite it coming from a highly respected source, and used in many academic papers. edit: Found a paper describing the construction of the World Bank data. It looks like they use what they describe as an admittedly crude method of attributing based on population shares, so e.g. immigrants to "Yugoslavia" are apportioned to Croatia/Serbia/etc. based on population share, assuming the immigration was uniform throughout. Not horrible as a first approximation, probably, though clearly not "right" either. Source: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=969242 http://papers.ssrn.com/sol3/papers.cfm?abstract_id=969242

aes256 14y ago
> Not horrible as a first approximation, probably, though clearly not "right" either. You really do have to leave any hope of perfectionism at the door when it comes to migration data. Even if record-keeping were consistent, a considerable amount of migration goes unrecorded — mass migration due to environmental disasters or conflict, illegal immigration, etc. There are very few countries that can claim to know the exact number of people within their borders at any given time, let alone where they came from.

vhf 14y ago

People are born in one country (Soviet Union) and die in another (Ukraine), even though they never moved from their home city (Odessa). People are even born in one city (Leningrad) in one country (Soviet Union) and die in another city (St-Petersburg) in another country (Russia), although they never moved.

deleted 14y ago
[deleted]
talaketu 14y ago
"Ever-newer waters flow on those who step into the same rivers."
webjunkie 14y ago
Wasn't the country Russia still Russia, just within the Union?
- guard-of-terra 14y ago
  It was Russian Soviet Federal Socialist Republic, and now it is Russian Federation. You probably still could go with "Russia" every time.

zalew 14y ago

> Transnistria is not listed, even though it has de facto sovereignty. if by 'de facto sovereignty' you mean not being recognized by any country aka their passports being worthless. a database should not make political statements. if they earn recognition, they'll be listed. there are dozens of separatist territories in the world who claim sovereignty. if Transnistria were to be listed as a country - why not Karabakh? not to mention autonomous regions like Iraqi Kurdistan.

gurkendoktor 14y ago
Is it even possible to build a database of countries without making political statements? Taiwan is one example where you can either use the ISO database and piss users off (and they will perceive it as a political statement), or you can make a change to the ISO database and make a political statement the other way around.
- deleted 14y ago
  [deleted]
- stevoski 14y ago
  Indeed, a database of countries is politically tough. IIRC, the Syrian government banned Facebook from Syria, because Golan Heights (occupied and annexed by Israel, but not recognised by most (or any?) other countries) was listed in Facebook as being in Israel.
  - deleted 14y ago
    [deleted]
guard-of-terra 14y ago
Well, Abkhasia is partially recognized (by perhaps a half dozen states), but it is not listed. Taiwan is partially recognized too (by a comparable number of states), but it is listed. There should perhaps be a "disputed" flag - raise, you got all the territories, clean - only solid recognized states.
14y ago

karaoke77 14y ago

For those who are pedantic when it comes to countries, the OxJS library comes with a useful database. See https://oxjs.org/#doc/Ox.COUNTRIES https://oxjs.org/#doc/Ox.COUNTRIES -- or open the console and type: Ox.load('Geo', function() { Ox.print(Ox.COUNTRIES); }) Being the author of said library, I can confirm that compiling and maintaining a list of countries is hard, and will never be 100% "objective".

h2s 14y ago

> Countries that ceased to exist are not listed Where I work, we have a very thoroughly compiled list of countries for our user profile "Country" dropdown. It never ceases to amuse me that people signing up for our services are given choices such as: - Neutral Zone - Union of Soviet Socialist Republics - Unknown or Invalid Region I think it's from the framework we use or something. I suppose it is somewhat reassuring that we're prepared if the USSR ever re-emerges or another Gulf War breaks out. We have a similarly thorough list of possible languages for another dropdown which is used to indicate the language of the things our users create. If SignWriting ever gets absorbed into Unicode, we'll be prepared with our "Sign Language" dropdown option.

dasil003 14y ago

In our case we had to simply use two different country tables. One for geolocation and film rights that matches our CDN's database, and one for historical film data such as the production country. This is one case where I think DRY would be worse because the special cases that apply to one do not apply to the other semantically, and therefore the logic would get very subtle and muddy.