5 ms·

EveryPolitician: Open dataset on politicians

270 points by tfgg 10y ago

eriknstr 10y ago
I was happy to see that they include politicians outside of the US. Unfortunately there is no data about any of the politicians in my country yet.
- mrtksn 10y ago
  It's crowdsourced, it will have more info as it gains popularity
  - terravion 10y ago
    Is this really a good idea? Open meetings laws, pre-vote publication, and a variety of transparency measures have often backfired and gotten us worse, more special interest controlled legislation.
    - george_ciobanu 10y ago
      Can you provide examples? Ideally of both positive and negative outcomes.
      - Taek 10y ago
        Sure, an easy example is public voting. When congressional votes are public, it is very easy for special interest groups to reward or favor politicians who vote correctly. It's easier to buy votes. Otoh lots more information available to the public often also makes it easier to detect lying and corruption.
        krick 10y ago
        It is a valid concern, but I think as usual it is more about if this data in possibly available, not if there exists such a service as this. Because if it is, then there surely is somebody, who has this information, as thus possesses power that you (and me) do not possess. In that case I'd argue that public availability is somewhat better. In the end, you do know that somebody is buying votes, it is just that you cannot do that. It is very much possible that they already have closed back-office system like this. It concerns me more that "politician" is a very ambiguous word. Basically everybody active enough is a politician, even though he may or may not be in senate, belong to some party and such. And an open dataset on "pretty much everybody" seems a bit more questionable than on those we consider "being in charge".
        notahacker 10y ago
        I agree that readily available to the average person tends to be better than possibly available In practice, aggregation services like this tend to have more benign results like enabling the bulk emailing of politicians. Others include the suspicion that the popularity of Theyworkforyou, a UK site run by the same people which parses open data on parliamentary speeches and votes, might have had the effect of encouraging MPs to make brief, inconsequential speeches in order to boost statistics on their relative activity in Parliament. Another area in which the effect of their data aggregation might be considered questionable is where it's used as an authoritative source that a particular MP is "strongly against" a particular "cause", when in actual fact they may simply have voted with the party line against a particular bill related to that cause purely due to concerns about one specific aspect of that bill.
        everypolitician 10y ago
        http://www.spectator.co.uk/2013/09/parliament-has-finally-woken-up-because-voters-are-keeping-their-mps-in-line/ http://www.spectator.co.uk/2013/09/parliament-has-finally-wo... has a good take on the impact that these sorts of projects can cause ("Today’s MPs are no longer scared of the whips. Instead, they are scared of their constituents.")
- haldora 10y ago
  You can help make that happen. :D http://docs.everypolitician.org/contribute.html http://docs.everypolitician.org/contribute.html
thomasfoster96 10y ago
233 countries? It would be much better to organise the data in some sort of heirarchy, given having the UK, Wales and Scotland all on the list is somewhat confusing (and it leaves out US and Australian state legislatures). Also, I'm wondering how the data was collected - the party affiliation information for the Australian parliament is very strange. Not entirely wrong, but probably misleading.
- everypolitician 10y ago
  > the party affiliation information for the Australian parliament is very strange. Not entirely wrong, but probably misleading. For Australia we get our information primary from http://www.openaustralia.org.au/ http://www.openaustralia.org.au/ — if you can give any pointers to where things are wrong, we can correct them.
  - Haydos585x2 10y ago
    And if you want more information about Members and Senators for Australia you can get a CSV with way more data over at: http://www.aph.gov.au/senators_and_members http://www.aph.gov.au/senators_and_members under "Contacting Senators and Members". I used that dataset to build a very incomplete project called http://reviewmy.mp http://reviewmy.mp as a way to learn Phoenix/Elixir. The data is missing a bit but the search results[0] have everything else and it's scrapable! 0: http://www.aph.gov.au/Senators_and_Members/Parliamentarian_Search_Results?expand=1&q=&mem=1&par=-1&gen=0&ps=12 http://www.aph.gov.au/Senators_and_Members/Parliamentarian_S...
  - thomasfoster96 10y ago
    For starters the speaker and deputy speaker of each chamber are listed as being not being in their own parties, which isn't true (although it is in the U.K.), instead they have strange abbreviations for the name of the position. Secondly, in Australia the Liberal National Party is a partial merger of the Liberal and National parties. MPs in the LNP can be a member of either the Liberals or Nationals. In the House of Representatives data for Australia, the breakdown of LNP MPs between the two parties is incorrect, and leaves one MP as a sole member of the LNP. The errors are similar in the Senate data. Additionally, two politicians (Bob Katter and Jacqui Lambie) are wrongly shown as being independents, despite being elected as the sole member of a party.
JumpCrisscross 10y ago
> mySociety Limited is a project of UK Citizens Online Democracy, a registered charity in England and Wales Contributing data [1] on powerful people carries risks. These risks depend on whose information you are sharing, how you got it and your country's strength of rule of law. Britain has very broad dragnet surveillance laws on its books [2]. If you are going to contribute, please consider the INFOSEC and OPSEC ramifications of those laws. [1] http://docs.everypolitician.org/contribute.html http://docs.everypolitician.org/contribute.html [2] https://www.theguardian.com/uk-news/2016/mar/01/snoopers-charter-to-extend-police-access-to-phone-and-internet-data https://www.theguardian.com/uk-news/2016/mar/01/snoopers-cha...
shakethemonkey 10y ago
Political Graveyard[1] is the world's richest open dataset on American politicians, and its size dwarfs everypolitician.org [1] http://www.politicalgraveyard.com/ http://www.politicalgraveyard.com/ [Disclaimer: I have no affiliation with the site, not even sure who runs it]
- freyir 10y ago
  > not even sure who runs it Lawrence Kestenbaum, attorney, politician, and the creator and webmaster of The Political Graveyard website.
- knowaveragejoe 10y ago
  I just wanna say, they could definitely use with a facelift.
  - unimpressive 10y ago
    Site loads fast, has information, not a UI trainwreck. Good enough for me.
    - Diederich 10y ago
      Better than good enough: it's good. It's also preferable to something more SPAish. To be clear: I've professionally written such javascript only Single Page Applications, and they worked out really well, since they were all dynamic, real-time changing content all the way.
    - knowaveragejoe 10y ago
      It doesn't need to be cutting edge or anything close to it - just less link soup. I'd hate to use this on mobile, on top of that.
- tfgg 10y ago
  That's really amazing. In terms of utility to effort ratio, I think it's right to start with national legislatures. However, I hope there will eventually be a dataset that's as deep as Political Graveyard for the entire planet, even if just to satisfy my inner data nerd :)
foxfired 10y ago
233 countries, 0 from my country.
- krick 10y ago
  Well, contribute.
  - everypolitician 10y ago
    Even letting us know where to _find_ data for your country would be a big help. For the few countries we're missing — http://everypolitician.org/needed.html http://everypolitician.org/needed.html — it's because we haven't been able to find any source of who the legislators are there.
Goldenromeo 10y ago
Have you considered allowing email signup for the gender balance game feature?
- newscracker 10y ago
  I attempted the trial, and when faced with a login requirement (with Twitter, Facebook, Google, LinkedIn and Github), I just left the site. Even an email signup wouldn't be fine for me. I don't want any achievements to brag about. I just wanted to learn and/or contribute.
nodesocket 10y ago
I quickly scanned the title and thought it would include each politician's net worth. Strange use of "richest", I would have personally gone with "largest" but...
devoply 10y ago
politicians are mere servants. need a better open data set on the people and corps they serve.
- Steeeve 10y ago
  Have a look at the sunlight foundation apis. I know they have one that spells out donations. (it was US based the last I worked with their datasets in detail, but the goal was to grow and it's been several years).
id122015 10y ago
Thank you for such a useful project, its a good start. Ill happily contribute with data sources, also I can translate the website in other languages if requested. And most importantly for those who live in countries with huge tax rates, next time when Ill protect my hard earned money that they try to steal as tax and inflation, Ill use the feature to donate it to this website.
- id122015 10y ago
  Also the word "every" can only be undestood by speakers of the english language. If its not too late you can try to buy a domain such as: politician.global
- maaaats 10y ago
  You may not like taxes, but calling it theft always belittles the discussion.
Fifer82 10y ago
I can't believe my tax pays for so many useless cretins and the UK is still on its knees.
DasIch 10y ago
They don't even have current data on who is currently in parliament for many countries and in the case they do that data is essentially worthless. I really don't see the point of this. If you want to bother at all, you should have data on the level of http://abgeordetenwatch.de http://abgeordetenwatch.de (for Germany only but surely similar projects exist in other countries). So how they voted, which committees are they part of, which jobs (beside being a politician) do they have. If you can get it, even which lobbyists they've met with (http://ec.europa.eu/transparencyregister/ http://ec.europa.eu/transparencyregister/).
- everypolitician 10y ago
  > They don't even have current data on who is currently in parliament for many countries Unfortunately, in the vast majority of countries, this data is largely only available by scraping — and often parliaments completely revamp their websites when a new term starts, meaning our scrapers need rewritten then. We're aware we're behind in some countries, but we're up to date in most, and help with bringing us forward on the others is always appreciated! (This is an unfunded project run by a tiny team within a charity). Or, when people let us know that they're using the data for a country (or would use it if it were up to date!), then we prioritise working on that. It's also possible that we simply haven't noticed that there's a new term somewhere. There's an average of one general election per week throughout the world, and sometimes we miss one, especially if there's a long gap between the election and the term starting, or even the list of legislators being published. So please let us know where we're missing something. > If you want to bother at all, you should have data on the level of http://abgeordetenwatch.de http://abgeordetenwatch.de Yes, that's the goal. But it's going to take us time to get there! We also run https://www.theyworkforyou.com/ https://www.theyworkforyou.com/ in the UK, so we know the value of having that level of information available. Our experience in helping groups in other countries set up similar sites led us to create EveryPolitician. Its goal is not to be a replacement for sites like that, which will always need local knowledge and context to be effective — it's to help people building such sites get up and running quicker, and free them up to focus more on holding politicians to account, not in spending all their time fixing broken scrapers etc. And, by transforming all the underlying data to a consistent format, there can also hopefully be significantly more tool reuse, rather than everyone reinventing the wheel every time. This whole sector is massively underfunded, and so much time, energy, and money is wasted simply replicating what exists in other countries. There'll never be a one-size-fits-all solution for most of this, but being able to get started quicker, and to re-use pieces that already exist (e.g. for letting people write to their representatives, or visualise gender-breakdown over time, or compare attendance records across parties, etc) means that groups can deliver more value within their usually very tight budgets. We prioritised going broad rather than deep to start with, because there's value in being able to do even very shallow comparisons across multiple countries — even just having the _names_ of the current national legislators in almost every country in the world turns out to be quite useful if, say, you're an investigative journalist needing to filter millions of documents (think Wikileaks, Panama Papers, etc) — and to give people something to build on top of while we work on going much deeper. That's slowly starting to happen now, but largely prioritised by user need, as we don't have the resources to go deep everywhere at once. But help is always very very welcome! Simply even telling us where we can get data for most countries is hugely valuable, though for the 95% of countries that don't supply that in structured formats, helping to write scrapers would be wonderful too…
nailer 10y ago
From the site's UI: > Find representatives from your country: Expected to use the box to find representatives. Instead it's a place to enter your country. You should already know my country. Let me type in a representative's name.
- notahacker 10y ago
  I can see the logic of having a default choice, but UX really shouldn't assume its users don't have a passport. Plus the real purpose of the feature is to showcase that it can return lists of politicians by nationality/body for many countries. At individual politician level, I can find a whole lot more information by typing their name into UK-specific sites (some of them actually run by the same people)
donmatito 10y ago
That's an awesome project. Some civic tech initiatives promise to bring transparency on representative activity/lazyness, vote records, or transparency/corruption. This promises to unify datasets in a consistent, comparable manner. Very interesting
- donmatito 10y ago
  And to everyone who says "this dataset is missing", well => http://docs.everypolitician.org/contribute.html http://docs.everypolitician.org/contribute.html
IanCal 10y ago
I have tried reading through the documentation and looked at the github repo, and cannot find a license. Could you please add one? I currently can't work out what I'd be able to use this for, so far I'd be concerned I cannot reuse it in any way at all.
pani00 10y ago
> If you know where to find more data for this country, please let us know. Wikipedia.

270 points by tfgg 10y ago

I was happy to see that they include politicians outside of the US. Unfortunately there is no data about any of the politicians in my country yet.

mrtksn 10y ago
It's crowdsourced, it will have more info as it gains popularity
- terravion 10y ago
  Is this really a good idea? Open meetings laws, pre-vote publication, and a variety of transparency measures have often backfired and gotten us worse, more special interest controlled legislation.
  - george_ciobanu 10y ago
    Can you provide examples? Ideally of both positive and negative outcomes.
    - Taek 10y ago
      Sure, an easy example is public voting. When congressional votes are public, it is very easy for special interest groups to reward or favor politicians who vote correctly. It's easier to buy votes. Otoh lots more information available to the public often also makes it easier to detect lying and corruption.
      - krick 10y ago
        It is a valid concern, but I think as usual it is more about if this data in possibly available, not if there exists such a service as this. Because if it is, then there surely is somebody, who has this information, as thus possesses power that you (and me) do not possess. In that case I'd argue that public availability is somewhat better. In the end, you do know that somebody is buying votes, it is just that you cannot do that. It is very much possible that they already have closed back-office system like this. It concerns me more that "politician" is a very ambiguous word. Basically everybody active enough is a politician, even though he may or may not be in senate, belong to some party and such. And an open dataset on "pretty much everybody" seems a bit more questionable than on those we consider "being in charge".
        notahacker 10y ago
        I agree that readily available to the average person tends to be better than possibly available In practice, aggregation services like this tend to have more benign results like enabling the bulk emailing of politicians. Others include the suspicion that the popularity of Theyworkforyou, a UK site run by the same people which parses open data on parliamentary speeches and votes, might have had the effect of encouraging MPs to make brief, inconsequential speeches in order to boost statistics on their relative activity in Parliament. Another area in which the effect of their data aggregation might be considered questionable is where it's used as an authoritative source that a particular MP is "strongly against" a particular "cause", when in actual fact they may simply have voted with the party line against a particular bill related to that cause purely due to concerns about one specific aspect of that bill.
      - everypolitician 10y ago
        http://www.spectator.co.uk/2013/09/parliament-has-finally-woken-up-because-voters-are-keeping-their-mps-in-line/ http://www.spectator.co.uk/2013/09/parliament-has-finally-wo... has a good take on the impact that these sorts of projects can cause ("Today’s MPs are no longer scared of the whips. Instead, they are scared of their constituents.")
haldora 10y ago
You can help make that happen. :D http://docs.everypolitician.org/contribute.html http://docs.everypolitician.org/contribute.html

thomasfoster96 10y ago

233 countries? It would be much better to organise the data in some sort of heirarchy, given having the UK, Wales and Scotland all on the list is somewhat confusing (and it leaves out US and Australian state legislatures). Also, I'm wondering how the data was collected - the party affiliation information for the Australian parliament is very strange. Not entirely wrong, but probably misleading.

everypolitician 10y ago
> the party affiliation information for the Australian parliament is very strange. Not entirely wrong, but probably misleading. For Australia we get our information primary from http://www.openaustralia.org.au/ http://www.openaustralia.org.au/ — if you can give any pointers to where things are wrong, we can correct them.
- Haydos585x2 10y ago
  And if you want more information about Members and Senators for Australia you can get a CSV with way more data over at: http://www.aph.gov.au/senators_and_members http://www.aph.gov.au/senators_and_members under "Contacting Senators and Members". I used that dataset to build a very incomplete project called http://reviewmy.mp http://reviewmy.mp as a way to learn Phoenix/Elixir. The data is missing a bit but the search results[0] have everything else and it's scrapable! 0: http://www.aph.gov.au/Senators_and_Members/Parliamentarian_Search_Results?expand=1&q=&mem=1&par=-1&gen=0&ps=12 http://www.aph.gov.au/Senators_and_Members/Parliamentarian_S...
- thomasfoster96 10y ago
  For starters the speaker and deputy speaker of each chamber are listed as being not being in their own parties, which isn't true (although it is in the U.K.), instead they have strange abbreviations for the name of the position. Secondly, in Australia the Liberal National Party is a partial merger of the Liberal and National parties. MPs in the LNP can be a member of either the Liberals or Nationals. In the House of Representatives data for Australia, the breakdown of LNP MPs between the two parties is incorrect, and leaves one MP as a sole member of the LNP. The errors are similar in the Senate data. Additionally, two politicians (Bob Katter and Jacqui Lambie) are wrongly shown as being independents, despite being elected as the sole member of a party.

JumpCrisscross 10y ago

> mySociety Limited is a project of UK Citizens Online Democracy, a registered charity in England and Wales Contributing data [1] on powerful people carries risks. These risks depend on whose information you are sharing, how you got it and your country's strength of rule of law. Britain has very broad dragnet surveillance laws on its books [2]. If you are going to contribute, please consider the INFOSEC and OPSEC ramifications of those laws. [1] http://docs.everypolitician.org/contribute.html http://docs.everypolitician.org/contribute.html [2] https://www.theguardian.com/uk-news/2016/mar/01/snoopers-charter-to-extend-police-access-to-phone-and-internet-data https://www.theguardian.com/uk-news/2016/mar/01/snoopers-cha...

shakethemonkey 10y ago

Political Graveyard[1] is the world's richest open dataset on American politicians, and its size dwarfs everypolitician.org [1] http://www.politicalgraveyard.com/ http://www.politicalgraveyard.com/ [Disclaimer: I have no affiliation with the site, not even sure who runs it]

freyir 10y ago
> not even sure who runs it Lawrence Kestenbaum, attorney, politician, and the creator and webmaster of The Political Graveyard website.
knowaveragejoe 10y ago
I just wanna say, they could definitely use with a facelift.
- unimpressive 10y ago
  Site loads fast, has information, not a UI trainwreck. Good enough for me.
  - Diederich 10y ago
    Better than good enough: it's good. It's also preferable to something more SPAish. To be clear: I've professionally written such javascript only Single Page Applications, and they worked out really well, since they were all dynamic, real-time changing content all the way.
  - knowaveragejoe 10y ago
    It doesn't need to be cutting edge or anything close to it - just less link soup. I'd hate to use this on mobile, on top of that.
tfgg 10y ago
That's really amazing. In terms of utility to effort ratio, I think it's right to start with national legislatures. However, I hope there will eventually be a dataset that's as deep as Political Graveyard for the entire planet, even if just to satisfy my inner data nerd :)

foxfired 10y ago

233 countries, 0 from my country.

krick 10y ago
Well, contribute.
- everypolitician 10y ago
  Even letting us know where to _find_ data for your country would be a big help. For the few countries we're missing — http://everypolitician.org/needed.html http://everypolitician.org/needed.html — it's because we haven't been able to find any source of who the legislators are there.

Goldenromeo 10y ago

Have you considered allowing email signup for the gender balance game feature?

newscracker 10y ago
I attempted the trial, and when faced with a login requirement (with Twitter, Facebook, Google, LinkedIn and Github), I just left the site. Even an email signup wouldn't be fine for me. I don't want any achievements to brag about. I just wanted to learn and/or contribute.

nodesocket 10y ago

I quickly scanned the title and thought it would include each politician's net worth. Strange use of "richest", I would have personally gone with "largest" but...

devoply 10y ago

politicians are mere servants. need a better open data set on the people and corps they serve.

Steeeve 10y ago
Have a look at the sunlight foundation apis. I know they have one that spells out donations. (it was US based the last I worked with their datasets in detail, but the goal was to grow and it's been several years).

id122015 10y ago

Thank you for such a useful project, its a good start. Ill happily contribute with data sources, also I can translate the website in other languages if requested. And most importantly for those who live in countries with huge tax rates, next time when Ill protect my hard earned money that they try to steal as tax and inflation, Ill use the feature to donate it to this website.

id122015 10y ago
Also the word "every" can only be undestood by speakers of the english language. If its not too late you can try to buy a domain such as: politician.global
maaaats 10y ago
You may not like taxes, but calling it theft always belittles the discussion.

Fifer82 10y ago

I can't believe my tax pays for so many useless cretins and the UK is still on its knees.

DasIch 10y ago

They don't even have current data on who is currently in parliament for many countries and in the case they do that data is essentially worthless. I really don't see the point of this. If you want to bother at all, you should have data on the level of http://abgeordetenwatch.de http://abgeordetenwatch.de (for Germany only but surely similar projects exist in other countries). So how they voted, which committees are they part of, which jobs (beside being a politician) do they have. If you can get it, even which lobbyists they've met with (http://ec.europa.eu/transparencyregister/ http://ec.europa.eu/transparencyregister/).

everypolitician 10y ago
> They don't even have current data on who is currently in parliament for many countries Unfortunately, in the vast majority of countries, this data is largely only available by scraping — and often parliaments completely revamp their websites when a new term starts, meaning our scrapers need rewritten then. We're aware we're behind in some countries, but we're up to date in most, and help with bringing us forward on the others is always appreciated! (This is an unfunded project run by a tiny team within a charity). Or, when people let us know that they're using the data for a country (or would use it if it were up to date!), then we prioritise working on that. It's also possible that we simply haven't noticed that there's a new term somewhere. There's an average of one general election per week throughout the world, and sometimes we miss one, especially if there's a long gap between the election and the term starting, or even the list of legislators being published. So please let us know where we're missing something. > If you want to bother at all, you should have data on the level of http://abgeordetenwatch.de http://abgeordetenwatch.de Yes, that's the goal. But it's going to take us time to get there! We also run https://www.theyworkforyou.com/ https://www.theyworkforyou.com/ in the UK, so we know the value of having that level of information available. Our experience in helping groups in other countries set up similar sites led us to create EveryPolitician. Its goal is not to be a replacement for sites like that, which will always need local knowledge and context to be effective — it's to help people building such sites get up and running quicker, and free them up to focus more on holding politicians to account, not in spending all their time fixing broken scrapers etc. And, by transforming all the underlying data to a consistent format, there can also hopefully be significantly more tool reuse, rather than everyone reinventing the wheel every time. This whole sector is massively underfunded, and so much time, energy, and money is wasted simply replicating what exists in other countries. There'll never be a one-size-fits-all solution for most of this, but being able to get started quicker, and to re-use pieces that already exist (e.g. for letting people write to their representatives, or visualise gender-breakdown over time, or compare attendance records across parties, etc) means that groups can deliver more value within their usually very tight budgets. We prioritised going broad rather than deep to start with, because there's value in being able to do even very shallow comparisons across multiple countries — even just having the _names_ of the current national legislators in almost every country in the world turns out to be quite useful if, say, you're an investigative journalist needing to filter millions of documents (think Wikileaks, Panama Papers, etc) — and to give people something to build on top of while we work on going much deeper. That's slowly starting to happen now, but largely prioritised by user need, as we don't have the resources to go deep everywhere at once. But help is always very very welcome! Simply even telling us where we can get data for most countries is hugely valuable, though for the 95% of countries that don't supply that in structured formats, helping to write scrapers would be wonderful too…

nailer 10y ago

From the site's UI: > Find representatives from your country: Expected to use the box to find representatives. Instead it's a place to enter your country. You should already know my country. Let me type in a representative's name.

notahacker 10y ago
I can see the logic of having a default choice, but UX really shouldn't assume its users don't have a passport. Plus the real purpose of the feature is to showcase that it can return lists of politicians by nationality/body for many countries. At individual politician level, I can find a whole lot more information by typing their name into UK-specific sites (some of them actually run by the same people)

donmatito 10y ago

That's an awesome project. Some civic tech initiatives promise to bring transparency on representative activity/lazyness, vote records, or transparency/corruption. This promises to unify datasets in a consistent, comparable manner. Very interesting

donmatito 10y ago
And to everyone who says "this dataset is missing", well => http://docs.everypolitician.org/contribute.html http://docs.everypolitician.org/contribute.html

IanCal 10y ago

I have tried reading through the documentation and looked at the github repo, and cannot find a license. Could you please add one? I currently can't work out what I'd be able to use this for, so far I'd be concerned I cannot reuse it in any way at all.

pani00 10y ago

> If you know where to find more data for this country, please let us know. Wikipedia.