I work as a research analyst for a Canadian provincial opposition party. Most government data is in terrible HTML tables, often dynamically generated, and almos

I work as a research analyst for a Canadian provincial opposition party. Most government data is in terrible HTML tables, often dynamically generated, and almost none of it is in an easily machine readable format. I spend a lot of time downloading PDF files of data and converting them to JSON formats. I have two main recurring scrapes: - political donations. Every donation to a political party in my province above ~$300 is posted publicly on a gov't website (in a PDF). I use the data to run machine learning algorithms to predict who is most likely to want to donate to my party. - public service expenses. My province has a "sunshine list" which publishes the salaries and contracts for all senior government officials. We grab it weekly (as once someone quits the gov't, their data disappears). One tool that you could consider building is an easily accessible expense website, where people can enter the name of a public official and see all their expenses, including a summary of the total amount spent. There have been a number of massive expenses here in Canada related to this [1, 2]. [1] http://news.nationalpost.com/tag/alison-redford/ http://news.nationalpost.com/tag/alison-redford/ [2] http://en.wikipedia.org/wiki/Canadian_Senate_expenses_scandal http://en.wikipedia.org/wiki/Canadian_Senate_expenses_scanda...